How hard would it be to show the line number where the error occurs when a test fails?
I’m speaking about Python, as I think the ease will vary between tracks. I believe we use Pytest, which does show the line number atleast when I use the CLI.
This feature would be very useful for me to debug in more complicated tasks, especially.
(My apologies if this has been considered previously and rejected, I couldn’t find anything on the forum.)
Hey @safwansamsudeen - can you give me the flags you are using to run pytest, and a few concrete examples of the CLI test output from exercism exercises vs what you are getting through the UI?
Many thanks!
I’ll take a look and see how easy it might be to modify our reporting.
Well, I haven’t tested it using the Exercism CLI, but here’s a case with the Forth exercise (which I manually wrote a test for here). The following output shows that line 54 in code.py called another function, which led to line 62 raising the AttributeError.
>>> pytest _test.py ✔ ============================= test session starts ==============================
platform darwin -- Python 3.11.4, pytest-7.2.0, pluggy-1.0.0
rootdir: /Users/safwan/programming
plugins: anyio-3.7.1
collected 1 item
_test.py F [100%]
=================================== FAILURES ===================================
___________________________________ test_one ___________________________________
def test_one():
> assert evaluate(["1 2 +"]) == [3]
_test.py:5:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
code.py:54: in evaluate
commands[item](stack)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
stack = [1]
def add(stack):
r = stack[-2] + stack[-1]
> stack.pop().pop().append(r)
E AttributeError: 'int' object has no attribute 'pop'
code.py:62: AttributeError
=========================== short test summary info ============================
FAILED _test.py::test_one - AttributeError: 'int' object has no attribute 'pop'
============================== 1 failed in 0.14s ===============================
I’m not sure how much context we should provide - the whole traceback, the whole traceback with code samples, or just the final line/code which raised the error? But we do need more context, I think.
No flags as I see it.