How hard would it be to show the line number where the error occurs when a test fails?
I’m speaking about Python, as I think the ease will vary between tracks. I believe we use Pytest, which does show the line number atleast when I use the CLI.
This feature would be very useful for me to debug in more complicated tasks, especially.
(My apologies if this has been considered previously and rejected, I couldn’t find anything on the forum.)
Hey @safwansamsudeen - can you give me the flags you are using to run pytest, and a few concrete examples of the CLI test output from exercism exercises vs what you are getting through the UI?
Many thanks!
I’ll take a look and see how easy it might be to modify our reporting.
Well, I haven’t tested it using the Exercism CLI, but here’s a case with the Forth exercise (which I manually wrote a test for here). The following output shows that line 54 in code.py called another function, which led to line 62 raising the AttributeError.
>>> pytest _test.py ✔ ============================= test session starts ==============================
platform darwin -- Python 3.11.4, pytest-7.2.0, pluggy-1.0.0
rootdir: /Users/safwan/programming
plugins: anyio-3.7.1
collected 1 item
_test.py F [100%]
=================================== FAILURES ===================================
___________________________________ test_one ___________________________________
def test_one():
> assert evaluate(["1 2 +"]) == [3]
_test.py:5:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
code.py:54: in evaluate
commands[item](stack)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
stack = [1]
def add(stack):
r = stack[-2] + stack[-1]
> stack.pop().pop().append(r)
E AttributeError: 'int' object has no attribute 'pop'
code.py:62: AttributeError
=========================== short test summary info ============================
FAILED _test.py::test_one - AttributeError: 'int' object has no attribute 'pop'
============================== 1 failed in 0.14s ===============================
I’m not sure how much context we should provide - the whole traceback, the whole traceback with code samples, or just the final line/code which raised the error? But we do need more context, I think.
No flags as I see it.
I’m seeing something similar on track javascript.
Line numbers and context are missing with the web editor.
If we start with Hello World:
export function hello () {
return "Hello, World!
}
We get the error sans line number or context:
"" string literal contains an unescaped line break
In a longer exercise, finding where the unclosed string is would not be great.
That is interesting. Because when I do that with Python, I don’t get a test failure comment, I get a stack trace with a line number:
I think part of this is due to the upgrade to Python 3.13. It has better error messages and context, and we return the full stack when code won’t evaluate.
But the case from the OP is something that fails with an error in test, but doesn’t toss a full stack…so that one is a harder replicate/deal with. I will have to play around a bit with things and see if the upgrade in Python as well as the change to a later version of Pytest give us enough info to pass along line numbers.
Unfortunately, an unclosed string is almost always going to trigger at the end of the file, not at the start of the string 
And this is why I love raku as design by empathy is actually a thing:
#! /usr/bin/env raku
say "H
e
l
l
o
;
Error messages which do not give clear guidance are called LTA (Less Than Awesome) and people are encourage to file bugs about them. Here the error message gives both the start (line 3) and end location of the error (line 10):
===SORRY!=== Error while compiling line-number.raku
Unable to parse expression in double quotes; couldn't find final '"' (corresponding starter was at line 3)
at line-number.raku:10
Another example would be when you’ve got a typo and a “did you mean…” sets you on your way again:
#! /usr/bin/env raku
says "Hello, World!";
===SORRY!=== Error while compiling says.raku
Undeclared routine:
says used at line 3. Did you mean 'say'?
If this tickles, come join us on track raku — we’re close to a hundred exercises now.
At least you won’t be left in the lurch wonder what’s up at the EOF (End of File) again, heh !
