Hi! I wonder why Test Runner fails on Matrix exercise (mine submission), probably because of presence of empty file? There is no way to see generic error of “run tests” action, when using Online editor. What’s weird, I don’t see that empty file in the exercise page, just on page when using Online editor. Any hints?
I had the same problem. Copying the contents of MatrixReader.class.st into Matrix.class.st fixed it. The tests are successful in my local pharo image. Maybe the Test Runner is using a different version of the test code?
I see the problem: in the exercise’s config, the solution file is Matrix.class.st, the stub is Matrix.class.st but the test setup uses matrixCalculator := MatrixReader new
I’ll submit a PR
@Bajger Matrix exercise: change solution and example filenames by glennj · Pull Request #577 · exercism/pharo-smalltalk · GitHub
1 Like
@glennj After integrating your PR, fix is there, so solution works for Matrix class. There is just some other thing with test runner: Do you know, how is generic error obtained (is it from resulting json?) I did intentionally typo (on highlighted line) and ran tests. But I don’t see any error from test run.
I don’t think I’ll have time to dig into the Pharo test runner in the near future. @ErikSchierboom ?
Sorry, I wrote that in wrong way - I’m aware about Pharo test runner details (implemented that). I just don’t know, from which resource this UI component takes error message. Is it from result.json or from error output of .sh script of test runner?
The results.json file
Hi Erik! Do you know, at which moment or event is the docker container re-created? I mean, if I did some changes in test runner functionality to cover this case, is online editor taking these changes immediately? The results.json file should contain validation error already, see: Added new example tests for empty class and typo in method by Bajger · Pull Request #20 · exercism/pharo-smalltalk-test-runner · GitHub
The test runner is recreated and deployed once a test runner PR is merged to main.
Ok, thanks for reply! I will test again after merging PR.
One last question! Is Test runner functionality, expects 0 error code from shell script? I’m pretty sure output.json is there, but it ends up with non-zero error code (because error happens prior executing individual tests).
Golden test covering this is here: pharo-smalltalk-test-runner/expected_results.json at main · exercism/pharo-smalltalk-test-runner · GitHub
But I don’t see that error in UI:
Yes, the test runner has to exit non-zero. Actual errors should be in the results.json file
Hi Erik! So if there is some generic error during test runner execution, it should return non-zero exit code and results.json is expected right? I’m asking, since it really works like this right now. Or it should return 0 exit code (success) always, otherwise results.json is not consumed by UI functionality?
Thanks!
That’s correct.