Arturo Test Runner upgraded to v2

The Arturo track’s test runner has been upgraded to v2.

Previously, the Exercism editor returned the raw output from running the tests. All the information for which tests passed or failed was technically there but presented as a large block of text.

Now, results are now split out at the individual test level. Each tab has its own pass/fail status, the test code that was run, and an appropriate message if a test fails. This makes the experience noticeably cleaner and is only the start to additional improvements to the test runner.

If you haven’t tried the Arturo track yet, now is a great time: Arturo on Exercism

Feedback on the track — exercises, the test runner, or anything else — is welcome here or on Discord. :slight_smile:

3 Likes

Got an unexpected error on hello-world. The test runner returns an error even though the result is a pass. Screenshot below:

1 Like

Wouldn’t be a major upgrade without issues. I can replicate but only on that exercise, which unfortunately is the most important one.

1 Like

arturo-test-runner/tests/success-if-only-test-passes at main · exercism/arturo-test-runner · GitHub should have covered this at first glance.

Edit - I’ve identified the issue. When comparing two strings, Unitt’s generated output has a different syntax within the assertions. Where I’d expect a [...] block, I instead have a {...} block so I’ll need to PR a fix to the Python code that parses that output.

Edit 2 - There’s another issue where the single quotes in the test name was breaking my parsing of test names to match Unitt’s generated output against the provided test suite. That only affects Hello, World as far as I can tell.

1 Like

bugfix - single quotes in test names and `{` `}` in Unitt output when… by BNAndras · Pull Request #58 · exercism/arturo-test-runner · GitHub should fix this once merged.

1 Like

Alright, that’s been merged. I was able to pass Hello World as well as Reverse String. Thanks for the prompt feedback, @oxe-b.

2 Likes