Currently working on Alphmetics. On my computer the tests run in 2 seconds but the tests always time out when I submit the solution via the CLI. I also saw some test timeouts on some of the easy problems earlier last week.
How much time is allotted to run the tests? The actual tests are taking around 2 seconds, but loading sbt, compiling from scratch and then executing the tests might take 30+ seconds.
The test runner has 20 seconds to run a student’s solution. A few tracks have custom overrides for 30 seconds, but that’s a staff-level decision. Increasing the timeout means fewer student solutions can be tested site-wide at any given time.
The Scala test runner is at GitHub - exercism/scala-test-runner · GitHub if you wanted to take a look, but I’d strongly recommend discussing any potential work with the Scala maintainers first.
Got the scala-test-runner and ran it locally in Docker. That took 11 seconds to run my solution.
Today I went back to my submission, copied that text into the online editor and ran the tests from there. That succeeded. Same code, same tests different results. I’m not sure if the difference was that I ran the tests from the online editor rather than from a CLI submission or just that on a different day the system performance was different.
When a CLI submission’s tests timeout the messages says to try re-running the tests, but I don’t see anything in the UI to re-run the tests for an existing submission. Am I missing something?