Groovy track tests initially pass, then go to failed

Exercism Support
1

Hi Exercism support team!
I’ve started on the Groovy track but when I submit an iteration the UI shows as processing for a few seconds, goes to passed, then loads again after a few seconds and switches to failed.

I’m submitting the solution via the CLI (exercism version 3.1.0) after successfully testing on my local machine (with @Ignores removed).

If I rerun the test with the same code via the website UI editor it will usually pass the tests and they stay passed. However, the most recent exercise I did (hamming) even timed out a few times in the web UI before finally passing.

Any insight on what might be going on or what I might be able to do to improve the experience?

2

Same issue is persisting so far on every Groovy exercise I’ve submitted, even very linear complexity problems (such as Hello World).

Even in the web UI it seems it takes a bunch of tries to get it to process successfully: no code changes, just repeatedly smashing test again and again until it happens not to time out.

3

Hey. Thanks for reporting :)

There’s some issue with the CLI flow that’s causing the “pass then fail” thing. I don’t understand what it is and keep failing to find the time to work it out. But it’s on my high priority list.

Re the groovy specific stuff, maybe there’s an issue with the Test Runner. @ErikSchierboom is off next week, but when he’s back the week after, I’m going to ask him to spend the week just going through all the test runners, checking them and understanding what’s slow/fast, etc.

4

Currently hello-world seems to be consistently passing for me.

5

That’s not to say that we can’t improve things, but my JVM knowledge is fairly limited.

If you have such expertise, the following two files are the key ones that determine how the Groovy test runner works: