Strange behavior of tests in online editor when updated exercise

Hi!
I’ve updated recently exercise in order to keep in-sync with latest problem specifications. Tests were updated on test class, however it keeps a reference (in online editor) about old tests that were removed, which obviously causes trouble when running tests. Is there a way to remove “old” tests and keep only updated tests?
Test class is here: pharo-smalltalk/exercises/practice/darts/DartsTest.class.st at main · exercism/pharo-smalltalk · GitHub
and configuration here: pharo-smalltalk/exercises/practice/darts/.meta/tests.toml at main · exercism/pharo-smalltalk · GitHub

Are you saying the test file in the editor doesn’t match the test file in the repo, or that the file in the repo has multiple copies of tests?

The test file in the editor should match either the old or the new version of the test file in the repo, depending on whether or not it’s been updated.

It is weird. In UI of editor I’ve been asked whether to move to latest version of exercise (and that existing solution may stop working). I confirmed, re-run the tests and it’s still showing old tests from previous version of exercise alongside with new tests. Old should not be there and they aren’t in repository now. But I guess it is because of test runner is still “old” (since it is in another repository) and need to be triggered CI to rebuild the docker container? I guess it maybe related (maybe the test runner refers to previous test class code still?) that test runner is out of sync…

Unless something changed in the test runner, it shouldn’t need a rebuild. The test runner shouldn’t cache the tests anywhere. I believe the solution and ought to be loaded on each run.

The test runner shouldn’t cache the tests anywhere.

Right :) That’s exactly case. Pharo is image based, so it uses vanilla specification of exercises from repository: image for test runner is built and vanilla exercises are loaded. During test run: only solution classes are removed, but not the test class itself: here

Therefore test class that is passed into test runner image from cmd line is merged with test class in image and that causes the trouble. I guess I will need to rewrite test runner project specification, so no vanilla exercises remain in image before test run: specification of packages for test runner