What files can users change for the test runner?

Exercism Building Exercism
1

So, basically a user can only change the “solution” file and not the “test” file, right? Otherwise users could cheat the test runner by deleting all tests. Am I correct in assuming what which file is the “solution” file and which one is the “test” file is determined by the keys files.solution and files.test in the config.json of an exercise? (The documentation doesn’t specifically mention an impact on test runner behavior.)

The reason I’m asking because on the Rust track, the solution file editable by users is usually src/lib.rs, but now I have a use case where I would like to lock down src/lib.rs as a test file and use a different one for the users solution.

Thank you for quickly confirming if my assumption is correct :slight_smile:

2

Add src/lib.rs to .files.editor so that it’s visible to people using the online editor but read only. Put some other file as the solution.

Of course for people working locally, they can edit whatever they want to, and submit any number of files. If they want to cheat, let them: they’re only cheating themselves.

The test runner just uses whatever files are in the user’s submission

1 Like
3

Are you sure that users can submit their own tests? I tried wiping a test file and explicitly submitting that, it doesn’t work. The test runner still reports the list of failed test cases.

Either way, I guess you’re right. It’s not a big deal if users can technically cheat. Solving our exercises isn’t a competitive activity.

4

What exactly did you submit?

5

Hope you can see this iteration:

https://exercism.org/tracks/rust/exercises/bob/iterations?idx=9

tests/bob.rs is part of the submission, but the test runner clearly still tries to compile and run the old bob.rs with the correct tests.

6

This one might be a better example, the tests actually compile and are run but all fail.

https://exercism.org/tracks/rust/exercises/bob/iterations?idx=10

7
  • editor => files that are visible but readonly
  • solution => files that are part of the solution, visible and editable
  • test => files that are part of the tests, not visible and not editable

This is what it’s meant to be.

In our case, if src/lib.rs must be visible but readonly, make it an editor file.

Are you sure that users can submit their own tests? I tried wiping a test file and explicitly submitting that, it doesn’t work. The test runner still reports the list of failed test cases.

I don’t recall if it now “keeps” the edited tests or if it re-replaces it with the git tests. A decision was made on that a while ago.

  • If you submit tests with your solution, these are stored
  • I don’t recall if that is now used in the test runner or no
  • It should not matter. If the tests are meant to be submitted and changed, it should be a solution file, not a test file
2 Likes
8

I think this is relevant here?

3 Likes
9

That’s it!

So indeed should work as I wrote.

1 Like