Adding tests to an exercise

Exercism
1

I would like to add additional tests to rust/exercises/allergies. The tests did not cover all cases. While I added the tests locally, how do I contribute those to exercism?

1 Like
2

The Rust tests likely are pulled from the problem specifications repo. Unless the test is Rust-specific, changes to the exercise ought to go there then be applied to the various language tracks.

However, before changes are made, they should be discussed. What tests are missing? Are they important? Why? The first step would be to provide details (here is probably a good place) to discuss whether or not there is a gap that merits closing.

3

I don’t know if this exercise is offered in other tracks but I completed it here: Allergies in Rust on Exercism. It is certainly possible to get the tests to pass in a brittle fashion. By adding say, something like this, the learner realizes that simply subtracting one from the score is brittle, though this works here: let allergies = Allergies::new(5).allergies();. I personally, while solving, added additional tests for a scores of 7 and 10.

4

The Allergies exercise exists across quite a few language tracks.

Can you explain why/how 7 and 10 are different from any of the other inputs?

Can you point to which test case (line number?) in the canonical data has the issue?

5

The point I am attempting to make is that the current tests are find but it would be better if there were tests for 7 and 10 as below:

#[test]
//#[ignore]
fn allergic_to_few_things() {
    let expected = &[
        Allergen::Eggs,
        Allergen::Peanuts,
        Allergen::Shellfish,
    ];
    let allergies = Allergies::new(7).allergies();
    compare_allergy_vectors(expected, &allergies);
}

To explain slightly differently from my last post, a novice (such as myself) could code something such as, if socre % 2 == 1 { let tmp = score - 1}. In fact, I got all tests to pass since I handled the larger scores differently. The additional tests catch my poor algorithm sooner rather than later.

6

For any exercise, it is possible to write a suboptimal solution which passes all the tests but would fail for some additional test. That usually is a sign that the implementation is written too much to solve the specific tests and not the broader question.

At a rough count, that exercise has over 50 tests and covers all the allergens. It would be possible to generate a test for every possible input, but that’s not desirable. The goal is to have a broad range of tests that are representative of the general problem.

The existing tests include requesting the allergies associated with inputs 1, 2, 3, 5, 8.

Do 7 and 10 have any specific behavior/value different from the rest? Is there a reason to add 7 and 10 but not 9, 11, 12 and 13? Do 7 and 10 catch a common issue? You might have special handling for that input, but testing those doesn’t stop someone from having special handling for another value.

It’s hard to justify or understand why those specific tests should be added without a specific justification. With 8 allergens, there are 256 possible distinct return values and possible inputs/tests which can be written for them. What’s special about 7 and 10 and how do they add value?

7

The Rust track has 14 tests. However, I see your point regarding suboptimal solutions solving for specific tests.