Improve testing for Animal Magic Exercise

Exercism Bugs & Feature Requests
1

I had a conversation with my mentor exercise, Animal Magic, about the tests for the exercise.

I have pasted some of my observations/questions, and added an in my eyes, improved and more correct test.

Me:

I have seen multiple community examples like
https://exercism.org/tracks/go/exercises/animal-magic/solutions/ro-jc
https://exercism.org/tracks/go/exercises/animal-magic/solutions/andrerfcsantos

Where they do rand.Intn(20) + 1 but still marks as tests passed. I don't get that. That would in my eyes from time to time result in a die roll of 21, and no only between 1 and 20 as stated in the requirements.

Can someone explain this to me?

Mentor:

"Intn returns, as an int, a non-negative pseudo-random number in the half-open interval [0,n) from the default Source. It panics if n <= 0."

Me:

I read that, also before posting. But doesn't it say between 0 and 100 if rand.Intn(100) or what am I reading/misunderstanding wrong.

Despite that either the two examples i linked to should fail or my should fail, they cannot both be right.

Not my code, this could in my eyes potentially also return 21.

// RollADie returns a random int between 1 and 20
func RollADie() int {
    return 1 + rand.Intn(20)
}

My code:

// RollADie returns a random int d with 1 <= d <= 20.
func RollADie() int {
	return rand.Intn(19) + 1
}

They cannot both be right.

Mentor:

rand.Intn(20) will return integers from range 0 to 19 - note that the 0 in included in the range while 20 it is not. In order to generate an int that can be in range 1 - 20 you need to use 1 + rand.Intn(20) or rand.Intn(20) + 1. The second example will generate an int in range 0-18 and add 1 to the final, returned value. The clue I meantioned is in the range - half-open interval [0,n)

Me:

Thanks for your explanation will check it out tomorrow, and see if I understand it better, or even suggest an improvement to the code base.

So long story short, I think the solutions, my solution, with the return rand.Intn(19) + 1 are incorrect, and my new improved tests will show that. The tests will now “ensure” more than before that all numbers are rolled, still within the scope of 1-20. Of course, there is still a small statics possibility that e.g. 20 is never rolled on 100 rolls.

I can of course be completely wrong as I’m new to go, but would be happy to discuss this, so that we can get an improved code base.

Let me know what you think.

Updated test:

func TestRollADie(t *testing.T) {
	missingRolls := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20}
	const tests = 100
	var got int
	foundDifferent := false
	var last int
	for i := 0; i < tests; i++ {
		got = RollADie()
		if got < 1 || got > 20 {
			t.Fatalf("RollADie() out of range: %d", got)
		}
		if i > 0 && got != last {
			foundDifferent = true
		}

		for i := 0; i < len(missingRolls); i++ {
			if missingRolls[i] == got {
				missingRolls = append(missingRolls[:i], missingRolls[i+1:]...)
			}
		}
		last = got
	}

	if len(missingRolls) > 0 {
		t.Errorf("RollADie() never rolled the numbers: %d", missingRolls)
	}

	if !foundDifferent {
		t.Errorf("RollADie() always generates the same number: %d", got)
	}
}

I have a branch ready for PR, GitHub - tomasnorre/go at improving-test-for-animal-magic-exercise but it got automatically rejected, and asked to add discussion here.

2

Did you test your code? If you roll a d20 100 times, there’s a decently high chance that you won’t see all 20 values. It would suck for students to write perfectly good code only to have it randomly fail because we’re relying on randomly generated sequences to contain specific values.

3

That’s true.

But as the test is running 100 times, the likelihood of it failing is smaller. But I get your point. But without my “improvement” there is a risk of having rolls never hitting 20 with the rand.Intn(19) + 1 roll, which is incorrect, but test isn’t failing.

4

I’m rusty on my stats so I just simulated this :smile:

>>> collections.Counter(len({random.randint(1, 20) for _ in range(100)}) for _ in range(10000))
Counter({20: 8885, 19: 1071, 18: 44})

Your proposed test seems to have an 11% likelihood of failing on correct solutions. That’s pretty terrible!!

5

I must honestly say, I cannot say if you are right or wrong, but I’ll consider a better way to test this. Testing Random functions have never been easy :)

6

You can test that yourself! It would be pretty easy to generate 100 random numbers and count how often you get an 20 numbers or not. Put that in a loop, run it 1000 times and you have some stats!