For RollDice, we should check that the returned value falls within the expected range but also that it changes at least once over the existing 100 iterations.
For SpellStrength, we should similarly iterate 100 times and check if the returned value is in the expected range and also whether it changes at least once.
When I was doing a captains-log implementation for R last year I had a similar issue. I ended up just testing that there was at least one value above mid-range and one below. Not perfect, and I was thinking about this again for Julia again earlier today - so I’ll be very interested in this discussion.
How easy is it to calculate the StdDev or variance of the 100 values, and check it is above some reasonable threshold?
It shouldn’t be that hard, but I’m not enough of a scientist to know what the right number of iterations is to make it extremely unlikely to have false negatives.
For dnd-character, Java has a track-specific test that generates a character 1000 times and fails as soon as the current character has all six attributes the same as the previous character.
I think that’d be a good starting place. I suppose a student could cheat, keep state with a list of numbers, and return the next number each time, but that seems more work than doing the exercise as intended. If they did just pick the next number, it seems the distribution would be indicative though.
And if they learned something from it, would it really be “cheating”?
In the Ruby track we do something similar for “random” checks. The mentoring/review will bring out the rest of it, even in the real word, checking if something is random or only appears to be random is not an easy task.
If it is required that some function or method or process is used, then there may be ways to force that check to happen.
In the Randomness concept document “about” it even says “generally used” about the System.Random class. So testing for that class being used would not even have 100% surety that something is not random. There are other ways to get randomness (to varying levels of security) than only that class.
I think passing behavior checks to some level of “works as expected” and having review/mentoring confirm and talk about those things are more valuable than only having a “pass/fail” indication, anyway.
I noticed the same thing and thought it would be a fun challenge to try and address it. Were I smarter, I could probably do better. Apologies if this is unwelcome.
[Fact]
[Task(3)]
public void RollDieDeep()
{
var player = new Player();
var total = 0;
var prevTotal = 0;
var totalMatches = 0;
for (var j = 0; j < 5; j++)
{
for (var i = 0; i < 1000; i++)
{
var dieRoll = player.RollDie();
Assert.InRange(dieRoll, 1, 18);
total += dieRoll;
}
Assert.InRange(total, 8450, 10550);
totalMatches = prevTotal == total ? totalMatches++ : 0;
Assert.NotEqual(5, totalMatches);
prevTotal = total;
total = 0;
}
}
[Fact]
[Task(4)]
public void GenerateSpellStrengthDeep()
{
var player = new Player();
var total = 0d;
var prevTotal = 0d;
var totalMatches = 0;
for (var j = 0; j < 5; j++)
{
for (var i = 0; i < 10000; i++)
{
var strength = player.GenerateSpellStrength();
Assert.InRange(strength, 0.0, 100.0);
total += strength;
}
Assert.InRange(total, 472614, 527386);
totalMatches = prevTotal == total ? totalMatches++ : 0;
Assert.NotEqual(5, totalMatches);
prevTotal = total;
total = 0;
}
}