Hangman: add canonical data

Exercism Building Exercism
,
1

I noticed Hangman does not have canonical data. (I am adding missing exercises to Go and it’s easier to add exercises with data.)

It is implemented on 7 tracks: csharp fsharp java ocaml powershell python tcl.

Based on the tests in those tracks, I propose the following test cases (plus the rest of the canonical data JSON file).

I would love feedback on this proposal, and/or a vote to implement this.

{
  "description": "game state",
  "comments": [
    "The expected values may be returned from the guess function, or returned from a separate game state call.",
    "They may also be accessed from a different call for each value, based on what is most idiomatic for the language.",
    "To explore objects, each guessed letter should be a separate call."
  ],
  "cases": [
    {
      "description": "Initially 9 failures are allowed and no letters are guessed",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": []
      },
      "expected": {
        "state": "Ongoing",
        "maskedWord": "____",
        "remainingFailures": 9
      }
    },
    {
      "description": "After 10 failures the game is over",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": [
          "a",
          "b",
          "c",
          "d",
          "e",
          "f",
          "g",
          "h",
          "i",
          "j"
        ]
      },
      "expected": {
        "state": "Lose",
        "maskedWord": "____",
        "remainingFailures": 0
      }
    },
    {
      "description": "Losing with several correct guesses",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": [
          "t",
          "o",
          "a",
          "b",
          "c",
          "d",
          "e",
          "f",
          "g",
          "h",
          "i",
          "j"
        ]
      },
      "expected": {
        "state": "Lose",
        "maskedWord": "_oot",
        "remainingFailures": 0
      }
    },
    {
      "description": "Feeding a correct letter removes underscores",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": [
          "t"
        ]
      },
      "expected": {
        "state": "Ongoing",
        "maskedWord": "___t",
        "remainingFailures": 9
      }
    },
    {
      "description": "Feeding a correct letter twice counts as a failure",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": [
          "t",
          "t"
        ]
      },
      "expected": {
        "state": "Ongoing",
        "maskedWord": "___t",
        "remainingFailures": 8
      }
    },
    {
      "description": "Guessing a repeated letter reveals all instances",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": [
          "t",
          "t",
          "o"
        ]
      },
      "expected": {
        "state": "Ongoing",
        "maskedWord": "_oot",
        "remainingFailures": 8
      }
    },
    {
      "description": "Getting all the letters right makes for a win",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": [
          "t",
          "t",
          "o",
          "l"
        ]
      },
      "expected": {
        "state": "Win",
        "maskedWord": "loot",
        "remainingFailures": 8
      }
    },
    {
      "description": "Winning on the last guess is still a win",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": [
          "a",
          "b",
          "c",
          "d",
          "e",
          "f",
          "g",
          "h",
          "i",
          "t",
          "o",
          "l"
        ]
      },
      "expected": {
        "state": "Win",
        "maskedWord": "loot",
        "remainingFailures": 0
      }
    },
    {
      "description": "Guessing after a lose is error",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": [
          "a",
          "b",
          "c",
          "d",
          "e",
          "f",
          "g",
          "h",
          "i",
          "j",
          "k"
        ]
      },
      "expected": {
        "error": "cannot guess after the game is lost"
      }
    },
    {
      "description": "Guessing after a win is error",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": [
          "t",
          "o",
          "l",
          "l"
        ]
      },
      "expected": {
        "error": "cannot guess after the game is won"
      }
    }
  ]
}
1 Like
2

In case of an error, many exercises have an object as expected:

"expected": {
  "error": "my error string"
}

Other than that, +1 here.

3

Updated to use that nested format :slight_smile:

2 Likes
4

I personally feel about the exercise name like about “minesweeper”: a non-violent alternative would be very welcome. We shouldn’t hang people here.

Somewhere in the forum there is a thread (which of course I could not find right now) about canonical data, where it says something along the lines of: Test cases in canonical data are in the recommended sequence for Test Driven Development.

To me, this implies a) having test cases for successful “guess” invocations, too and b) successful “guess” invocations should use “gameState” to assert on expected outcomes. I have not yet attempted to design / implement a solution, so I have no recommendation for that. (see next comment).

Also the currently last description is duplicated from the test case above it. Probably it should be Guessing after a win is error.

1 Like
5

I went through some examples in problem-specifications, and I think it would be good to stick with "property": "gameState" for all test cases.

We have bank-account et al. using operations for a sequence of different steps, underlining the state modifying nature of the problem. But I think that is not the way to go here.

A better approach, I think, is what high-scores et al. do: Feed a series of values into a single property (function), following a stateless approach. The key is the single property, which allows functional approaches.

That simply replaces "property": "guess" with "property": "gameState". What do you think?

6

I’m all for renaming this exercise but I think that’s a separate discussion from creating the canonical data for the tests. I would prefer keeping this topic focused on the test data.

I corrected the duplicate description. Thank you.

I believe this data does cover both the happy path as well as the error path, but the two are recorded as separate sets of tests.

From the perspective of using the data, I find it is much easier in Go to work with the JSON when the shape of the expected field is predictable. That’s why the error cases were separated from the non error cases. It sounds like you’re essentially suggesting having the exact same data but using the same property name for all the cases. Do you find that structure easier to deal with? Or is that purely for consistency?

7

It’s not for “consistency”. While I do see the problem to handle the JSON in a test generator, I am concerned about the students interface required when having 2 properties.

Having gameState and guess properties is fundamentally requiring a kind of object to handle the same state. I cannot think of a functional solution that doesn’t look weird somehow. Should guess feed aggregated state into gameState to produce the output? Or should gameState invoke guess to add a guess to a state that’s not local to guess? That’s where I see 2 properties being in conflict with the rest of the exercise design.

Could I clarify my concern?

8

The properties describe what the tests check. They may to test properties, not solution properties. There’s no requirement that properties map to different functions in the solution.

9

Well, yes, there is no “hard requirement” what “property” stands for in canonical data:

The ‘property’ is a string in lowerCamelCase identifying the type of test, but most of the times it is just the name of a function being tested.
Test Data Format

So your way to look at it is, in my eyes, as well correct as in opposition to the exercise implementations I have seen so far. I got to the understanding that “property” is used as the name of a function or object property (field / method) whenever possible. And to me, it would be one of the annoying exceptions when property does not serve as a function name here.

So I suggest to do so. It’s easily possible here. There are many exercises where expected is used to hold a result and an error, that’s not a surprise to me. Not using property for the recommended function name would be.

1 Like
10

That makes sense. Thanks for talking it out with me. I updated the data to use a single property.

1 Like
11

As an aside, I got curious about properties mapping to functions or not.

These has properties that describe what is being tests:

  • diffie-hellman -> keyExchange privateKeyIsInRange privateKeyIsRandom secret publicKey
  • forth -> evaluateBoth evaluate
  • gigasecond -> isEqual add
  • high-scores -> latest latestAfterBest latestAfterTopThree personalBest scores scoresAfterBest scoresAfterTopThree personalTopThree
  • run-length-encoding -> consistency decode encode
  • zebra-puzzle -> drinksWater ownsZebra
  • zipper -> sameResultFromOperations expectedValue

These do validation by suggesting a two step setup. In the former at least, I felt the code and tests would have been much cleaner and logical if only one function was used.

  • queen-attack -> create canAttack
  • robot-simulator -> create move
1 Like
12

I wasn’t around when those exercises (queen-attack and robot-simulator) were added, but I think both have an OOP flavor. The create key would map to a constructor and the other key to a method. Combining the functions would make the maintainer’s job a little harder in an OOP track, whereas the current configuration complicates things somewhat for functional tracks. But nothing insurmountable in either case.

13

I believe this exercise is used as an OOP exercise in most the tracks that implement it, too. Though the data really could be used either way.

14

Should there be a second “Lose” test with at least one correct guess? At the moment, I can’t for sure say whether I’m supposed to mask the word completely like the existing “Lose” test or show any correct guesses like in the “Ongoing” state tests.

15

Added this case for that:

    {
      "description": "Losing with several correct guesses",
      "property": "guess",
      "input": {
        "word": "loot",
        "guesses": ["t", "o", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j"]
      },
      "expected": {
        "state": "Lose",
        "maskedWord": "_oot",
        "remainingFailures": 0
      }
    },
1 Like
16

Looks good to me. +1

1 Like
17

Thanks, everyone! Here’s a PR with all the above changes.

3 Likes
18

3 approvals

1 Like
19

Thanks, all, for the input and help here! I’ll let the PR sit for a bit before merging in case anyone else has something to add.

2 Likes
20

Merged! This exercise now has canonical data.

3 Likes