Wordy tests require non-meaningful error messages

Hey,
Just recently I got stuck on the Wordy exercise due to what seems like some quite arbitrary error requirements.

Largely the syntax of a correctly formed sentence seems to match the following regex:
^What is( (\d [\ -*/]))* \d ?$

How come a value like “Who is the President of the United States?” is supposed to raise ValueError(“unknown operation”) when the syntax of the whole sentence mismatches a correctly formed question. It seems like a ValueError(“syntax error”) would be more fitting here.

Additionally we could argue the same for the “What is 52 cubed?”. You could pose this question in a way that aligns with the syntax more using a statement like “What is 52 to power of 2”. This way we could keep the syntax like and avoid having to use so many exceptions.

Linking to the PRs so you can read the rationales given at the time:

Is something like “What is 10 mod 2?” (or “What is 52 to power of 2”) really a syntax error? It’s a properly formed “wordy” expression, just that the requirements don’t make you implement the mod operation – hence “unknown”.

For expressions that don’t start with “What is” , there is a debate about what kind of error that is.

We recently updated the Clojure track tests, and this was the only test where we decided against an "unknown operation” error message. We classified it as “syntax error”.

Hi @Hatster :wave:

Python maintainer here. Welcome to the Exercism forums.

While you and @tasx may have a point, I think
“Who is the President of the United States?” is on the line and could go either way. The decision at the time was for “unknown operation” in the cross-track problem specifications.

When Python implemented its test generation template for the exercise, we decided to accept the error test case as-is.

I’m not sure that changing the case it at this point and invalidating 4400+ student solutions would be helpful or add to the learning in the exercise.

However, I will think about if we want to “reject” the case and omit it from the test suite going forward.

What if we were to relax the problem specs by dropping specific error messages? I think that would avoid breaking any tracks/solutions. Tracks could opt to keep those specific messages or opt to remove message requirements or to implement custom/uniform messages. Would that simplify the exercise in a way that keeps things clean and simple to all tracks?

Tracks already have the ability to deviate from problem specs. Most tracks do to some extent. Relaxing specs to the point that they are confusing and offer no guidance to tracks would create more work, IMHO. If a track really disagrees with the messages or the case, they can reject or modify it - as @tasx has pointed out with the Clojure track deviation.

In this case, Python didn’t disagree. And maybe that will change, and we will remove the test case, which will also NOT break previous solutions. As I said, I will think about it.

Python used to just accept anything for an error message. We decided it was terrible for student learning., and so changed to use the prob spec error messages.

An occasional debate about errors on the track seems better than letting students through with error messages that read . or _ or .

1 Like

Once the CI clears, I will merge [Wordy]: Removed "President" Test Case & Regenerated Test file by BethanyG · Pull Request #3868 · exercism/python · GitHub, which removes the
Who is the President of the United States
from the test cases for Wordy.