Create a validation system for exercises

I have done some exercises on the python and crystal track.
What I discovered is, that some exercises are well documented with comprehensive and detailed information about what to do and how the classes and methods are expected to work.
On the other hand there are some exercises where there is hardly any description at all and you have to reverse engineer the missing descriptions from the tests.
I would appreciate to have a system in place, where users can make remarks on exercises and rate the exercises. Those exercises where there is not enough description should be marked for completion and removed if not completed in a certain time.

I’m curious about what exercises you consider to be lacking description. Care to share?

A counterpoint: you are expected to read the tests and use the expected interface to solve exercises – What is Test-Driven Development?

3 Likes

Keep in mind that on exercism, there are 2 possible types of exercises in a track: practice exercise and concept exercise.

Concept exercises exist in a syllabus to teach learners about certain concept of a language, thus it has more details about what to do, examples and expectations.

Practice exercises just as the name implied, is for people to practice. So beside a quick introduction, you are expected to do the research yourself regarding the concept or algo involved with that exercise and go through the test suites to solve it.

Another thing many track just have the bare necessary predefined stub, some might give you a bit more info and docs just to look a bit nicer, but ultimately you are expected to do the work yourself.

5 Likes

Take for example the React-Exercise in the Python or Crystal-Track.
There is almost no Description.
There should be at least a Description about the purpose and function of such a system. Further there should be a description about the expected functionality which the user’s program should provide.

Test driven development Imho means, you write your program and test parts and functions thoroughly as you go.
It certainly doesn’t mean you program along the tests, because that means, the tests won’t fail although you don’t really know, what you are doing.
Sorry this was meant to be a reply to glennj.

I do agree that the exercises should have some progression so that later exercises don’t explain simple concepts anymore. Still it is very discouraging when there is no explanation at all. Even if the required concept is something very common, like a linked list, you still need to know what functionality is expected and how your methods should be named.
I also think a short explanation about any new concepts like recursive programming etc. would be helpful.
Good exercises take a step by step approach. They tell the user how to begin and what kind of functionality should be provided first. Then other methods are added one after the other.

Test-driven development is a software development methodology and it has specific meaning. It is essentially a way to encode the system’s requirements in tests that informs the developer about the expected interface and scope of the code to be written.

Most exercises take their descriptions/instructions/introductions from the exercism/problem-specification repo in Github. If you’re finding the descriptions lacking for a particular exercise, we would welcome a discussion about it here on the forum. Please be specific: what exercise? what improvements would you suggest?

Looking at react specifically, it’s generally one of the higher-rated difficulties, and by the time the student is working on it, they should have experienced solving lots of exercises and are comfortable with Exercism’s TDD approach.

1 Like
  1. Specific techniques like recursion are implementation details. Almost all of the practice exercises don’t test how you implement the solution. There are several exercises where you might opt to use recursion but could also solve them without recursion. As such, introducing concepts like recursion can’t really be tied to any one exercise.
  2. The practice exercises can be completed in any order so even if recursion could be tied to specific exercises there is no specific exercise where it would first be encountered and be a new concept.
  3. Specific concepts like recursion really benefit from language/track specifics. The practice exercises are shared across all 70+ tracks so they can’t easily be used to introduce concepts. That’s what the syllabus and concept exercises are for.
1 Like

Checking the Python version, it certainly needs a reasonably advanced knowledge of the language. I was interested to see that the exercise was reduced from difficulty 8 (Hard) to 6 (Medium) about 4 years ago. Obviously I never try to second-guess @BethanyG, but this made me gently raise a eyebrow and think there must be a story behind it.

Crystal is also difficulty 6, the nine other tracks I looked at are all in the 8-10 range. Expect to be challenged.

Nine out of eleven tracks rating 8-10 is in line though with what Glenn was saying. Python might have it as a medium exercise, but OP’s critique wasn’t about the difficulty.

The OP might be expecting something similar to a design document that gives a broad overview on what they are trying to solve, how the pieces might fit together, and that sort of context that isn’t easily gleaned from just looking at individual test cases. That then lets a student break down the process and identify an approach.

However, for the reasons IsaacG mentioned, that doesn’t quite align with how we approach track-agnostic practice exercise documentation so I’m stuck here. I suppose we can add an introduction that at least introduce a fictional problem we’re trying to solve in the exercise.

I think the OP wants the same level of detail that is given in concept exercises, which isn’t something we do for practice exercises. We could, however give some general advice on how someone might implement a reactive approach in general … but I do worry that that might be problematic because of language differences and techniques.

All that being said, react is an older exercise that doesn’t have the problem spec introduction + instructions that say, anagram does.

I think it could benefit from both a motivating story, and perhaps some links to general reactive programming resources (like this wiki article or maybe this article from Medium or this JS focused one or maybe this one from Baeldung or even this gist) in an instructions.md.

Tracks could then do an append for more language-specific instructions or links.

I have no idea or story behind the difficulty change. It’s lost (sadly) to the mists of time and my aging brain. :smile:

But I do know that I won’t be re-leveling the track any time soon. I’ve still got upgrade work left to do. Sorry. :blue_heart:

2 Likes

Just for clarification:
My request was not so much about this specific exercise (React) although ist’s a very good example. What I have in mind is a system that allows the users to make comments and rate the exercises.
At the moment even the worst exercises still stay in the system and annoy the users without any possibility to give any feedback.
Another idea is to create the possibility to users to add some clarification and explanations.
I think all sides could greatly benefit from such a solution.
Just have a look at the LinkedList exercise of the python track. It still leaves every part of the solution to the user, but explains at least what is expected from the program. So you write your program according to the instructions and then use the test for validation.

I agree with your point one, at least, if the concept of recursion is not specifically intended to be used in the actual exercise.
With respect to the order of the parts of the exercise I think that a step by step approach will often be helpful and can still be ignored by experienced programmers.
Your last aspect seems a bit abstract. Where in the Crystal-Track would I find introducing exercises on such fundamental concepts as abstract base classes, callback functions or just subclasses. If there are any, I haven’t stumbled across them yet.

Life finds a way. I give my feedback in comments in my solutions.

Exercism tracks are curated by track maintainers, and are not cloud sourced :slight_smile: Exercism users will never have direct control over what is and isn’t in the track.

There is a mechanism for users to provide feedback and make changes, though! It’s what you’re doing right now – engaging with the maintainers via this forum. The exercises are all developed, built and maintained by volunteers from the community. If you see something and have a specific improvement you’d like to suggest, we’re happy to discuss that change.

We provide those steps for the concept exercises. The practice exercises are meant to be something you figure out as part of practicing programming; in the real world you’re given a problem with the requirements and you need to work out the steps. Some exercises give more details than others and we’re open to specific suggestions for exercise improvements. We’re unlikely to ever give step-by-step instructions on how to solve the exercise, though. Figuring out the steps is part of solving coding exercises.

All the Crystal concepts can be found on the Crystal concepts page. If you think something is missing and ought to be added, I believe the Crystal maintainers would be delighted to work with you and have you add additional concepts. Building out a syllabus is a lot of work and all done by volunteers. We always welcome new volunteers to improve what we have.

If you want to suggest improvements to the track, the maintainers need to see the feedback. The forum is the best place for that!

5 Likes