It happens again and again as I see requests for mentoring. Student gets overexited about comptime and uses it where it does not make any sense. You can search “comptime” in Isogram exercise Community Solutions. You will find comptime function parameter there. This makes the function useless in real program where you take user input in run-time. We do not want to teach the wrong lesson. We need to add a test.
Recent example. Raindrop exercise. Here is the solution that pissed me off so much that I went here on the forum:
It passed all the tests! This function abuses comptime and it can not process run-time known input. It also ignores the buffer. The implied API contract is that the returned slice references the buffer. All kinds of bad things could happen in real program if you return something else. But no test checkes it.
Here is an example of what tests we could add:
test "can handle run-time known user input" {
var data: u32 = undefined;
data = 42;
var buffer: [15]u8 = undefined;
try std.testing.expectEqualStrings("PlingPlong", convert(&buffer, data));
}
test "buffer is used for output" {
var buffer: [15]u8 = undefined;
const actual = convert(&buffer, 42);
try std.testing.expectEqual(buffer[0..actual.len], actual);
}
We should add tests against comptime function parameters in most exercises. We may need to add tests against ignored buffer in other exercises that involve such buffer.
The students are well aware comptime is only for input known at compile time, and are asking for guidance in how to best use their newly discovered comptime superpowers. They might have already solved a dozen exercises without comptime, and not asked for mentoring on those solutions.
I had a long back and forth with a student for a few days and I had hard time convincing them that comptime and const are different concepts and that Zig does not compile at run-time unlike JavaScript.
No, they are not. Every student I had a chance to chat with had some kind of misunderstanding about these concepts.
Yes, they are. Recent example is Nucleotide Count. It did not involve comptime function parameter though. You can use inline for in this exercise to do some smart comptime meta-programming.
No. Isogram is literally the first exercise after Hello World. And you can see the abuse of comptime there.
Actually the opposite is true. They solve dozens of exercise with comptime abuse, and do not ask for mentoring because they are unaware of the issues with their solutions.
I can deal with people who ask mentoring. The ones who do not ask worry me. They convince themselves that they have learned a good performance optimization and they will be very disappointed when time comes to apply it to a real program. Will they feel that Exercism did a good service to them?
I think it is better to enforce the assumption that parameters are run-time known. Add some tests.
I do not know nothing about analyzer. I do not think we have one on Zig track.
I assume adding one test against comptime to every exercise does not need much effort compared to creating a brand new analizer. The tests are straight forward. They are visible to the student. They are part of the language. Anyways, I would be happy with any automation that would give some feedback to students.
A good case for analyzer would be restricting allocators. Some exercises are designed to be solved without heap allocation. They do not pass the allocator as function parameter. Still, some students try to use an allocator. For example, they use heap allocated hashmap in Isogram and Pangram exercises.
I have now generated tests for all the existing exercises (some in PR 555, most of the rest in draft PR 556, two more exercise generators not yet uploaded).
One option is to add a test with different logic, another is to change the logic in existing tests.
Either way, we want existing submissions retested, so now-failing comptime submissions are no longer listed in Community Solutions. This retesting has has a cost. For example, Zig Track Management shows Raindrops has 653 completions.
Perhaps we should only update a handful of exercises.