TDD and skipping tests after the first failed test

Programming Zig
1

Zig track docs read:

you would have to comment out tests that are not passing if you wish to go through each test one by one.

This sounds like an inconvenience for the student.

I do not know if going through tests one by one is the prefered way. If it is we have a bunch of ways to do it without commenting out the tests manually. We can set up a custom test runner. Or we can work around it with this simple hack:

// test_exercise_name.zig
var skip = false;

test "first test" {
    if (skip) return error.SkipZigTest;
    errdefer skip = true;
    // ...
}

test "second test" {
    if (skip) return error.SkipZigTest;
    errdefer skip = true;
    // ...
}

Basically it uses a global variable to detect a failed test and returns error.SkipZigTest from the rest of the tests.

Do we need this kind of convenience?

2

If you’re working on the track using the web UI, that doesn’t apply as you cannot modify the tests. It’s specific to working locally.

If you’re working locally, many tracks default to having all the tests except one commented out, which forces you to uncomment them one by one. This is the inconvenience in the reverse direction.

You could also not comment out the tests, run them all, and fix them one by one without touching the test file.

3

Should we then mention this trick in the track docs?

4

Which trick?

To run the tests, all you need to do is run the following command:

zig test test_exercise_name.zig

That tells you how to run all the tests. That is sufficient and all you need to run all the tests.

you would have to comment out tests that are not passing if you wish to go through each test one by one

This tells you what you can do if you prefer to run one test at a time, an optional approach.

5

Add

if (true) return error.SkipZigTest;

line instead of commenting out the test.

Okay. I understand now that going through tests one by one is not always the prefered way. No need to skip tests by default.

6

That’s the same as simply return error.SkipZigTest;. You can certainly use that, but that’s a whole lot more typing vs commenting out a line :slight_smile:

Different people have different preferences here.

7

Actually, no. The compiler will complain about unreachable code without if().

You can copy/paste the line pretty quickly. And if you set up the global variable you will be able to use the same file without further modifications. Second test will automagically enable itself after you pass the first test. How many times do you need to comment/uncomment if there are fifty tests in the exercise?

8

There is no “exit on first failing test” commandline switch? Write your own test runner seems to be an option…

9

Can you do this with exercism test?

10

exercism test runs a single command. If you wanted a custom test runner, it should be separate from that.

11

Is this about the exercism test command? I wasn’t even aware this exists before reading about it here. I naturally started to use the mentioned zig test test_exercise.zig. To run single tests or tests under a similar topic I just add --test-filter "second test" or --test-filter "common-test-topic-substring". No need to touch the test file to comment out test cases.

12

No. Not really. exercism test is a thin wrapper that checks the current track then runs the correct test command for you, eg zig test test_exercise.zig. It’s basically a language-to-test-command map. Anything more involved has nothing to do with exercism test.

1 Like