Tests and excluded tag :pending

Programming Elixir
1

Hi

Some of the tests have the tag :pending that are excluded when I run mix test locally. When I submit all the tests are run.

  1. Whats the point of not running these tests?
  2. Can I disable this behaviour, ie. can I somehow run mix tests and have all the tests executed?

cheers
mg

image
image1366×1040 82.1 KB

2

Re 1: Practice exercises on Exercism are supposed to encourage Test-Driven Development. This is a global platform guideline, see building practice exercises:

Exercism favors Practice Exercises being completed via Test Driven Development. To achieve this, there are two options:

  • The test runner must run the tests in the order defined in the file AND the test suite must bail on the first failure; or
  • All but the first test should be skipped by default.

The idea is that the student focuses on solving the exercise in small increments, striving to make one test pass at a time, and not the whole test suite at once.

Re 2: See the documentation about running Elixir test on Exercism

1 Like
3

understood, thank you

4

Given the existence of --max-failures, are pending tests the best way to implement a TDD workflow? Having to dig into the test file to comment out tags seems a little inefficient.

(I’ll probably just use mix test --include pending --max-failures 1 myself)

5

In a typical TDD workflow, you’re supposed to:

  1. Write a test and see it fail
  2. Implement missing functionality
  3. See the tests pass
  4. Repeat those steps by writing new tests until all functionalities are implemented

On Exercism, you cannot exactly do that because all tests are already there, written for you. To mimic this process, you can progressively extend how many tests are run by removing the pending flag. Replacing this with --max-failures 1 would make this process even further away from the intended TDD workflow because you would never see a fully green output of your test suite until after you finish implementing all functionalities. Each time you would run the test suite, you would need to stop and read the output to figure out whether the one failing test is a failure related to the functionality that you’re trying to implement currently or a functionality that you haven’t started implementing yet.

Both ways of working are perfectly fine! It’s a question of where you prefer to put more effort - adjusting the test suite multiple times or reading the test output more carefully.

But yes, I am pretty sure that pending tests are the better way of estimating a TDD workflow.

2 Likes