Exercism's CI and me: 2 + 2 = 3

axtens · January 18, 2023, 1:12pm

As the title says, I don’t get Exercism’s CI and the docs make no sense to me. Maybe I should take them into ChatGPT and see if I can get a meaningful summary.

I’ve been asked to make sure the CI for the 8th track is in place before launch. I’m tempted to launch it anyway and let someone else deal with it.

Yes, it is doing my head in. And every time I try to articulate the problem I run out of words.

Is there a video I can watch where someone goes through the process of setting up CI for a track from scratch? I normally don’t need videos – they take too much time – but I need a learn-by-watching.

ErikSchierboom · January 18, 2023, 1:32pm

You’ll need two files:

.github/workflows/test.yml: this is the workflow file that runs when a commit is pushed to main or to a PR. You already have this file: 8th/test.yml at main · exercism/8th · GitHub
bin/verify-exercises: this script will iterate over all the exercises and temporarily copy the exemplar/example implementation over the stub file and run the tests. A good starting point is generic-track/verify-exercises at main · exercism/generic-track · GitHub You have a variation of this file in 8th/test at main · exercism/8th · GitHub, which is fine (the name that is)

Some examples of the second file:

iHiD · January 18, 2023, 2:04pm

@axtens So to start you could implement this line:

github.com

exercism/8th/blob/main/bin/test#L34


      
                  echo "Checking $(basename "${concept_exercise_dir}") exercise..."
                  # TODO: run command to verify that the exemplar solution passes the tests
              fi
          done
          
          
# Verify the Practice Exercises
          for practice_exercise_dir in ./exercises/practice/*/; do
              if [ -d $practice_exercise_dir ]; then
                  echo "Checking $(basename "${practice_exercise_dir}") exercise..."
          	    8th -f ${practice_exercise_dir}.meta/example.8th -f ${practice_exercise_dir}test-words.8th -f ${practice_exercise_dir}$(basename "${practice_exercise_dir}")-tests.8th 
                  # TODO: run command to verify that the example solution passes the tests
              fi
          done

axtens · January 19, 2023, 4:02am

Is there some kind of implicit behaviour going on in the V example? How does the v -stats test temp/run_test.v signal failure? In the unison example exit ${exit_code} is explicit.

ErikSchierboom · January 19, 2023, 10:43am

I suspect that the V example crashes and stops execution? Or it might be a bug.

ErikSchierboom · January 19, 2023, 1:31pm

It was a bug: ci: fail when one of the exercise's example/exemplar does not pass the tests by ErikSchierboom · Pull Request #40 · exercism/vlang · GitHub