Docker issue with Pyret test runner

Programming
1

bin/run-tests-in-docker.sh works locally in about 6 seconds, but the CI workflow hangs during the testing. I don’t know anything about Docker or GitHub Actions so is there anything obvious I’m doing wrong?

Log from successful local Docker run 

[Running] /usr/bin/env sh "/Users/anagy/Documents/pyret-test-runner-main/bin/run-tests-in-docker.sh"

#1 [internal] load .dockerignore

#1 transferring context: 213B done

#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile

#2 transferring dockerfile: 298B done

#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/alpine:3.17

#3 DONE 1.2s

#4 [1/4] FROM docker.io/library/alpine:3.17@sha256:e95676db9e4a4f16f6cc01a8915368f82b018cc07aba951c1bd1db586c081388

#4 DONE 0.0s

#5 [internal] load build context

#5 transferring context: 2.95kB 0.0s done

#5 DONE 0.0s

#6 [3/4] WORKDIR /opt/test-runner

#6 CACHED

#7 [2/4] RUN apk add --no-cache jq coreutils nodejs npm && npm install -g pyret-npm

#7 CACHED

#8 [4/4] COPY . .

#8 CACHED

#9 exporting to image

#9 exporting layers done

#9 writing image sha256:1f7f7748e3dc031e178782caaf942544b4bbc6c01b94c622d60ed7cd736abc64 done

#9 naming to docker.io/exercism/test-runner done

#9 DONE 0.0s

example-all-fail: testing...

example-all-fail: done

example-all-fail: comparing results.json to expected_results.json

example-empty-file: testing...

example-empty-file: done

example-empty-file: comparing results.json to expected_results.json

example-partial-fail: testing...

example-partial-fail: done

example-partial-fail: comparing results.json to expected_results.json

example-success: testing...

example-success: done

example-success: comparing results.json to expected_results.json

example-syntax-error: testing...

example-syntax-error: done

example-syntax-error: comparing results.json to expected_results.json

[Done] exited with code=0 in 6.319 seconds
2

Python uses a docker-compose file to invoke the runner when doing CI checks. Don’t know if that would help in this situation or not.

I will give it a try in my fork.

1 Like
3

I can look at it next week.

4

When I use docker-compose, I get a permissions denied error. It can’t pull the test runner image from the store, so it fails.

Probably because the ci.yml file is not actually pushing and/or we don’t have an official test runner repo as of yet.

If I don’t use compose, I get this weird thing where the runner complains that it can’t find pyret. Here is a screengrab of the run I just did with this ci.yml file:

image
image2252×2693 643 KB

5

Thanks for looking into this. With Meatball’s help, we’re now at twenty exercises in the bag. I’ll be a bit busy this upcoming week, but I’m hoping to get another crack at tweaking the sanitizing of the test runner output for the results.json.

6

This is a bit weird. I got the hanging test too the first time, but the second time it did work. Looking at the code, I don’t immediately see anything weird happening. The only thing that I noticed was that -t "test.sock" was being used. Maybe the socket is not ready in CI?

7

Yeah, that’s an artifact from my earlier troubleshooting. The WebSocket server that compiles the Pyret code normally defaults to a socket file stored in the temp folder. They do note it’s experimental and can get stuck. We’d need to stop the server process, delete the socket file, and start again I think in that case.

8

Bugfix/sanitize output for empty file by BNAndras · Pull Request #9 · BNAndras/pyret-test-runner · GitHub resolved the sanitizing issue (just needed to simplify how I removed the folder paths). Now I’m having the same issue as Bethany where the CI doesn’t see Pyret.

Otherwise, I’m happy with how the test runner runs locally and in the Docker image so I’d like to request the official test runner repo, @iHiD. We can continue troubleshooting the CI itself there.

9

@ErikSchierboom’s been upgraded to be the creator of these now, so I’ll leave that to him. Congrats to getting to this stage! :slight_smile:

1 Like
10

GitHub - exercism/pyret-test-runner has been created

2 Likes