CLI silently creates 0-byte files when file downloads hit HTTP 429 rate limit

Exercism Bugs & Feature Requests
1

Bug Description

When downloading exercises, the CLI fetches each file individually via separate HTTP requests. When these per-file requests receive HTTP 429 (Too Many Requests) responses, the CLI silently creates 0-byte files instead of reporting an error or retrying. The overall download reports success (“Downloaded to /path/to/exercise”) even though all files are empty.

This is particularly easy to trigger when downloading multiple exercises in sequence, since each exercise requires ~10-12 file downloads and the Exercism API enforces rate limits across all of them.

Steps to Reproduce

  1. Download several exercises in quick succession:
    for name in bird-watcher two-fer hello-world lasagna matching-brackets; do
  exercism download --exercise="$name" --track=javascript --force
done
  2. Check the file sizes in the downloaded directories:
    ls -la ~/Exercism/javascript/bird-watcher/

Expected Behavior

  • The CLI should respect the Retry-After header and retry failed file downloads
  • At minimum, the CLI should fail with an error message when file downloads return 429, rather than silently creating empty files
  • The final “Downloaded to” message should not be printed if files were not actually written

Actual Behavior

  • The initial solution metadata request (to api.exercism.org) succeeds with HTTP 200
  • All subsequent per-file requests (to exercism.org/api/v1/solutions/<id>/files/<filename>) return HTTP 429 with Retry-After: 23
  • The CLI creates the directory structure and all files, but every file is 0 bytes
  • The CLI prints “Downloaded to /path/to/exercise” as if it succeeded
  • Exit code is 0 (success)

Verbose Output Evidence

With --verbose, the per-file requests clearly show 429 responses:

GET /api/v1/solutions/<id>/files/package.json HTTP/1.1
HTTP/2.0 429 Too Many Requests
Retry-After: 23
Content-Type: text/plain
Retry later

Yet the CLI still reports success.

Environment

  • CLI version: 3.5.8
  • OS: macOS 15.5 (Darwin 24.5.0, Apple Silicon)
  • Track: javascript (but likely affects all tracks)

Impact

This bug is especially problematic because:

  1. The silent failure makes it hard to diagnose — users see “Downloaded” and assume success
  2. Running tests on 0-byte exercise files produces confusing, unrelated errors
  3. The --force flag compounds the issue by deleting existing valid files before the (failing) re-download
1 Like
2

Many thanks for the detailed report.

The relevant code is in exercism/cli cmd/download.go

1 Like
3

This should help.

2 Likes
4

Thanks both, that was fast, and the fix looks right: returning an error instead of continue means the failure surfaces where it should.

One note for anyone finding this later. The merged change covers the second of the two behaviours I asked for,the CLI now errors out on a non-200 instead of writing empty files. Retrying on Retry-After isn’t part of it, so bulk downloads will still hit the rate limit; they’ll just fail loudly now instead of silently. That’s the important half, and arguably the right scope for one PR. If retry is worth having, that’s probably a separate issue rather than something to reopen here.

Also worth checking if you downloaded during a 429 burst on 3.5.8 or earlier, you may have 0-byte files already on disk:

find ~/Exercism -name "*.js" -size 0 -not -path "*/node_modules/*"

Re-download any hits with --force and a sleep between calls.

Happy to see this closed.

1 Like
5

Yup. Retry handling is a bit more tricky and involved. I may take a crack at it at some point.

The zero-byte write is both a simpler fix and a worse bug :slight_smile:

We also need to release a new version of the CLI at some point.

3 Likes