Link checker workflow

problem-specifications uses Lychee for link checking in the CI workflow

Also Dart, Factor, Moonscript and Tcl:

https://github.com/search?q=lycheeverse+org%3Aexercism+path%3A.github%2Fworkflows&type=code

It would be too disruptive for all the other tracks to add a link-checking workflow to the org-wide-files. But I think it would be helpful to add into the generic-track repo so that new tracks get it by default.

Thoughts?

1 Like

Org wide might be good :grin:
Barring that, adding it to the generic track would be good, too.

Perhaps it should be a monthly GitHub action instead of a workflow check.

stackoverflow.com and rosettacode.org added CAPTCHA or other blocking of lychee’s checks, and this meant PRs could not be merged until I added the domains to .lycheeignore

The website https://factorcode.org/ was down for a day or two, and this also blocked merges.

Even without Require status checks to pass before merging, workflow failures are distracting for reviewers.

3 Likes

I’d welcome a monthly link check in my repo. Maybe like the configlet-sync as a workflow to add or may be opt-in via .github/org-wide-files-config.toml?

You can copy .github/workflows/links.yml and .lycheeignore from the exercism/dart repo.

Add workflow to check links with Lychee by glennj · Pull Request #263 · exercism/github-actions

and then

Add link checker GHA by glennj · Pull Request #215 · exercism/generic-track

2 Likes

OK, the exercism/github-actions PR has been merged. If you, as a maintainer, want monthly link checking in your repo (and not run as something that will block CI), you can add this workflow:

name: Check Links

on:
  workflow_dispatch:
  schedule:
    - cron: 0 0 15 * * # 15th day of each month

jobs:
  call-gha-workflow:
    uses: exercism/github-actions/.github/workflows/link-checker.yml@main

It also can be manually dispatched to do link checking on demand.

I added it to moonscript and here’s what a manual dispatch looks like Links · exercism/moonscript@a4fc3b2

4 Likes

I forgot to mention: If you use this workflow, you should add a .lycheeignore file to the repo containing patterns for domains known to not accept bot requests. At minimum

exercism\.org

But stackoverflow, reddit and rosettacode also.

This is what happens without the .lycheeignore: Link Checker Report · Issue #398 · exercism/jq · GitHub

2 Likes

A handy bash function to run lychee checks locally without having to install anything (assuming you already have Docker installed)

lychee() {
    cd "$(git rev-parse --show-toplevel)" || return 1
    local args=(
        --cache
        --accept ..=299,429
        --suggest
        --verbose
        --require-https
        --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36"
        "/input/**/*.md" "/input/**/*.html" "/input/**/*.toml" "/input/**/*.json"
    )
    docker run --init --rm -it -v "$PWD":/input lycheeverse/lychee "${args[@]}"
}

(edit; added --rm to tidy up on completion)

2 Likes

Tried to add the link checker to PHP track. Using it locally provided these challenges:

  • lychee dives into installed libraries like vendor/ and there is no “exclude files by path” in the examples given here. I had to add lychee.toml and fill it with the ignore patterns (and add it to the docker args for local testing)
  • Exercism supports relative links into the website (/tracks/php/... and lychee cannot handle this without a configuration file, too. I added the base_url config entry, but it now ignores all links to local files (prefixed by base_url, which is exercism.org and ignored). I’m undecided whether to convert the relative links to ones including the domain or accepting file links to be ignored.
  • I have not yet tried if the config file is picked up by the action automatically. That might become another challenge.
  • Of course the first run did what is expected: Several links are dead and must be repaired / replaced. That’s a challenge to expect, but maintainers like me might not be prepared for the real number of findings…

Thanks for providing the action and all the useful information around it! It will take some time to overcome the obstacles…

1 Like