That would make the action slightly less useful for that particular track, since they already have the opportunity to review the files before merging the PR, which also acts as a final checkpoint to ensure everything looks good before accepting the changes from the GHA.
That said, if a track prefers issues over PRs, the change can be made quite easily!
I generally see a bit of an issue with the metadata sync. Since that will sync the toml file. Toml files are really good for keep track of whether a test file is updated or not. I have seen tracks which just have synced them so there is you there is no easy way of telling that the tests are out of sync. You could go back in the commits and see that the toml file was updated and not the test file, but that is annoying and a lot of just extra work (or the test file will just stay outdated). I suppose it is here that the issue system comes into the picture. But for that to work well someone actually has to take care of those issues; otherwise, the test might be unsynced to the docs, or issues stack on top of each other. I am thereby generally against this being implemented exercism-wide, but if individual maintainers want this, it is up to them. I think it is better to let it be out of date until someone takes a real sweep and updates the test files also.
The tests.toml file is only affected if one uses configlet sync with the --tests [mode] option. When I (and possibly Jagdish) refer to syncing metadata, I mean configlet sync --metadata which would sync the .meta/config.json files for each exercise.
Syncing tests would be almost certainly out of scope out of necessity. Not all tracks have generators in the first place, and those that do don’t necessarily have a standard API for the GHA to hook into.
Sure, but if you don’t know things are out of sync for a given repo, would you stop and check if it is? It’s a multiple-step process to fork the repo, fetch configlet, check if something needs to be synced, and then sync it. I’m not sure I would think to do it unless I already had the repo locally.
I mean if you do it like once every 6 months, there is very likely something out of sync. I have never done a sync and there hasn’t been any changes so I can’t say I have experienced that. I also often just spin up a codespace for this kinds of stuff. But if it doesn’t update the toml file i guess it is fine for unmaintained tracks.
As @BNAndras said, when I refer to the automatic PR, this command is used in the background: configlet sync --docs --metadata --filepaths -u -y. Hence the test.toml files are not updated with the automatic PR but rather an issue is raised mentioning all the exercises that have missing or reimplemented tests, so it’s up to someone to come and manually update the test cases after updating the test.toml for each exercise.
I’ve pushed a commit with the necessary changes, the workflow will now avoid creating duplicate issues. If an issue already exists, it will simply update the existing one instead of opening a new one.
I’d also like to have such an Exercism wide GHA, but would prefer having an allow- or deny-list of exercises to sync.
PHP track has it’s own script for sync’ing, reading the list of exercises to sync from a file. This allows us to have appr. 50% of the exercises to sync automatically while the rest is still outdated too much for configlet sync.
The PR on the Java track has been successfully merged, and the testing has been completed by @kahgoh, and everything has been found to be working as expected.
This updated workflow is now fully standalone and functions independently without relying on additional directories, unlike the earlier version. It has also been refined to follow the Exercism GitHub Actions best practices, notably by pinning the runner OS and setting an appropriate timeout.
The workflow has been split into two distinct jobs:
One for syncing docs, metadata, and filepaths, and raising a PR.
Another for checking unsynced test files and raising an issue.
This makes it modular, so maintainers like @IsaacG who are interested in only one part can remove the other, and tracks that already have existing systems (e.g., @BethanyG’s) can opt out entirely. As I understand, using all actions from the github-actions repo is not mandatory.
The workflow is scheduled to run monthly (currently on the 15th), but this frequency can be easily adjusted by modifying the cron field in the workflow file. It has built-in safeguards to avoid redundancy. It will only raise a PR if changes are detected and will not create duplicate issues for tests.
That’s the full proposal, happy to hear any feedback!
Finally, the workflow is designed to run monthly (on the 15th) without raising duplicate issues. It raises a PR only if syncing is needed.
I know we’re starting with running it on 15th of each month on the Java track, but wouldn’t each track decide when to run the action - similar to how the tracks schedule the configlet action in their github workflow (Java’s configlet action config for example)?