Updating Representers

Exercism Building Exercism
1

Our new representers mean that mentor’s can give feedback once but have it delivered to many solutions in the future.

However, when a representer is updated to produce different output, all of the existing feedback ceases to be automatically delivered to students. This is highly problematic as it means that mentors need to repeatedly give identical feedback to the same solutions.

This is the solution we are building to address this.

  1. A new representer is deployed, triggering a processing script.
  2. The script goes through all submissions that have been given feedback via the automation UI, and generates new representations for them.
  3. For each new representation, two new versions fields are checked (see below):
    a) If the version is the same, the existing feedback is copied onto the new representation and that representation is considered as one having received feedback.
    b) If the version is the same, the existing feedback is copied as a draft on the new representation, and the submission reappears in the queue.

The two version fields are:

  1. In a track’s config.json, we add a representer_version key to an exercise (if it’s missing, it defaults to 1).
  2. In the output of a representer, we add a version key which specifies the version of the representer used for this (the exact location and key to be decided but I’m leaning to {version: 1} in a results.json file. The file or key missing would also default to 1.

Most of the work for this is now done. The remaining steps are:

  • [ ] The automatic triggering on a representer update.
  • [ ] Finalizing (and reading) the versions.
  • [ ] Updating docs.

When we launch, we’ll re-run all feedback that’s been given back through the system.

WIP PR is here: Handle representers being updated by iHiD · Pull Request #3041 · exercism/website · GitHub

2 Likes