HLSL track. Seeking feedback

Exercism Support
1

Is it Turing complete? Yes, for modern HLSL. @SleeplessByte was right about Shader Model 1.0/2.0, but those limits went away in SM 3.0 back in 2004. Shader Model 6.x has unbounded loops, user-defined functions, and read-write memory through RWByteAddressBuffer/RWStructuredBuffer. It’s an imperative language with an unusual execution model, not a fixed-function pipeline description.

I’ve already built it:

:point_right: GitHub - lheintzmann1/exercism-hlsl · GitHub

  • 40 practice exercises, 409 test cases
  • Working Dockerised test runner
  • configlet lint passes
  • CI proves every exercise’s example solution passes and every stub fails rather than erroring
  • active is false, awaiting your call

How the tests actually run

This was the part I expected to be blocking, and it isn’t. Solutions compile with Microsoft’s DXC targeting SPIR-V, then execute as a Vulkan compute shader on Mesa’s lavapipe, a CPU implementation of Vulkan.

No GPU is needed anywhere, in CI or on Exercism’s infrastructure. A full exercise runs in about 0.15s against the 20s budget.

The design decision I’d most like feedback on

Everything runs as a compute shader, including the graphics exercises.

That sounds like a dodge, but a fullscreen pixel shader genuinely is a compute kernel mapped over a 2D domain, that’s what Shadertoy is. So an exercise can ask for float CircleDistance(float2 p, ...), sample it over a grid from a compute shader, and compare the results numerically with a tolerance.

That means real graphics content graded deterministically, with a failure message naming the exact input, instead of image diffing and its flakiness. Students write shader maths; they never fight a screenshot comparison.

Curriculum

Two halves, roughly “the language” then “the graphics”:

19 language exercises from problem-specifications: leap, luhn, matching-brackets, atbash-cipher and so on. HLSL has no string type, so text is character codes in fixed-size buffers, much the same shape of solution the WebAssembly track uses.

21 graphics exercises specific to the track: signed distance fields and their boolean ops, sRGB/linear and ACES tone mapping, Lambert and Schlick’s Fresnel, normal-map unpacking and TBN, an integer hash into value noise into fBm, ray-sphere and ray-box, and sphere tracing. These are modelled on functions that actually appear in engine source, so what a student writes resembles what they’d read in a renderer.

Things I already know are weak

I’d rather raise these than have you find them:

  1. Local setup is the worst part. Students need a Vulkan loader and a small host binary that dispatches the shader. There’s a one-command setup script, prebuilt-binary plans and a Docker fallback, but it isn’t “install one package”. This is genuinely worse than most tracks and I’d welcome advice.
  2. No concept exercises yet, and no representer or analyzer.
  3. A single maintainer. I’d want a second before launch.
  4. HLSL has no official logo, so an icon needs designing from scratch rather than adapting one.

What I’m asking

Mainly: does the team want this track at all? I’m aware of the “no tools, libraries, frameworks or technologies” rule, and that a shading language sits near that line, though MIPS Assembly, WebAssembly and Vim script suggest the line is about whether you can actually learn programming in it, which I think HLSL clears.

If yes, I’d like a repo under exercism/ and I’ll keep going. If there are blockers I haven’t spotted, I’d much rather hear them now than after another 40 exercises.

Happy to walk through any of the tooling decisions, they’re all documented in the repo README.

1 Like
2

Impressive work.

The Futhark track currently only has exercises from problem-specifications. Perhaps the tracks can learn from each other.

A hint for any future tracks: You might like to use exercism/generic-track and exercism/generic-test-runner as starting points.

3

While “kicking the tyres” I asked Claude what the test runner might look like if it used gawk/jq/bash, like a few other tracks, allowing python to be omitted from the docker image.

prototype-hlsl-test-runner

4

A tiny observation:

I think parts of

are currently worded for a contributor who has access to the track repo.

Those parts could move to README.md or a new CONTRIBUTING.md, or new pages linked from those.

The intended audience for a track’s docs/INSTALLATION.md docs/TESTS.md exercises/shared/.docs/tests.md
contains students who might not know what git is, or not be aware of the existence of the track repo.

They are accessing the exercises through the website, or the Exercism CLI.

If they are using the CLI, they will locally have a directory like
~/exercism/hlsl/atbash-cipher/

5

Thanks, I rewrote the docs for students now, with the contributor bits moved to a CONTRIBUTING.md, and I’ve switched to your runner approach.

6

This likely is going to be a problem, but not insurmountable. The online editor is popular for a lot of people, but after a while folks likely want to work offline on the more involved exercises. If it’s not fairly straightforward to set up HLSL locally, I think adapting the test runner to also work locally (see GDScript) would work so students can focus on writing HLSL, not fighting the setup.

This isn’t a requirement for launch in any way. The representer and analyzer are completely optional even after launch. Concept exercises require a v2 or v3 test runner, but you can freely launch with a v1 test runner as long as you have multiple practice exercises.

Not a problem for proposing a track but yeah ideally someone else will be involved at least to help out. If we do go forward with this track, plenty of the regulars have launched tracks so we’re available if you have any questions at all.

This isn’t a problem for creating a track, and Exercism staff designs the track logos for consistency’s sake. You can even go live without a logo in place.

7

These sound like exercises intended to teach specific concepts related to one another and so would be more naturally organized as concept exercises, not practice exercises. Then, you’d have a track syllabus with interrelated exercises that unlock as you progress. Practice exercises are intended to be solved multiple ways and are available by default. That means a student could do them in any order desired, which might not be optimal. However, to have concept exercises, you need to have a v3 test runner first and a syllabus for them to reinforce. That’s a lot of work up front before launch.

8

That’s very helpful, thanks. It clears three of the four things I was worried about.

On local setup: I’ve followed the GDScript pattern. The runner now ships a script students keep next to their exercises:

cd ~/exercism/hlsl/leap
../test-local-hlsl-solution.sh

It takes the exercise from the directory name, uses a local toolchain if there is one and Docker otherwise, and prints readable results instead of raw JSON. Tested against a directory containing only the files the CLI actually delivers.

On concept exercises: you’re right that several have that shape, the noise ones are already chained, where each is handed the previous exercise’s result. Worth noting the test runner keiraville prototyped is already v3, so that prerequisite is mostly met.

My inclination is still to launch with practice exercises and build the syllabus afterwards, rather than hold the track back for it. But if the team would rather see a syllabus first, I’d sooner know now than after launch; happy to go either way.

9

Generally, practice exercises should be doable in any order at any time. You could lock a practice exercise behind a concept on the syllabus so it’s available once the concept exercise behind the concept is solved. That’s fairly common practice.

I suspect (and correct me if I’m wrong) you’re going by the exercise order in the track config. That affects the display order on Exercism, but that only works if a student goes one exercise at a time in order. However, we often tell students that the shown exercise order isn’t the order in which you must do exercises so that’s a fragile assumption.

If you do make them practice exercises but want them as concept exercises later, you’ll need to deprecate the practice exercises, keep their files in the repo, and then add the concept exercise version as a new exercise in the right locations. So that adds some maintenance work for you too.