That really looks great so far! Is it possible to generate PHPDocBlocks, too? These would help to link the UUIDs and test descriptions to the generated code.
There, of course, is much more complexity in some of the problem specifications. But it is a promissing POC.
Because I stumbled upon it right now: Advice for writing a test generator from the forum. It is not “documentation”, but contains helpful information about all the stuff around taking the JSON to output a test file. Many of your questions might already be answered there.
And be assured: Having a test generator in 6 months is better than having none forever.
One thing I would suggest the generator do is to be as “dumb” as possible. I went with a very complex setup for the C# generator, but that is much harder to write and maintain.
If I were to start again, I would go a different route, where the test generator is just a small shell around reading and pre-processing the JSON data from prob-specs and rendering that data via some templating system. I quite like what Crystal has done, where an exercise template looks something like this:
require "spec"
require "../src/*"
describe "<%-= to_capitalized(@json["exercise"].to_s) %>" do
<%- @json["cases"].as_a.each do |cases| %>
<%= status()%> "<%-= cases["description"] %>" do
<%= to_capitalized(@json["exercise"].to_s) %>.<%= cases["property"].to_s.underscore %>("<%= cases["input"]["phrase"].to_s %>").should eq("<%= cases["expected"].to_s %>")
end
<% end %>
end
