Related to Online Editor for CFML Track - #2 by BNAndras, but running the CFML test runner locally in Docker, it takes about eight seconds to spin up everything before student code is run. I’ve done some preliminary testing with BoxLang, which runs CFML code natively, and the spin up takes one or two seconds in Docker. I’m still fairly far from having a PR ready, but things look promising without much disturbance. The hope is students should be able to still use the legacy CommandBox setup or switch to the more modern BoxLang.
2 Likes
The PR has been merged, and the online editor is turned on and updated to v2. Multiple exercises pass so I think it’s safe to keep the editor enabled.
The CFML track gives students working locally a runner script, but that’s written for CommandBox. A BoxLang-equivalent should be provided. Docs will need to be updated at that point to reflect how to install BoxLang and run/interpret the tests being reported locally.
It’s great to hear that you seem to have fixed the online test runner! I hope this rather moribund track can be revitalized: I’m wildly out of practice, but have fond memories of Adobe ColdFusion back in the 2003-2011 period.
Clearly, local testing still needs work. I just installed BoxLang plus the optional CommandBox, but have so far failed to re-run any of my old solutions locally (they’re fine in the online editor). Missing or renamed dependencies, version conflicts, the whole mess.
Time for more coffee, then I’ll try again. Any quick advice to get me started? Or should I just leave you in peace to work on this more systematically?
Update: running 3-year-old local tests is clearly hopeless, but new downloads are OK in CommandBox. I’ll look into the newer BoxLang testing, while cursing them for getting the marketing people to write so much of the “documentation”.
It’s a bit awkward, but you can use cfml-test-runner/bin/test-runner.bxs at main · exercism/cfml-test-runner · GitHub for now.
boxlang path/to/test-runner.bxs path/to/solution/folder path/to/output/folder should kick out a v2 results.json after running the tests as well.
The two dependencies you’ll need are
FORGEBOX: TestBox and FORGEBOX: Globber
1 Like
I think it will need a bit more wiring to let test-runner.bxs find all the necessary library functions.
FYI, some existing solutions for Grains no longer pass in the online editor. Maybe not surprising if you look at the tests, which switch between integers (32-bit in CFML, more in java.math.BigInteger) and their string representation. I guess BoxLang is a bit stricter than 2017-vintage CFML, used to create the (now failing) .meta/Example.cfc.
If I was the maintainer (I’m not), I might be tempted to deprecate this exercise. Only 9 completions, and at least some of these now fail.
I was a maintainer for a while several years back, but it’s currently unmaintained. Personally, I wouldn’t deprecate this exercise since it’s a way to show some of the limitations of CFML. We did something similar for Vim script and then bumped the exercise difficulty up.
Does that mean something is missing from test-runner.bxs and needs to be added to the CFML test runner? Or that the file isn’t not quite ready for students to use offline? The latter isn’t surprising since an updated offline runner is a WIP.
After I removed everything box-related and reinstalled from scratch, this is the output (for Darts, on Linux Mint):
{
"message": "The requested class [testbox/solutionsystem/solutionTestBox] has not been located in any class resolver/solution",
"version": 2,
"status": "error"
}
Like you, I’m not at all shocked that this didn’t work unmodified. Good luck with the WIP offline runner!
1 Like
It’s nobody’s priority at present, but if the CFML track migrates to BoxLang tooling, presumably if would be relatively easy for somebody (the mythical “somebody”) to create a BoxLang track with modern syntax and similar tooling.
Add it to the long list of things Exercism could do, if given more people and more money…
boxlang.json
{
"mappings": {
"/testbox": "${user-home}/.boxlang/modules/testbox",
"/exercise": "${user-dir}"
}
}
TestRunner.bxs
exerciseDir = getDirectoryFromPath( getCurrentTemplatePath() );
// Hide an existing testbox (potentially from TestRunner.cfc) to avoid accessing it
localTestBox = exerciseDir & "testbox";
hiddenTestBox = exerciseDir & ".testbox_cfml";
hasConflict = directoryExists( localTestBox );
if( hasConflict ) directoryRename( localTestBox, hiddenTestBox );
exitCode = 0;
try {
testBox = new testbox.system.TestBox( reporter = "console" );
testFiles = directoryList( exerciseDir, false, "name", "*Test.cfc" );
for( file in testFiles ) {
testBox.addBundles( "exercise." & file.listFirst( "." ) );
}
results = testBox.runRaw();
new testbox.system.reports.ConsoleReporter().runReport( results, testBox, {}, true );
if( results.getTotalFail() > 0 || results.getTotalError() > 0 ) {
exitCode = 1;
}
} catch( any e ) {
systemOutput( "ERROR: " & e.message, true );
if( len( e.detail ?: "" ) ) systemOutput( e.detail, true );
exitCode = 1;
}
// Restore hidden testbox directory
if( hasConflict && directoryExists( hiddenTestBox ) ) {
directoryRename( hiddenTestBox, localTestBox );
}
if( exitCode ) {
createObject( "java", "java.lang.System" ).exit( 1 );
}
$ boxlang --bx-config boxlang.json TestRunner.bxs
█▓▒▒░░░ TestBox v7.0.0+19 ░░░▒▒▓█
? exercise.EtlTest (50 ms)
[Passed: 4] [Failed: 0] [Errors: 0] [Skipped: 0] [Suites/Specs: 1/4]
? My Etl class
? single letter (16 ms)
? single score with multiple letters (0 ms)
? multiple scores with multiple letters (8 ms)
? multiple scores with differing numbers of letters (0 ms)
================================================================================
Final Stats
================================================================================
[ ? Passed: 4 ]
[ ? Failed: 0 ]
[ ? Errors: 0 ]
[ ?? Skipped: 0 ]
[ ?? Duration: 138 ms ]
[ ? Bundles/Suites/Specs: 1/1/4 ]
[ ?? Labels: None]
================================================================================
TestBox: v7.0.0+19
Engine: BoxLang 1.12.0+53
It might be cleaner though to work within the existing Test Runner.cfc if both BoxLang and CommandBox play nicely with it somehow. Then the Exercism CLI doesn’t need to be updated.
Setup seems non-trivial. I keep getting this:
$ boxlang ./TestRunner.bxs
ERROR: The requested class [testbox.system.TestBox] has not been located in any class resolver.
Though TestBox was installed (in multiple places), ${user-home}/.boxlang/modules/ was empty until I ran
install-bx-module testbox
One success was running line 11 of TestRunner.bxs from the BoxLang REPL.
BoxLang> testBox = new testbox.system.TestBox( reporter = "console" );
boxgenerated.boxclass.testbox.system.Testbox$cfc@5f45b3f8
So this is presumably just (JUST!) a matter of sorting out file paths for each OS, and writing clear instructions for installation (Step 1: Ignore Google!).
After more coffee (and a couple of ibuprofen), I belatedly spotted that I was using the wrong command.
Now a different error message:
The requested class [testbox.system.modules.globber.models.PathPatternMatcher] has not been located in any class resolver
testbox and globber are both installed in ~/.boxlang/modules/.
Add local Boxlang test runners by BNAndras · Pull Request #322 · exercism/cfml · GitHub drops the old index.cfm runners that predate v3 and adds the Boxlang runners to each exercise. Doc updates need to be drafted to reflect both options being available. Unlike the online test runner, these local runners will print results to console.
As for the globber issue, installing the BoxLang modules as siblings normally makes sense, but Globber is an internal dependency for TestBox. It’s unclear why that’s not picked up, but in the test runner setup, I manually copied Globber in ~/.boxlang/modules/testbox/system/modules/ as a workaround. Updating the boxlang.json mappings to point /testbox/system/modules/globber there might be an option to consider. I haven’t tried that in the draft PR yet.