bats captures both stdout and stderr as the run command’s output.
However bats provides file descriptor 3 as a mechanism for tests to output uncaptured text. For TAP compliant output, this text must start with a #. Ref Printing to the terminal and File descriptor 3
All test cases for the jq track use
run jq ...
With a jqfunction that redirects debugging output from fd 2 to fd 3, and a small change to the test runner, I can enable debugging output in the same manner as ruby.
This will be a really nice addition for jq learners working online.
implementation
Each jq exercise gets a new file named bats-jq.bash
#!/usr/bin/env bash
# TODO some explanatory comments here ...
jq() {
command jq "$@" 2> >(
while IFS= read -r line; do
if [[ $line == '["DEBUG:",'* ]]; then
echo "# $line" >&3
else
echo "$line" >&2
fi
done
)
}
Every jq test-*.bats file gets a new “include” statement
load bash-jq
the result of 1 and 2 will not affect existing solutions: those will not contain any debug output (or else the tests would not have passed)
the test-runner is adapted to handle bats TAP output that might look like this
1..4
# ["DEBUG:","Input is: 10"]
ok 1 double a number
# ["DEBUG:","Input is: foo"]
ok 2 double a string successfully
# ["DEBUG:","Input is: foobar"]
not ok 3 double a string unexpected output
# (from function `assert_output' in file bats-extra.bash, line 394,
# in test file test-with_debug.bats, line 20)
# `assert_output '"foofoo"'' failed
#
# -- output differs --
# expected : "foofoo"
# actual : "foobarfoobar"
# --
#
# ["DEBUG:","Input is: [1,2,3]"]
ok 4 double an array is an error
with the debug output before the test result, and error output after.
This will help CLI users too: debug messages will no longer break the tests. The pretty bats output would look like (with colours)
$ command bats test-with_debug.bats
test-with_debug.bats
✓ double a number
["DEBUG:","Input is: 10"]
✓ double a string successfully
["DEBUG:","Input is: foo"]
✗ double a string unexpected output
["DEBUG:","Input is: foobar"]
(from function `assert_output' in file bats-extra.bash, line 394,
in test file test-with_debug.bats, line 20)
`assert_output '"foofoo"'' failed
-- output differs --
expected : "foofoo"
actual : "foobarfoobar"
--
✓ double an array is an error
["DEBUG:","Input is: [1,2,3]"]
4 tests, 1 failure
I was going to mention that in regards to the Ruby track, the debug breaks for anyone that downloads with that debug statement there, and i would be great to not have that happen. The jq language as a debug function that exists and so it should end up being uniform for all users, rather than providing only a benefit for those using the Exercism platform test runner/reporter.
@ihid On the track-specific docs page, what determines the order of the left-hand panel? It’s not the same order as the config.json file. I notice the same is try for the Python docs.
I think it’s unordered, so I guess whatever the database returns. I’d have thought it would be primary key but that doesn’t align with the results. We should probably align it to the config.json - one for @ErikSchierboom when he’s back
