Track-specific testing documentation

batch/exercises/shared/.docs/tests.md at main · exercism/batch · GitHub and batch/docs/TESTS.md at main · exercism/batch · GitHub could use some further explanation of what failing or passing tests look like, preferably with some code examples.

Does this track have skipped tests? If so, how do I enable or disable them when working locally? How does a skipped test look in the test suite vs. an unskipped test?

Do I need to be in a batch shell to run the test suite?

The test docs and the Exercism CLI use call but that doesn’t seem to work on Windows 11 inside the command prompt.

Running tests via `call HelloWorldTest.bat`

2024/09/04 21:31:16 Failed to get error from failed subcommand: exec: "call": executable file not found in %PATH%
C:\Users\anagy\Documents\Exercism\batch\hello-world>HelloWorldTest.bat
Test failed: Your output have to be 'Hello, World!'

In the current test suite output, I’m told what tests failed, but I don’t know which tests passed since the names of passing tests aren’t logged. I also can’t tell why a particular test may have failed. It would be useful to have the expected values and the actual results logged for each test.

C:\Users\anagy\Documents\Exercism\batch\darts>DartsTest.bat
Test passed
Test passed
Test failed: On the middle circle.
Test failed: On the inner circle.
Test failed: Exactly on center.
Test failed: Asymmetric position between the inner and middle circles.

It would be handy to have a line at the end that says X tests passed, Y tests failed. That lets me know I’m making progress at a glance without checking specific tests.

Hi BNAndras, yes it a great idea.

Does this track have skipped tests?

Yes this track has some skipped tests, for an example

If so, how do I enable or disable them when working locally?

Remove REM keyword from lines of testing code (REM means comment in batch)

How does a skipped test look in the test suite vs. an unskipped test?

Actually as I said, if a test code lines starts with REM It’s means it’s commented so not will be executed.

Do I need to be in a batch shell to run the test suite?

yes, it is an imperative of language, also batch shell is means default windows cmd(terminal) so by default build-in in windows.

that doesn’t seem to work on Windows 11 inside the command prompt.

It’s quite not possible, Are you using any tool(like nushell) that affects your terminal because if you enter an incorrect command, you normally get an error like this

C:\Users\Groop>callx
'callx' is not recognized as an internal or external command,
operable program or batch file.

Thank you for feedback, It would be cool to users will know what be passed, I’ll update as soon as.


That will be great tho

Thank you for all feedback :heart:

Can that be documented in the track docs? I noticed that darts doesn’t have skipped tests, but it’s customary that tracks that can should skip tests after the first one when run locally. That lets the student focus on passing a single test at a time and then enabling the next one.

No, it’s just the regular command. I don’t have access to the Windows 11 computer any more, but I can still replicate the problem with exercism test on my other Windows 10 computer.

exercism test

C:\Users\andras\Documents\Exercism\batch\darts>exercism test
Running tests via `call DartsTest.bat`
2024/09/16 21:03:06 Failed to get error from failed subcommand: exec: "call": executable file not found in %PATH%  

call DartsTest.bat

C:\Users\andras\Documents\Exercism\batch\darts>call DartsTest.bat
Test passed
Test passed
Test passed
Test passed
Test passed
Test passed
Test passed

call .\DartsTest.bat forces a student to use Command Prompt on this track. For me, this is suboptimal because my default terminal in VSCode is PowerShell. I need to manually create a Command Prompt tab in the Terminal section to run the test suite.

cmd /c DartsTest.bat works inside PowerShell and Command Prompt. If I change the test configuration for Batch to use cmd /c {{test_files}}, exercism test works. In the below snippet, I rebuilt the exercism CLI with this fix.

PS C:\Users\andras\Documents\Exercism\batch\darts> .\exercism.exe test
Running tests via `cmd /c DartsTest.bat`

Test passed
Test passed
Test passed
Test passed
Test passed
Test passed
Test passed
1 Like

Thank you fixing that problem, you’re a lifesaver :heart: