Test explorer: Tests not showing up (Visual Studio 2022) C++ Track

Hi everyone,

I’m currently working on the pangram excercise from the C+±Track. I’m having problems with the test-suite.

The tests do not show up in the test explorer, so I cannot run the tests from there. If, however, I run pangram_tests.cpp and the tests succeeds, a console window pops up telling me my tests passed. But if a test fails, all I get is a fail-message in the output-tab.

I’m using Visual Studio Community 2022, version 17.4.3 with the CMake tools installed.
I open the exercise by clicking File->Open->Cmake and then select the CMakeLists.txt.
Clicking ‘Run all’ in the test explorer doesn’t help. Also doing a rightclick on CMakeList.txt and building it does not resolve the issue.

Is it even possible to make the tests show up in the test explorer?

Thank you and kind regards,

-Tom

It’s been a long time since I used Visual Studio extensively. From what I remember the test explorer was very helpful for projects in languages like C# where some kind of introspection mechanism allowed the tests to be discovered automatically.
C++ does not have that so you would need to create some kind of XML file yourself or install a plugin like the “Test Adapter for Catch2”.

Currently the CMakeLists.txt file builds the tests and subsequently executes them. I think that was done to avoid introducing a separate step for running the tests, and that way it works the same way on all platforms.

If you want to use the test explorer I fear you’re on your own


What works for me with VisualStudio 2019 is following CMake projects in Visual Studio: Open Visual Studio, click “Open a local folder”, select the folder with exercise (for me that’s C:\Users\matth\Code\exercism.io\cpp\pangram), wait for it to finish loading, and then select “Build” → “Build Solution”.
That builds and executes the tests and I either get an error that looks like this:

>------ Alle erstellen gestartet: Projekt: pangram, Konfiguration: x64-Debug ------
  [1/4] Building CXX object CMakeFiles\pangram.dir\pangram.cpp.obj
  [2/4] Building CXX object CMakeFiles\pangram.dir\pangram_test.cpp.obj
  [3/4] Linking CXX executable pangram.exe
  [4/4] cmd.exe /C "cd /D C:\Users\matth\Code\exercism.io\cpp\pangram\out\build\x64-Debug && C:\Users\matth\Code\exercism.io\cpp\pangram\out\build\x64-Debug\pangram.exe"
  FAILED: CMakeFiles/test_pangram
  cmd.exe /C "cd /D C:\Users\matth\Code\exercism.io\cpp\pangram\out\build\x64-Debug && C:\Users\matth\Code\exercism.io\cpp\pangram\out\build\x64-Debug\pangram.exe"


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  pangram.exe is a Catch v2.13.6 host application.
  Run with -? for options


-------------------------------------------------------------------------------
  sentence_empty

-------------------------------------------------------------------------------
  ..\..\..\pangram_test.cpp(8)

...............................................................................

  ..\..\..\pangram_test.cpp(10): FAILED:
    REQUIRE( !pangram::is_pangram("") )
  with expansion:
    false


-------------------------------------------------------------------------------
  pangram_with_only_lower_case

-------------------------------------------------------------------------------
  ..\..\..\pangram_test.cpp(15)

...............................................................................

  ..\..\..\pangram_test.cpp(17): FAILED:
    REQUIRE( pangram::is_pangram("the quick brown fox jumps over the lazy dog") )
  with expansion:
    false


-------------------------------------------------------------------------------
  missing_character_x

-------------------------------------------------------------------------------
  ..\..\..\pangram_test.cpp(20)

...............................................................................

  ..\..\..\pangram_test.cpp(22): FAILED:
    REQUIRE( !pangram::is_pangram("a quick movement of the enemy will jeopardize five gunboats") )
  with expansion:
    false


-------------------------------------------------------------------------------
  another_missing_x

-------------------------------------------------------------------------------
  ..\..\..\pangram_test.cpp(25)

...............................................................................

  ..\..\..\pangram_test.cpp(27): FAILED:
    REQUIRE( !pangram::is_pangram("the quick brown fish jumps over the lazy dog") )
  with expansion:
    false


-------------------------------------------------------------------------------
  pangram_with_underscores

-------------------------------------------------------------------------------
  ..\..\..\pangram_test.cpp(30)

...............................................................................

  ..\..\..\pangram_test.cpp(32): FAILED:
    REQUIRE( pangram::is_pangram("the_quick_brown_fox_jumps_over_the_lazy_dog") )
  with expansion:
    false


-------------------------------------------------------------------------------
  pangram_with_numbers

-------------------------------------------------------------------------------
  ..\..\..\pangram_test.cpp(35)

...............................................................................

  ..\..\..\pangram_test.cpp(37): FAILED:
    REQUIRE( pangram::is_pangram("the 1 quick brown fox jumps over the 2 lazy dogs") )
  with expansion:
    false


-------------------------------------------------------------------------------
  missing_letters_replaced_with_numbers

-------------------------------------------------------------------------------
  ..\..\..\pangram_test.cpp(40)

...............................................................................

  ..\..\..\pangram_test.cpp(42): FAILED:
    REQUIRE( !pangram::is_pangram("7h3 qu1ck brown fox jumps ov3r 7h3 lazy dog") )
  with expansion:
    false


-------------------------------------------------------------------------------
  pangram_with_mixed_case_and_punctuation

-------------------------------------------------------------------------------
  ..\..\..\pangram_test.cpp(45)

...............................................................................

  ..\..\..\pangram_test.cpp(47): FAILED:
    REQUIRE( pangram::is_pangram("\"Five quacking Zephyrs jolt my wax bed.\"") )
  with expansion:
    false


-------------------------------------------------------------------------------
  upper_and_lower_should_not_be_counted_separately

-------------------------------------------------------------------------------
  ..\..\..\pangram_test.cpp(50)

...............................................................................

  ..\..\..\pangram_test.cpp(52): FAILED:
    REQUIRE( !pangram::is_pangram("the quick brown fox jumps over with lazy FX") )
  with expansion:
    false


===============================================================================
  test cases: 9 | 9 failed
  assertions: 9 | 9 failed

  ninja: build stopped: subcommand failed.

Alle erstellen fehlgeschlagen.

Or if all tests pass:

>------ Alle erstellen gestartet: Projekt: pangram, Konfiguration: x64-Debug ------
  [1/3] Building CXX object CMakeFiles\pangram.dir\pangram.cpp.obj
  [2/3] Linking CXX executable pangram.exe
  [3/3] cmd.exe /C "cd /D C:\Users\matth\Code\exercism.io\cpp\pangram\out\build\x64-Debug && C:\Users\matth\Code\exercism.io\cpp\pangram\out\build\x64-Debug\pangram.exe"

===============================================================================
  All tests passed (9 assertions in 9 test cases)


Alle erstellen erfolgreich. 

I’m German, you will probably get something similar in English.


One more thing:

Exercism promotes Test Driven Development (TDD). In the original setup only the first test is enabled and the other tests are disabled by the #if defined(EXERCISM_RUN_ALL_TESTS) macro.

The intended use is that you just do enough coding that the current test succeeds, refactor the code to keep it clean, and then proceed to the next test by moving the #if defined(EXERCISM_RUN_ALL_TESTS) line one test further.

In a real TDD setting you would write the tests yourself. One test at a time, first making sure that it fails, then doing just enough coding that it succeeds, and finally refactoring it to remove inefficiencies, duplications, etc. That whole process is often called the “Red-Green-Refactor cycle”.


Does that help?

Hi,

thank you for replying. Yes, this is what I ended up doing.
Too bad there’s no easy way to get it working.

Regard,

-Tom