Exercise "Pangram" has broken tests

Programming COBOL
1

This is the minimal example to break the tests:

        IDENTIFICATION DIVISION.
        PROGRAM-ID. PANGRAM.
        ENVIRONMENT DIVISION.
        DATA DIVISION.
        WORKING-STORAGE SECTION.
        01 WS-SENTENCE      PIC X(60).
        01 WS-RESULT        PIC 9.
        PROCEDURE DIVISION.
        PANGRAM.

      * Iteration 3: This is the minimum to break the tests.    
      
            goback.

grafik
grafik766×331 21.7 KB

2

@axtens added this exercise 2 years ago and might have an idea what’s wrong with the tests…maybe it’s another case of having the answer on the stack?

3

I expect it’s exactly that.

It’s on the list (along with a lot of other stuff), but I’ll give it a closer look in the next couple of days.

  1. 8th testing needs not to put the answer on the stack first
  2. cobol testing needs to have detect the presence of STOP RUN and GOBACK and complain about it. That or,
  • tweak cobolcheck to be able to handle those keywords, or
  • change to another unit test tool
4

So in COBOL it’s more like a test tool issue? In PHP we have a pre-step to lint the student code to prevent test tool breakage (see php-test-runner/bin/run.sh at ab11b7183ba3f1a382d0bed1049e802ddc785c3f · exercism/php-test-runner · GitHub). Maybe grep for those keywords before running the test tool?

5

Good idea. And someone even wrote me a demo of the idea. I just haven’t had the time.

1 Like
6

@ErikSchierboom or @iHiD, rather than fiddle with the test routines themselves could I have a part of test.ps1/test.sh that detects stoprun/gobacks the way the PHP linting works?

7

I don’t really know what you’re asking here to be honest.

8

Okay, I’ll try to explain. No guarantees.

cobol uses a 3rd party unit test tool, cobolcheck. Cobolcheck gives false positives if the code being tested has GOBACK or STOP RUN in it. We were all discussing the possibility of adding another step to the test-runner but I couldn’t get my head around it. Then I saw something being done on the PHP track that I thought might be easier to implement: linting.

So I’m asking, how hard would it be to add a linting pass to the cobol tests?

9

Not that hard, if you know a bit of bash. This is where the tests are executed: cobol-test-runner/bin/run.sh at main · exercism/cobol-test-runner · GitHub