Tests pass locally but fail upon submission without an explanation

I am solving the Knapsack assignment. I have submitted multiple iterations. Each submission passes all tests locally but fails when submitted. No explanation is given:

image

image

I have searched the forum, but I am not finding a clear answer.

You might be hitting the test timeout based on the durations in that screenshot.

What’s the full code for your knapsack solution? We can run it against the Docker image for the test runner and see if there’s anything diagnostic.

module Knapsack

type Item = { weight: int; value: int }

let maximumValue items maximumWeight = 
    //thanks to EspenBerget's solution for the combinations function:
    let rec combinations list =
        match list with
        | [] -> [[]]  
        | head :: tail ->
            let includeH = combinations tail |> List.map (fun combo -> head :: combo)
            let excludeH = combinations tail 
            includeH @ excludeH

    items
    //find all combinations of items
    |> combinations
    //ensure only valid weights are considered
    |> Seq.filter (fun pack -> pack |> List.sumBy (fun item -> item.weight) <= maximumWeight)
    //then find the maximum value among them
    |> Seq.maxBy (fun pack -> pack |> List.sumBy (fun item -> item.value)) 
    //finally sum the values of the items in that pack
    |> Seq.sumBy (fun item -> item.value)

Alas, the test runner, /opt/test-runner/Exercism.TestRunner.FSharp doesn’t output very much.

 # /opt/test-runner/Exercism.TestRunner.FSharp knapsack /solution /output
[2026-06-15 18:51:04Z] Running test runner for 'knapsack' solution...
[2026-06-15 18:51:10Z] Ran test runner for 'knapsack' solution

# cat /output/results.json | tr \\n \ ; echo
{   "version": 3,   "status": "error",   "tests": [] }

# wc -l msbuild.log
0 msbuild.log

I’m not sure what’s going on … but it looks like maybe it’s not picking up the tests for some reason?

/solution # dotnet test
Restore succeeded with 1 warning(s) in 0.6s
    /solution/Knapsack.fsproj : warning NU1900: Error occurred while getting package vulnerability data: Unable to load the service index for source https://api.nuget.org/v3/index.json.
  Knapsack net10.0 succeeded with 1 warning(s) (2.2s) → bin/Debug/net10.0/Knapsack.dll
    /solution/Knapsack.fsproj : warning NU1900: Error occurred while getting package vulnerability data: Unable to load the service index for source https://api.nuget.org/v3/index.json.
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.3+1b45f5407b (64-bit .NET 10.0.3)
Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.
  Knapsack test net10.0 succeeded with 1 warning(s) (1.4s)
    /usr/share/dotnet/sdk/10.0.103/Microsoft.TestPlatform.targets(48,5): warning No test is available in /solution/bin/Debug/net10.0/Knapsack.dll.
Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Test summary: total: 0, failed: 0, succeeded: 0, skipped: 0, duration: 1.3s
Build succeeded with 3 warning(s) in 4.7s

I also visited the Affine Cipher exercise that I had solved years ago. I updated it, and now those tests fail similarly–no explanation. I am accustomed to the need to make small reworks after such updates, but without these test failure explanations, I am left to guess at the issues.

I updated other exercises in the same way. Some pass; others fail without a reason. Is this a problem with the F# test runner?

Failing without any explanation is definitely something the test runner ideally wouldn’t do.
+cc @ErikSchierboom

You’re using an older adapter version than OP’s screenshot. Yours lines up with fsharp/exercises/practice/knapsack/Knapsack.fsproj at 2ede51720a083142f4812f82ffcf6d7e54fd3702 · exercism/fsharp · GitHub so maybe the .fsproj needs to be updated. Looking at other .fsproj files, they have 3.x versions listed for the adapter.

@IsaacG , I notice in the test runner screenshot you provided that the test adapter is using v2.4.3. I think these are out of date. When I look at the “See what’s changed” alerts in the UI, most show something like this.

1 Like

Affine Cipher has the same issue.

fsharp/exercises/practice/affine-cipher/AffineCipher.fsproj at 2ede51720a083142f4812f82ffcf6d7e54fd3702 · exercism/fsharp · GitHub uses an old version of the test detector that doesn’t support xUnit 3 tests. What might be happening is that the test detector runs and fails to see any compatible tests to run. No tests to run means there’s nothing to report so that’s why we don’t get anything shown in the online editor.

However, the one mystery is that the original screenshot shows a newer version for this package. Did you modify the .fsproj file? dotnet restore should have pulled the same version as the one Isaac got.

I use the link in the UI to download locally. I typically use VSCode locally as my editor. At times, immediately after download, the test runner fails to recognize the tests provided. I noticed the .fsproj file supplied from the UI download uses a beta version of packages and does not match the versions shown in “See what’s changed”. If I update the .fsproj to match those, the test runner finds the tests and I am good…locally. With some exercises, tests that pass locally still do not pass when submitted.

I have also tried copying/pasting my solution into the UI editor and using it to run the tests. In those cases where I have had to update .fsproj, the tests usually fail in the UI editor.

Did I answer your question, @BNAndras ?

1 Like

Yeah, so you updated the .fsproj locally. That lines up with what I was thinking may be the issue. Thanks! We’ll see what Erik says.

These exercises all target v2.4.3 of xunit.runner.visualstudio. I checked a couple in the online editor and they also give that empty error as originally reported.

  1. Affine Cipher
  2. Bottle Song
  3. Eliud’s Eggs
  4. Flower Field
  5. Killer Sudoku Helper
  6. Knapsack
  7. Resistor Color
  8. Resistor Color Duo
  9. Resistor Color Trio
  10. Satellite
  11. Square Root
  12. State Of Tic Tac Toe

Well, I wrote up a PR to correct those.

2 Likes

@birdbrain2734 do the tests work with the updated package version?

Yes, Knapsack works now. I did not go through all the exercises, but those I checked now work. Thank you so much for the updates

2 Likes