Hi again!
I created a very simple tests runner for GDScript. I decided to create a custom solution based on assertions, not Godot Unit Test, as I’m not sure how many benefits the plugin would bring. The code should be refactored and cleaned, I just wanted to present my initial idea:
extends SceneTree
const ERROR_MESSAGE = "Expected output was '{}', actual output was '{}'."
func run_tests(solution, method_name, test_cases):
var test_results = []
for test_case in test_cases:
var test_name = test_case["test_name"]
var args = test_case["args"]
var expected = test_case["expected"]
# TODO: check for errors
var output = solution.callv(method_name, args)
var status = run_single_test(solution, method_name, args, expected)
var message = ERROR_MESSAGE.format([expected, output], "{}")
test_results.append({
"name": test_name,
"status": "pass" if status == OK else "fail",
"message": null if status == OK else message,
})
return test_results
func run_single_test(solution, method_name, args, expected):
var output = solution.callv(method_name, args)
assert(output == expected)
return OK
func _init():
# TODO: read names from params
var solution = preload("user_solution.gd").new()
var test_suite = preload("test_suite.gd").new()
var method_name = test_suite.METHOD_NAME
var test_cases = test_suite.TEST_CASES
var test_results = run_tests(solution, method_name, test_cases)
var results = {
"version": 2,
"tests": test_results,
"status": "pass"
}
# TODO: check for errors
for test_result in test_results:
if test_result["status"] == "fail":
results["status"] = "fail"
break
# TODO: save to results.json instead of printing
var pretty_results = JSON.print(results, " ")
print(pretty_results)
quit()
A single test suite could look like this:
# Assuming that we always test the same method in a given test suite
# If not, this can be moved to TEST_CASES
const METHOD_NAME = "add_2_numbers"
const TEST_CASES = [
{"test_name": "Test One", "args": [1, 2], "expected": 3},
{"test_name": "Test Two", "args": [10, 20], "expected": 30},
]
And an example of the user solution would be:
func add_2_numbers(a, b):
return 3
In this case, the solution is wrong, so the output from the test runner looks like this:
➜ test_runner ./Godot_v3.5.1-stable_linux_headless.64 -s test_runner.gd
Godot Engine v3.5.1.stable.official.6fed1ffa3 - https://godotengine.org
SCRIPT ERROR: Assertion failed.
at: run_single_test (res://test_runner.gd:31)
{
"version": 2,
"tests": [
{
"name": "Test One",
"status": "pass",
"message": null
},
{
"name": "Test Two",
"status": "fail",
"message": "Expected output was '30', actual output was '3'."
}
],
"status": "fail"
}
Godot Engine is oriented toward games, so it’s unusual to do things like running scripts in separation from 2D/3D scenes or running unit tests ;) It also doesn’t have a try ... catch mechanism, so my example lacks error detection (it only checks failures). I think the closest in GDScript I can get to detecting errors is splitting the code into many methods with assert() inside and checking if the results are ERR* or OK. I’ve tested this on a simple method and I think this approach could work. There is also a discussion here.
I’ve seen that Test Runner Interface also includes things like test_code and output. For the former, I think we could use Script.source_code, although in this parametrized version we can also build the test code from available pieces. For the latter, I will have to do some digging. There are things like Expression.execute but I’m not sure if they will work for custom methods.
Please let me know what do you think @Meatball . If this looks like a good approach, I think the next step will be to create a Docker version in a new repository. If you have any remarks or notice any issues, let’s discuss ;)
I will be traveling this weekend, so I will only get back to my laptop on Monday. My responses might be delayed until then ;)
)
My timezone is also CET, my OS is Ubuntu, I’m ok with either this forum or with Slack. In case of Slack, I will need an invitation though.