add common method for registering different test types

[spoonincode]

This is a change I was required to make for #1000, but with that effort being buried I thought I'd still cherry pick this change out for opinions on taking it stand alone.

This removes a lot of boilerplate when declaring tests (especially LR & NP tests) by removing, for example,

add_test(NAME nodeos_sanity_test COMMAND tests/nodeos_run_test.py -v --sanity-test ${UNSHARE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
set_property(TEST nodeos_sanity_test PROPERTY LABELS nonparallelizable_tests)

and replacing it with

add_np_test(NAME nodeos_run_test COMMAND tests/nodeos_run_test.py -v ${UNSHARE})

There is add_p_test(), add_np_test(), add_lr_test(), and add_wasmspec_test() that all do what you'd expect. Any place that continues to just use add_test() (like in a submodule) will still just keep working as expected.

add_p_test() is probably the most questionable, as it adds little utility over just add_test() since the parallelizable tests don't need a label. Though it does remove the WORKING_DIRECTORY ${CMAKE_BINARY_DIR} boilerplate used in spots.

Certainly a big fear with a change like this is inadvertently disabling or altering a test somehow. To ensure no test left behind, one thing we can look at is the number of tests printed in the CI run compared to last run on main. This is easiest to compare for parallelizable tests since there is a number printed at the end. A grep | wc can be used on the log files of NP & LR tests to ensure the count matches up.

Another more through check is to create json output from ctest before and after the change,

ctest --show-only=json-v1 > before.json
ctest --show-only=json-v1 > after.json

Unfortunately there is a bunch of changes in the json due to "backtrace" related stuff (a graph of tests), so filter that out,

cat before.json | jq 'del(.version)|del(.backtraceGraph)|del(.tests[].backtrace)' > before-filtered.json
cat after.json | jq 'del(.version)|del(.backtraceGraph)|del(.tests[].backtrace)' > after-filtered.json

Now can do a diff -u1 before-filtered.json after-filtered.json. Unfortunately again this still a bit large because of changes such as

@@ -46872,3 +46872,3 @@
           "name": "WORKING_DIRECTORY",
-          "value": "/spring/build/unittests/wasm-spec-tests/generated-tests"
+          "value": "/spring/build"
         }

At this point I just scrolled through the output and manually verified the only thing in the diff was WORKING_DIRECTORY changes (due to the common WORKING_DIRECTORY "${CMAKE_BINARY_DIR}" now present that wasn't the case with every test in the past). Clearly the working directory being switch from, for example, /spring/build/unittests/wasm-spec-tests/generated-tests to /spring/build hasn't affected the ability of the test to run.

One other minor annoyance with this change I noticed is that the indirection removes the ▶️ from CLion's editor next to the add_test() to easily run the test.

[heifner]

Looks like an improvement to me.

[greg7mdp]

lgtm

[ericpassmore]

Note:start
category: Tests
component: Internal
summary: Improve test helpers by adding common method for registering different test types.
Note:end