[check_swatinit] Tests fail on RHEL8 builds

[andreas-el]

Update:

The tests are now skipped instead:

SKIPPED [2] tests/test_check_swatinit_simulators.py:287: Test require flow dev version
SKIPPED [2] tests/test_check_swatinit_simulators.py:338: Test require flow dev version
SKIPPED [2] tests/test_check_swatinit_simulators.py:482: Test require flow dev version
SKIPPED [2] tests/test_check_swatinit_simulators.py:573: Test require flow dev version
SKIPPED [2] tests/test_check_swatinit_simulators.py:606: Test require flow dev version

Subscript tests for check_swatinit fails during komodo bleeding builds using python 3.8:
https://github.com/equinor/komodo-releases/actions/runs/8041838251/job/21963435048

As a sidenote, these also fails for python version 3.11
https://github.com/equinor/komodo-releases/actions/runs/8041838251/job/21962580908

FAILED tests/test_check_swatinit_simulators.py::test_swatinit_1_far_above_contact[flow] - assert False
 +  where False = <function isclose at 0x7f2ae5d8a700>(0.1, 1)
 +    where <function isclose at 0x7f2ae5d8a700> = np.isclose
FAILED tests/test_check_swatinit_simulators.py::test_swatinit_1_slightly_above_contact[flow] - assert False
 +  where False = <function isclose at 0x7f2ae5d8a700>(0.88782424, 1)
 +    where <function isclose at 0x7f2ae5d8a700> = np.isclose
FAILED tests/test_check_swatinit_simulators.py::test_swatinit_less_than_1_below_contact_neg_pc[flow] - assert False
 +  where False = <function isclose at 0x7f2ae5d8a700>(0.7906856, 1.0)
 +    where <function isclose at 0x7f2ae5d8a700> = np.isclose
FAILED tests/test_check_swatinit_simulators.py::test_swu_equal_swatinit[flow] - assert False
 +  where False = <function isclose at 0x7f2ae5d8a700>(0.5152699, 0.9)
 +    where <function isclose at 0x7f2ae5d8a700> = np.isclose
FAILED tests/test_check_swatinit_simulators.py::test_swu_lessthan_swatinit[flow] - assert False
 +  where False = <function isclose at 0x7f2ae5d8a700>(0.46336114, 0.9)
 +    where <function isclose at 0x7f2ae5d8a700> = np.isclose
= 5 failed, 781 passed, 19 skipped, 1 xfailed, 1 xpassed, 1 warning in 190.71s (0:03:10) =

[sondreso]

Note that all of these run flow, which is using a different version on RHEL7 and RHEL8

[larsevj]

All of these tests check for different values depending on if you are running ecl or opm-flow. With opm-flow for rhel8 it seems like the output is more similar to the values expected from eclipse. So if you for these 5 tests check for exactly the same output for ecl and opm-flow (rhel8) you only get 3 failing test, and the numerical differences for these 3 tests are:

where False = <function isclose at 0x7fee5b8abf30>(0.5152699, 0.5151365333333333)
where False = <function isclose at 0x7fee5b8abf30>(0.7906856, 0.7915066)
where False = <function isclose at 0x7fee5b8abf30>(0.46336114, 0.4632444666666667)

[alifbe]

@larsevj You are right. I think all 5 tests will pass if we are using the same criteria with Eclipse

But it's interesting that only RHEL8 has converging results. The RHEL7 seems to give the same "old" result.

[alifbe]

flowdaily RHEL7 build was updated yesterday and now give a more consistent result.

We will update the test value accordingly. But the challenge is official flow which is used for GitHub action test will not be updated until April.

[alifbe]

Solved in #672