Add Litestream Apsis db replica test #338
Conversation
|
LGTM. This is promising. Not necessarily here, but it would also be good to test that the replica is usable if litestream goes down hard, i.e. SIGKILL. Also network partition, which you can't necessarily test, but perhaps suspend the process as a proxy. Likewise killing Apsis rather than orderly shutdown. I think stress tests would also be a good idea. Kill Apsis and/or litestream while Apsis is busy and in particular some runs are running. |
|
I've added another test where:
@alexhsamuel, could you please elaborate a bit more on this idea? |
|
Sorry if I wasn't clear. Please don't add new tests/functionality after the initial PR. Let's get this passing CI and merged, and use new PRs for further tests. |
test/int/litestream/test_replica.py
Outdated
| assert any(restored_db_name in l for l in log) | ||
|
|
||
| # check runs data is accurate in the restored db | ||
| client = inst.client |
There was a problem hiding this comment.
You already have this above.
| # ------------------------------------------------------------------------------- | ||
|
|
||
|
|
||
| def test_replica(): |
There was a problem hiding this comment.
Add a @pytest.mark.skipif if litestream isn't available.
| expected_states = ["success", "failure", "error", "skipped"] | ||
| for state in expected_states: | ||
| run_id = client.schedule("job1", {})["run_id"] | ||
| inst.wait_run(run_id) |
There was a problem hiding this comment.
Check that the run succeeded.
|
Ok sounds good, thanks!
@alexhsamuel I don't have permissions for neither of the two, so please try to trigger the GH workflow. |
test/int/litestream/test_replica.py
Outdated
| ) | ||
| except (subprocess.CalledProcessError, FileNotFoundError): | ||
| return False | ||
| return True |
There was a problem hiding this comment.
Use shutil.which() for this.
Add test that mimics the situation where Apsis must be restarted using a db created from a Litestream replica.