From af20e683cd8b93e449f94702ef322d6f2b57ee36 Mon Sep 17 00:00:00 2001 From: Louis Dorard Date: Mon, 13 Jan 2025 17:43:51 +0200 Subject: [PATCH] Info on compatible connections --- dss-plugin-visual-edit/tests/README.md | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/dss-plugin-visual-edit/tests/README.md b/dss-plugin-visual-edit/tests/README.md index 5977641..04e9152 100644 --- a/dss-plugin-visual-edit/tests/README.md +++ b/dss-plugin-visual-edit/tests/README.md @@ -1,8 +1,27 @@ -# Integration tests +# Tests + +## Which Connections are compatible with Visual Edit? + +As explained in the [deployment guide](https://dataiku.github.io/dss-visual-edit/deploy), we strongly recommend using an SQL connection for the `editlog` and `edits` datasets; it is not necessary to change the connection of the Original Dataset nor of the `edited` dataset. + +You may use other connections for demo/evaluation purposes (e.g. looking at the webapp UX and the integration with the Flow). + +Here is an overview of what was explicitly tested and what is expected to work. + +* **Persisting edits** (i.e. appending rows to the `editlog` dataset): + * ✅ We run automated tests on **PostgreSQL** and **FileSystem**. + * ✅ We successfully tested manually on **Snowflake**. + * ⚠️ We successfully tested manually on **S3** but each edit resulted in a new file, which is not ideal. + * ❓ Persisting edits may not work with other connections, including SQL ones. For instance, it fails on BigQuery. + * This is because the implementation is based on the `write_dataframe` method of the Dataiku API, which may fail when the dataset is in "append only" mode (as is the case with the `editlog`). +* **Building `edits` and `edited` datasets** from an `editlog`, and showing edited data in the webapp, is expected to work on all connections as it is based on the `get_dataframe` and `write_dataframe` methods of the Dataiku API. +* **Initializing a secure `editlog`**: successful manual tests on PostgreSQL (see deployment guide). + +## Automated Integration Tests These tests are written using the Gherkin specification language and a [Dataiku library containing generic Gherkin steps](https://github.com/dataiku/dss-gherkin-steps/blob/main/README.md). See the [parent folder's README](../README.md) for more context. -## Contents +### Contents We test the following: @@ -14,7 +33,7 @@ We test the following: All tests are performed both on a FileSystem and on a PostgreSQL connection. We test that edits are stored in the editlog dataset as expected; we also build the edits and edited datasets and we test their contents as well. -## How to set up an environment to run tests locally +### How to set up an environment to run tests locally - Create a Pyhton environment with test requirements found in this directory. ```bash