Support Sqlalchemy 1.4 part 2

* Disable unimplemented sql_exprssion_limit_offset tests
* Adjust function call to comply with latest api changes
* Disable tests requiring an explicit index which isn't supported by exasol
* Make sure metadata queries do not create a second connection
* Fix args for report command
* Make sure test schemas are in place before integration tests get executed
* Fix setup of test cases
* Temporarily add migration notes and todos to PR/changes
* Mark test depending on driver update as xfail
    see also #232
* run sphinx via poetry in nox task
* Update dependencies
* Suppress failing turbodbc test
* Disable incompatible unit tests
* Disable merge tests

Author: Nicoretti

.github/workflows/ci-cd.yml

@@ -117,7 +117,7 @@ jobs:
           ITDE_TAG: "0.11.0"
 
       - name: Run Test for Python ${{ matrix.python }} using ${{ matrix.connector }}
-        run: poetry run nox -s integration-tests -- --connector ${{ matrix.connector }} --db-version ${{ matrix.exasol_version }}
+        run: poetry run nox -s integration-tests -- -- --connector ${{ matrix.connector }} --db-version ${{ matrix.exasol_version }}
 
   upload_to_pypi:
     runs-on: ubuntu-latest

.github/workflows/ci.yaml

@@ -121,7 +121,7 @@ jobs:
           ITDE_TAG: "0.11.0"
 
       - name: Run Test for Python ${{ matrix.python }} using ${{ matrix.connector }}
-        run: poetry run nox -s integration-tests -- --connector ${{ matrix.connector }} --db-version ${{ matrix.exasol_version }}
+        run: poetry run nox -s integration-tests -- -- --connector ${{ matrix.connector }} --db-version ${{ matrix.exasol_version }}
 
   build_package:
     runs-on: ubuntu-latest

.github/workflows/gh-pages.yaml

@@ -28,7 +28,7 @@ jobs:
           poetry run python -m nox -s build-docs
           touch doc/build/.nojekyll
       - name: Deploy
-        uses: JamesIves/github-pages-deploy-action@v4.3.0
+        uses: JamesIves/github-pages-deploy-action@v4.4.1
         with:
           branch: gh-pages
           folder: ./doc/build

Migration-Notes.md

@@ -0,0 +1,49 @@
+# Current Status
+
+```
+# Status Update
+
+## Test(s) Status
+
+### sqlalchemy
+* pyodbc 3 :no_entry_sign: failed, 190 :heavy_check_mark: passed , 359 :yellow_circle:  skipped, 0 :boom: errors
+    * 1 test is caused by a potentially by an escaping issue
+    * 2 failing tests are caused by incorrectly returned rowcounts
+* turbodbc 6 :no_entry_sign: failed, 185 :heavy_check_mark: passed , 361 :yellow_circle:  skipped, 0 :boom: errors
+    * 1 test is caused by a potentially by an escaping issue
+    * 5 failing tests are caused by incorrectly returned rowcounts
+
+### exasol
+* pyodbc 9 :no_entry_sign: failed, 202 :heavy_check_mark: passed , 10 :yellow_circle:  skipped, 0 :boom: errors
+    * All the failing 9 tests, are caused by the custom merge statement.
+* turbodbc 9 :no_entry_sign: failed, 189 :heavy_check_mark: passed , 23 :yellow_circle:  skipped, 0 :boom: errors
+    * All the failing 9 tests, are caused by the custom merge statement.
+
+## Next Steps
+* Investigate the escaping issue (DifficultParametersTest_*::test_round_trip[q?marks])
+* Investigate the invalid returned row counts
+* Refactor/Implement Merge statement in respect to new sqla syntax/api
+```
+
+# TODO's:
+
+* TODO: Investigate/Fix the escaping issue (DifficultParametersTest_*::test_round_trip[q?marks])
+* TODO: Investigate/Fix the invalid returned row counts
+* TODO: Refactor/Implement custom merge statement based on new sqla api
+* TODO: Move DB setup/preperation from noxfile to conftest.py
+
+# Post Migration TODO's (tickets)
+
+* TODO: Create ticket to Update to ITDE to 1.1.0
+* TODO: Create ticket for migrating exasol tests to non sqla based fixture setup (e.g. pyexasol)
+* TODO: Create ticket to additional details to developer guide
+    # Architecture
+        * https://www.aosabook.org/en/sqlalchemy.html
+    # Custom SQL Constructs
+        * https://docs.sqlalchemy.org/en/14/core/compiler.html
+    # Other dialects with custom commands
+        * https://github.com/SAP/sqlalchemy-hana/blob/master/sqlalchemy_hana/dialect.py
+        * https://github.com/snowflakedb/snowflake-sqlalchemy
+        * https://github.com/clach04/ingres_sa_dialect
+        * https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/blob/main/sqlalchemy_redshift/ddl.py
+        * https://github.com/IBM/nzalchemy/blob/master/sqlalchemy-netezza/nzalchemy/base.py

doc/_static/data/features.csv

@@ -0,0 +1,8 @@
+sqla requirement,test case,supported by exasol db,supported by exasol sqla dialect,test status,comment
+sql_expression_limit_offset,FetchLimitOffsetTest*.test_expr_limit,yes,no,skipped,not implemented
+sql_expression_limit_offset,FetchLimitOffsetTest*.test_expr_limit_offset,yes,no,skipped,not implemented
+sql_expression_limit_offset,FetchLimitOffsetTest*.test_expr_limit_simple_offset,yes,no,skipped,not implemented
+sql_expression_limit_offset,FetchLimitOffsetTest*.test_expr_offset,partially,no,skipped,not implemented
+sql_expression_limit_offset,FetchLimitOffsetTest*.test_simple_limit_expr_offset,yes,no,skipped,not implemented
+,HasIndexTest*.*,no,no,skipped,not supported
+,ComponentReflectionTest*.test_get_indexes*,no,no,skipped,not supported

noxfile.py

@@ -178,8 +178,9 @@ def populate() -> None:
             transaction(
                 connection,
                 (
-                    "CREATE SCHEMA TEST_SCHEMA;",
-                    # "CREATE SCHEMA TEST_SCHEMA_2;",
+                    "CREATE SCHEMA IF NOT EXISTS TEST",
+                    "CREATE SCHEMA IF NOT EXISTS TEST_SCHEMA;",
+                    "CREATE SCHEMA IF NOT EXISTS TEST_SCHEMA_2;",
                 ),
             )
             connection.close()
@@ -318,7 +319,15 @@ def clean(session: Session) -> None:
 def build(session: Session) -> None:
     """Build the documentation"""
     session.run(
-        "sphinx-build", "-b", "html", "-W", f"{DOC}", f"{DOC_BUILD}", external=True
+        "poetry",
+        "run",
+        "sphinx-build",
+        "-b",
+        "html",
+        "-W",
+        f"{DOC}",
+        f"{DOC_BUILD}",
+        external=True,
     )