Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker/api/requirements: fix api container build failure #305

Closed
wants to merge 3 commits into from

Conversation

JenySadadia
Copy link
Collaborator

@JenySadadia JenySadadia commented Jul 18, 2023

Fixes #303

Fix build failures of api container. Error logs can be found at https://github.com/kernelci/kernelci-api/issues/303.

Root cause:
fastapi[all] 0.68.1 package installs pyyaml 5.4.1 package internally. That again depends on CPython package. Due to the recent upgrade to CPython 3.0.0, pyyaml 5.4.1 stopped building with python 3.10. To fix this, upgrading fastapi[all] to 0.99.1 version as that uses pyyaml 6.0.1. The selected fastapi version is compatible with current fastapi-pagination and fastapi-versioning packages.

Fix build failures of `api` container. Error logs can be found
at ```https://github.com/kernelci/kernelci-api/issues/303```.

Root cause:
`fastapi[all] 0.68.1` package installs `pyyaml 5.4.1` package
internally. That again depends on `Cython` package.
Due to the recent upgrade to `Cython 3.0.0`, `pyyaml 5.4.1` stopped
building with `python 3.10`. To fix this, upgrading `fastapi[all]`
to `0.99.1` version as that uses `pyyaml 6.0.1`. The selected
`fastapi` version is compatible with current `fastapi-pagination`
and `fastapi-versioning` packages.

Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>
After changing `fastapi` package version, API returns
error while trying to get node by ID when node does not
exist matching provided ID in the DB.
Add `None` to response model `Union` in `get_node`
handler to fix it.

Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>
@JenySadadia
Copy link
Collaborator Author

Fixed pytest error from test_node_handler.test_get_node_by_id_endpoint_empty_response.
The issue occurred while trying to get node by ID when node does not exist matching provided ID:

pydantic/error_wrappers.py:63: in pydantic.error_wrappers.ValidationError.errors
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = typing.Union[api.models.Regression, api.models.Node], attr = '__pydantic_model__'

    def __getattr__(self, attr):
        if attr in {'__name__', '__qualname__'}:
            return self._name or self.__origin__.__name__
    
        # We are careful for copy and pickle.
        # Also for simplicity we don't relay any dunder names
        if '__origin__' in self.__dict__ and not _is_dunder(attr):
            return getattr(self.__origin__, attr)
>       raise AttributeError(attr)
E       AttributeError: __pydantic_model__

/usr/lib/python3.10/typing.py:984: AttributeError

@gctucker
Copy link
Contributor

@JenySadadia Some checks are still failing. Otherwise I think this is good to be merged and necessary so other PRs can pass checks too.

Due to `fastapi` version change, the newer `TestClient` from
`fastapi` is causing issues by using `anyio` and creating
a different event loop.
This results into failing async pubsub e2e tests with below
errors:
```
RuntimeError: Task got Future attached to a different loop
```
Remove sync test client and use async one for all the tests
to fix it.

Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>
@JenySadadia
Copy link
Collaborator Author

e2e tests were failing after fastapi version change.
Error logs:

ernelci-api-e2e-tests | _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
kernelci-api-e2e-tests | 
kernelci-api-e2e-tests | self = <api.db.Database object at 0x7f25275935e0>
kernelci-api-e2e-tests | model = <class 'api.models.User'>
kernelci-api-e2e-tests | attributes = {'profile.username': 'test_user'}
kernelci-api-e2e-tests | 
kernelci-api-e2e-tests |     async def find_one_by_attributes(self, model, attributes):
kernelci-api-e2e-tests |         """Find one object with matching attributes without pagination
kernelci-api-e2e-tests |     
kernelci-api-e2e-tests |         The attributes dictionary provides key/value pairs used to find an
kernelci-api-e2e-tests |         object with matching attributes.
kernelci-api-e2e-tests |         """
kernelci-api-e2e-tests |         col = self._get_collection(model)
kernelci-api-e2e-tests | >       obj = await col.find_one(attributes)
kernelci-api-e2e-tests | E       RuntimeError: Task <Task pending name='anyio.from_thread.BlockingPortal._call_func' coro=<BlockingPortal._call_func() running at /home/kernelci/.local/lib/python3.10/site-packages/anyio/from_thread.py:217> cb=[TaskGroup._spawn.<locals>.task_done() at /home/kernelci/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:661]> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.10/asyncio/futures.py:385]> attached to a different loop

The newer TestClient from fastapi is causing issues by using anyio and creating a different event loop.

Fixed it by removing synchronous TestClient and used async one for all the tests.
This documentation also recommends using async test client in this kind of scenarios:https://www.starlette.io/testclient/#asynchronous-tests

@gctucker
Copy link
Contributor

@JenySadadia See also #306 to avoid the issue altogether without changing the code. If this PR #305 is all fine and has no side-effect issues then we can just merge it I guess. If more work is required then we can have #306 merged first to get back to working image builds and fix this properly later with an upgrade to the latest FastAPI and Pydantic.

@JenySadadia
Copy link
Collaborator Author

@JenySadadia See also #306 to avoid the issue altogether without changing the code. If this PR #305 is all fine and has no side-effect issues then we can just merge it I guess. If more work is required then we can have #306 merged first to get back to working image builds and fix this properly later with an upgrade to the latest FastAPI and Pydantic.

Thanks for the other fix. I think we can merge it as it doesn't require any code changes.
If we want to upgrade packages in the future, it would need some code rework and we can refer to this PR for it.

@gctucker gctucker added the staging-skip Don't test automatically on staging.kernelci.org label Jul 20, 2023
@gctucker
Copy link
Contributor

OK so I'm adding the skip label so it doesn't interfere with the staging runs. I guess we could have a GitHub issue about upgrading to the latest FastAPI with all related dependencies, and then resume progress on this PR.

@gctucker
Copy link
Contributor

gctucker commented Aug 9, 2023

Replaced with #321 to do a full upgrade for the initial production rollout.

@gctucker gctucker closed this Aug 9, 2023
@JenySadadia JenySadadia deleted the fix-api-build branch December 1, 2023 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
staging-skip Don't test automatically on staging.kernelci.org
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fail to build API Docker image with Python 3.10
2 participants