Different error when run the pytest

[chens2i]

After follow the instruction, all mentioned requirement packages has been satisfied, until Run the test.

(env) root@t-cisubuntu20:/opt# type -a python
python is /opt/env/bin/python
(env) root@t-cisubuntu20:/opt# type -a pytest
pytest is /opt/env/bin/pytest
(env) root@t-cisubuntu20:/opt# python3 -m pytest
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /opt
plugins: flask-1.2.0
collected 0 items / 1 error

================================================================================================================== ERRORS ===================================================================================================================
_______________________________________________________________________________________ ERROR collecting moodle-mlbackend-python/test/test_webapp.py ________________________________________________________________________________________
ImportError while importing test module '/opt/moodle-mlbackend-python/test/test_webapp.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/init.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
moodle-mlbackend-python/test/test_webapp.py:60: in
import webapp
E ModuleNotFoundError: No module named 'webapp'
========================================================================================================== short test summary info ==========================================================================================================
ERROR moodle-mlbackend-python/test/test_webapp.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================= 1 error in 0.23s ==============================================================================================================

From the docs, If you hit a different error at this point, something is missing.
What am I missing? Go ahead for next step?

[douglasbagnall]

“No module named 'webapp'” means python3 it hasn't found webapp.py.

Usually I run the tests in a venv virtual env, along the lines of https://github.com/moodlehq/moodle-mlbackend-python/blob/master/INSTALL-venv.md from the directory (/opt/moodle-mlbackend-python/ in this case).

It should also be possible to run without a virtual environment. Have you tried running it from the install directory

cd /opt/moodle-mlbackend-python/ 
python3 -m pytest

or using a PYTHONPATH?

PYTHONPATH=/opt/moodle-mlbackend-python/ python3 -m pytest

[chens2i]

Thanks for your reply @douglasbagnall
After add the PYTHONPATH, the TEST Run starts, but with different error.

env/lib/python3.8/site-packages/joblib/externals/cloudpickle/cloudpickle.py:133: in _make_cell_set_template_code
return types.CodeType(
E TypeError: an integer is required (got type bytes)

After done some research, I found that joblib v0.13.2 (installed version) fails in python 3.8 (installed version).
joblib/joblib#917 (comment)

I upgrade the joblib to version 0.14.0, via
pip install joblib==0.14.0

Run the test again, I got those:
(env) root@t-cisubuntu20:/opt/moodle-mlbackend-python# python3 -m pytest
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /opt/moodle-mlbackend-python
plugins: flask-1.2.0
collected 26 items

test/test_webapp.py .......s.ss..............x [100%]

============================================================================================================= warnings summary ==============================================================================================================
../env/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22
/opt/env/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp

../env/lib/python3.8/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py:1
../env/lib/python3.8/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py:1
/opt/env/lib/python3.8/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
from collections import Mapping, MutableMapping

-- Docs: https://docs.pytest.org/en/stable/warnings.html
====================================================================================== 22 passed, 3 skipped, 1 xfailed, 3 warnings in 81.94s (0:01:21) ======================================================================================

Still not the same error mentioned on instruction docs, and I found that the docs was release on 15 Jan 2020, might be out of date.

Especially for requirements.txt file, from docs it said tensorflow works with the older 1.14, is that works only for v1.14 and not newer? I have got version 2.4.3 installed. I will attached the the result of
pip install -r requirements.txt

requirements result.txt

From the issue ticket #8 , it support up to Moodle version 3.8dev, how about Moodle V3.9.7 and python version 3.x please, as you can from above, my current python version is 3.8.10.

[danmarsden]

Hi Star! - Great to see an interest from LU in using Moodle's machine learning analytics!

Yes, the version we're running here at Catalyst supports 3.9.7 (UC are still running 3.9) keep in mind that HQ are in the process of rebuilding and updating the official builds in MDL-70887 too.

Feel free to reach out privately if you're still stuck and can't get it running - you could look at using our hosted tensorflow server or getting us to set it up on a VM for you.

thanks!

[ilyatregubov]

HI @chens2i ,

Have you tried to run with different python version? I see CI in github uses python3.7

Cheers,
Ilya

[douglasbagnall]

Have you tried to run with different python version? I see CI in github uses python3.7

To clarify, the mlbackend itself runs under newer versions (I use 3.8 and 3.9 for development), but the dependencies in requirements.txt and setup.py are fragile and imperfectly maintained. So Ilya is right. This is a good thing to look at.

The approach I usually use in these situations is to remove < constraints, under the assumption they have never been added with good knowledge of the future.

For example, your problem here (from https://github.com/moodlehq/moodle-mlbackend-python/files/7239839/requirements.result.txt):

Collecting joblib<0.14,>=0.13.0
  Using cached joblib-0.13.2-py2.py3-none-any.whl (278 kB)
  [... other things happen ...]
Installing collected packages: joblib
  Attempting uninstall: joblib
    Found existing installation: joblib 0.14.0
    Uninstalling joblib-0.14.0:
      Successfully uninstalled joblib-0.14.0
Successfully installed joblib-0.13.2

looks to be that something (e.g. tensorflow) requires joblib 0.14+, and we say joblib<0.14, so pip briefly installs one and then replaces it with the other. If we stop claiming to need joblib<0.14 (a claim for which we have no real basis, AFAIK), the problem will disappear.

Having said that, I am not a packaging person.