Tensorflow privacy not compatible with tensorflow version 2.18.0

[namilus]

Tensorflow privacy v0.8.12 fails to import when using tensorflow v2.18.0. See below for full errors:

$ python3 test.py 
2025-02-05 13:20:25.533199: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1738761625.545500 1378328 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1738761625.549316 1378328 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-02-05 13:20:25.563588: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
  File "/home/mohamed/git/mbm-audit/test.py", line 1, in <module>
    import tensorflow_privacy
  File "/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow_privacy/__init__.py", line 26, in <module>
    from tensorflow_privacy import v1
  File "/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow_privacy/v1/__init__.py", line 28, in <module>
    from tensorflow_privacy.privacy.estimators.v1.dnn import DNNClassifier as DNNClassifierV1
  File "/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow_privacy/privacy/estimators/v1/dnn.py", line 19, in <module>
    from tensorflow_privacy.privacy.estimators.v1 import head as head_lib
  File "/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow_privacy/privacy/estimators/v1/head.py", line 20, in <module>
    from tensorflow_estimator.python.estimator import model_fn
  File "/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow_estimator/__init__.py", line 8, in <module>
    from tensorflow_estimator._api.v1 import estimator
  File "/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow_estimator/_api/v1/estimator/__init__.py", line 8, in <module>
    from tensorflow_estimator._api.v1.estimator import experimental
  File "/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow_estimator/_api/v1/estimator/experimental/__init__.py", line 8, in <module>
    from tensorflow_estimator.python.estimator.canned.dnn import dnn_logit_fn_builder # line: 45
  File "/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow_estimator/python/estimator/canned/dnn.py", line 26, in <module>
    from tensorflow_estimator.python.estimator import estimator
  File "/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 34, in <module>
    from tensorflow.python.distribute import estimator_training as distribute_coordinator_training
ImportError: cannot import name 'estimator_training' from 'tensorflow.python.distribute' (/home/mohamed/miniconda3/envs/mbm-audit/lib/python3.10/site-packages/tensorflow/python/distribute/__init__.py)

This is python3.10 with the following versions of packages:

$ python3 -m pip list | grep tensor
tensorboard                  2.18.0
tensorboard-data-server      0.7.2
tensorflow                   2.18.0
tensorflow-estimator         2.15.0
tensorflow-io-gcs-filesystem 0.37.1
tensorflow_privacy           0.9.0
tensorflow-probability       0.22.1

Recreate with the following commands:

$ conda create -n test python=3.10
$ conda activate test
$ python3 -m pip install tensorflow tensorflow-privacy
$ pyton3 -c "import tensorflow_privacy"

[niral28]

Is it because estimator has been deprecated? https://www.tensorflow.org/guide/estimator

Warning: TensorFlow 2.15 included the final release of the tf-estimator package. Estimators will not be available in TensorFlow 2.16 or after. See the migration guide for more information about how to convert off of Estimators.

[niral28]

And the problem with downgrading to supported versions like 2.14/2.15 is that they don't support Google colab's GPU

• https://www.reddit.com/r/tensorflow/comments/1fdfc12/cant_install_tensorflowprivacy_on_colab_urgent/
• https://stackoverflow.com/questions/79163353/tensorflow-2-14-0-fails-to-detect-gpu-on-google-colab