Heterogenous Quantization

Running the Code

cd examples
python3 train.py --workdir=/my/fav/dir --config=efficientnet/configs/efficientnet-lite0_mixed.py

Methods

Pretraining

Alternating Phases

Penalty Scheduling

Gradient Scaling

Finetuning

Granular Quantization

References

• Uhlich, Stefan, et al. "Mixed precision dnns: All you need is a good parametrization." arXiv preprint arXiv:1905.11452 (2019).
• Lee, Junghyup, Dohyung Kim, and Bumsub Ham. "Network Quantization with Element-wise Gradient Scaling." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
• Kim, Jangho, KiYoon Yoo, and Nojun Kwak. "Position-based scaled gradient for model quantization and pruning." arXiv preprint arXiv:2005.11035 (2020).
• Darvish Rouhani, Bita, et al. "Pushing the limits of narrow precision inferencing at cloud scale with microsoft floating point." Advances in Neural Information Processing Systems 33 (2020).