Skip to content

Latest commit

 

History

History
28 lines (18 loc) · 991 Bytes

README.md

File metadata and controls

28 lines (18 loc) · 991 Bytes

Heterogenous Quantization

Running the Code

cd examples
python3 train.py --workdir=/my/fav/dir --config=efficientnet/configs/efficientnet-lite0_mixed.py

Methods

Pretraining

Alternating Phases

Penalty Scheduling

Gradient Scaling

Finetuning

Granular Quantization

References

  • Uhlich, Stefan, et al. "Mixed precision dnns: All you need is a good parametrization." arXiv preprint arXiv:1905.11452 (2019).
  • Lee, Junghyup, Dohyung Kim, and Bumsub Ham. "Network Quantization with Element-wise Gradient Scaling." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
  • Kim, Jangho, KiYoon Yoo, and Nojun Kwak. "Position-based scaled gradient for model quantization and pruning." arXiv preprint arXiv:2005.11035 (2020).
  • Darvish Rouhani, Bita, et al. "Pushing the limits of narrow precision inferencing at cloud scale with microsoft floating point." Advances in Neural Information Processing Systems 33 (2020).