Skip to content

Latest commit

 

History

History
12 lines (9 loc) · 265 Bytes

README.md

File metadata and controls

12 lines (9 loc) · 265 Bytes

Transformer_Compression

My goal is to adapt and understand open-source research on state-of-the-art(SOTA) Transformer compressing for my own research purposes.

📦 Implemented

Pruning on MHA:

  • CoFi
  • DynaBERT

Retraining Pruning:

  • DynaBERT
  • EBERT