CoRR 2022Published in Computing Research Repository (CoRR)

Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers

Rui Liu, Young Jin Kim, Alexandre Muzio, Barzan Mozafari, Hany Hassan Awadalla

Related project: Efficient and Scalable Machine Learning