Research question
How can learning algorithms reduce sampling, communication, memory, and optimization costs without losing rigor?
Approach
Develop sampling, projection-free optimization, communication-efficient training, memory replay, and analytical capacity methods.
Key results
- Adam with Bandit Sampling uses online bandit models to focus optimization effort during deep learning.
- Communication-efficient distributed learning reduces coordination costs in large-batch optimization.
- Transformer work develops memory replay and characterizes provable memorization capacity.