Motivation

Despite the success of Mixture-of-Expert Transformers, the modular design of the human brain enables far more flexibility.

I’m working on designing more flexiable Transformers with more flexible modular design.

Milestones

• Learning Multi-Objective Curricula for Robotic Policy Learning, CoRL 2022. We propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula that are generated by a set of parametric curriculum modules. Each curriculum module is instantiated as a neural network and is responsible for generating a particular curriculum.

Next

• Make modular designs more scalable to handle large-scale benchmark tasks, e.g., MineDojo.