Modular Neural Architectures
Despite the success of Mixture-of-Expert Transformers, the modular design of the human brain enables far more flexibility.
I’m working on designing more flexiable Transformers with more flexible modular design.
- Learning Multi-Objective Curricula for Robotic Policy Learning, CoRL 2022. We propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula that are generated by a set of parametric curriculum modules. Each curriculum module is instantiated as a neural network and is responsible for generating a particular curriculum.
- Make modular designs more scalable to handle large-scale benchmark tasks, e.g., MineDojo.