Selected Publications

I’m (still…) training myself (slowly…). The full list of my publications can be found in my Google Scholar.

Unlocking Emergent Modularity in Large Language Models
NAACL 2024, Outstanding Paper Award, (6 out of 2434 submissions)
Zihan Qiu, Zeyu Huang, Jie Fu
PDF Code
Unlocking Emergent Modularity in Large Language Models
LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters
arXiv
Xinyu Zhou, Boris Knyazev, Alexia Jolicoeur-Martineau, Jie Fu
PDF Code Twitter
LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters
Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training
NeurIPS 2024 Spotlight
Wenyu Du, Tongxu Luo, Zihan Qiu, Zeyu Huang, Yikang Shen, Reynold Cheng, Yike Guo, Jie Fu
PDF Website Twitter
Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training
Tracking single cell evolution via clock-like chromatin accessibility
Nature Biotechnology 2024
Yu Xiao, Wan Jin, Lingao Ju, Jie Fu, Gang Wang, Mengxue Yu, Fangjin Chen, Kaiyu Qian, Xinghuan Wang, Yi Zhang
PDF Code Twitter
Tracking single cell evolution via clock-like chromatin accessibility
Think Before You Act: Decision Transformers with Working Memory
ICML 2024
Jikun Kang, Romain Laroche, Xingdi Yuan, Adam Trischler, Xue Liu, Jie Fu
PDF Code Twitter
Think Before You Act: Decision Transformers with Working Memory
HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts
ACL 2024
Hao Zhao, Zihan Qiu, Huijia Wu, Zili Wang, Zhaofeng He, Jie Fu
PDF Code
HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts
Massive Editing for Large Language Models via Meta Learning
ICLR 2024
Chenmien Tan, Ge Zhang, Jie Fu
PDF Code
Massive Editing for Large Language Models via Meta Learning
MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training
ICLR 2024
Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge, Roger Dannenberg, Ruibo Liu, Wenhu Chen, Gus Xia, Yemin Shi, Wenhao Huang, Yike Guo, Jie Fu
PDF Huggingface Code
MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training
AI Alignment: A Comprehensive Survey
arXiv
Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, Fanzhi Zeng, Kwan Yee Ng, Juntao Dai, Xuehai Pan, Aidan O'Gara, Yingshan Lei, Hua Xu, Brian Tse, Jie Fu, Stephen McAleer, Yaodong Yang, Yizhou Wang, SongChun Zhu, Yike Guo, Wen Gao
PDF Webpage Huggingface Data Media Twitter
AI Alignment: A Comprehensive Survey
When Do Graph Neural Networks Help with Node Classification: Investigating the Homophily Principle on Node Distinguishability
NeurIPS 2023
Sitao Luan, Chenqing Hua, Minkai Xu, Qincheng Lu, Jiaqi Zhu, Xiao-Wen Chang, Jie Fu, Jure Leskovec, Doina Precup
PDF
When Do Graph Neural Networks Help with Node Classification: Investigating the Homophily Principle on Node Distinguishability
Running Ahead of Evolution - AI based Simulation for Predicting Future High-risk SARS-CoV-2 Variants
IJHPCA, ACM Gordon Bell COVID Finalist 2022
Jie Chen, Zhiwei Nie, Yu Wang, Kai Wang, Fan Xu, Zhiheng Hu, Bin Zheng, Zhennan Wang, Guoli Song, Jingyi Zhang, Jie Fu, Xiansong Huang, Zhongqi Wang, Zhixiang Ren, Qiankun Wang, Daixi Li, Dongqing Wei, Chao Yang, Yonghong Tian
PDF Code
Running Ahead of Evolution - AI based Simulation for Predicting Future High-risk SARS-CoV-2 Variants
Interactive Natural Language Processing
arXiv
Zekun Wang, Ge Zhang, Kexin Yang, Ning Shi, Wangchunshu Zhou, Shaochun Hao, Guangzheng Xiong, Yizhi Li, Mong Yuan Sim, Xiuying Chen, Qingqing Zhu, Zhenzhu Yang, Adam Nik, Qi Liu, Chenghua Lin, Shi Wang, Ruibo Liu, Wenhu Chen, Ke Xu, Dayiheng Liu, Yike Guo, Jie Fu
PDF Website Media
Interactive Natural Language Processing
MUDiff: Unified Diffusion for Complete Molecule Generation
LoG 2023
Chenqing Hua, Sitao Luan, Minkai Xu, Rex Ying, Jie Fu, Stefano Ermon, Doina Precup
PDF
MUDiff: Unified Diffusion for Complete Molecule Generation
Learning Multi-Objective Curricula for Robotic Policy Learning
CoRL 2022
Jikun Kang, Miao Liu, Abhinav Gupta, Chris Pal, Xue Liu, Jie Fu
PDF Code
Learning Multi-Objective Curricula for Robotic Policy Learning
Biological Sequence Design with GFlowNets
ICML 2022
Moksh Jain, Emmanuel Bengio, Alex-Hernandez Garcia, Jarrid Rector-Brooks, Bonaventure Dossou, Chanakya Ekbote, Jie Fu, Tianyu Zhang, Micheal Kilgour, Dinghuai Zhang, Lena Simine, Payel Das, Yoshua Bengio
PDF Code
Biological Sequence Design with GFlowNets
Unifying Likelihood-free Inference with Black-box Sequence Design and Beyond
ICLR 2022 Spotlight
Dinghuai Zhang, Jie Fu, Yoshua Bengio, Aaron Courville
PDF
Unifying Likelihood-free Inference with Black-box Sequence Design and Beyond
Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters
ICLR 2021 Outstanding Paper Award (8 out of 2997 submissions)
Aston Zhang, Yi Tay, Shuai Zhang, Alvin Chan, Anh Tuan Luu, Siu Cheung Hui, Jie Fu
PDF
Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters
CoCon: A Self-Supervised Approach for Controlled Text Generation
ICLR 2021
Alvin Chan, Yew-Soon Ong, Bill Pung, Aston Zhang, Jie Fu
PDF Code
CoCon: A Self-Supervised Approach for Controlled Text Generation
Interactive Machine Comprehension with Information Seeking Agents
ACL 2020
Xingdi Yuan, Jie Fu, Marc-Alexandre Cote, Yi Tay, Chris Pal, Adam Trischler
PDF Code
Interactive Machine Comprehension with Information Seeking Agents