Selected Publications | Big AI Dream

Unlocking Emergent Modularity in Large Language Models

NAACL 2024, Outstanding Paper Award

Zihan Qiu, Zeyu Huang, Jie Fu

LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters

arXiv

Xinyu Zhou, Boris Knyazev, Alexia Jolicoeur-Martineau, Jie Fu

LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

arXiv

Wenyu Du, Tongxu Luo, Zihan Qiu, Zeyu Huang, Yikang Shen, Reynold Cheng, Yike Guo, Jie Fu

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

Tracking single cell evolution via clock-like chromatin accessibility

Nature Biotechnology 2024

Yu Xiao, Wan Jin, Lingao Ju, Jie Fu, Gang Wang, Mengxue Yu, Fangjin Chen, Kaiyu Qian, Xinghuan Wang, Yi Zhang

Tracking single cell evolution via clock-like chromatin accessibility

Think Before You Act: Decision Transformers with Working Memory

ICML 2024

Jikun Kang, Romain Laroche, Xingdi Yuan, Adam Trischler, Xue Liu, Jie Fu

HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts

ACL 2024

Hao Zhao, Zihan Qiu, Huijia Wu, Zili Wang, Zhaofeng He, Jie Fu

HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts

Massive Editing for Large Language Models via Meta Learning

ICLR 2024

Chenmien Tan, Ge Zhang, Jie Fu

Massive Editing for Large Language Models via Meta Learning

MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

ICLR 2024

Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge, Roger Dannenberg, Ruibo Liu, Wenhu Chen, Gus Xia, Yemin Shi, Wenhao Huang, Yike Guo, Jie Fu

MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

TACO: Topics in Algorithmic COde generation dataset

arXiv

Rongao Li, Jie Fu, BoWen Zhang, Tao Huang, Zhihong Sun, Chen Lyu, Guang Liu, Zhi Jin, Ge Li

TACO: Topics in Algorithmic COde generation dataset

AI Alignment: A Comprehensive Survey

arXiv

Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, Fanzhi Zeng, Kwan Yee Ng, Juntao Dai, Xuehai Pan, Aidan O'Gara, Yingshan Lei, Hua Xu, Brian Tse, Jie Fu, Stephen McAleer, Yaodong Yang, Yizhou Wang, SongChun Zhu, Yike Guo, Wen Gao

When Do Graph Neural Networks Help with Node Classification: Investigating the Homophily Principle on Node Distinguishability

NeurIPS 2023

Sitao Luan, Chenqing Hua, Minkai Xu, Qincheng Lu, Jiaqi Zhu, Xiao-Wen Chang, Jie Fu, Jure Leskovec, Doina Precup

When Do Graph Neural Networks Help with Node Classification: Investigating the Homophily Principle on Node Distinguishability

MARBLE: Music Audio Representation Benchmark for Universal Evaluation

NeurIPS 2023, Datasets and Benchmarks Track

Ruibin Yuan, Yinghao Ma, Yizhi Li, Ge Zhang, Xingran Chen, Hanzhi Yin, Le Zhuo, Yiqi Liu, Jiawen Huang, Zeyue Tian, Binyue Deng, Ningzhi Wang, Wenhu Chen, Gus Xia, Wei Xue, Si Liu, Shi Wang, Ruibo Liu, Yike Guo, Jie Fu

MARBLE: Music Audio Representation Benchmark for Universal Evaluation

Running Ahead of Evolution - AI based Simulation for Predicting Future High-risk SARS-CoV-2 Variants

IJHPCA, ACM Gordon Bell COVID Finalist 2022

Jie Chen, Zhiwei Nie, Yu Wang, Kai Wang, Fan Xu, Zhiheng Hu, Bin Zheng, Zhennan Wang, Guoli Song, Jingyi Zhang, Jie Fu, Xiansong Huang, Zhongqi Wang, Zhixiang Ren, Qiankun Wang, Daixi Li, Dongqing Wei, Chao Yang, Yonghong Tian

ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory

Symposium on Large Language Models, IJCAI 2023

Chenxu Hu, Jie Fu, Chenzhuang Du, Simian Luo, Junbo Zhao, Hang Zhao

ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory

Interactive Natural Language Processing

arXiv

Zekun Wang, Ge Zhang, Kexin Yang, Ning Shi, Wangchunshu Zhou, Shaochun Hao, Guangzheng Xiong, Yizhi Li, Mong Yuan Sim, Xiuying Chen, Qingqing Zhu, Zhenzhu Yang, Adam Nik, Qi Liu, Chenghua Lin, Shi Wang, Ruibo Liu, Wenhu Chen, Ke Xu, Dayiheng Liu, Yike Guo, Jie Fu

MUDiff: Unified Diffusion for Complete Molecule Generation

LoG 2023

Chenqing Hua, Sitao Luan, Minkai Xu, Rex Ying, Jie Fu, Stefano Ermon, Doina Precup

MUDiff: Unified Diffusion for Complete Molecule Generation

Learning Multi-Objective Curricula for Robotic Policy Learning

CoRL 2022

Jikun Kang, Miao Liu, Abhinav Gupta, Chris Pal, Xue Liu, Jie Fu

Learning Multi-Objective Curricula for Robotic Policy Learning

Biological Sequence Design with GFlowNets

ICML 2022

Moksh Jain, Emmanuel Bengio, Alex-Hernandez Garcia, Jarrid Rector-Brooks, Bonaventure Dossou, Chanakya Ekbote, Jie Fu, Tianyu Zhang, Micheal Kilgour, Dinghuai Zhang, Lena Simine, Payel Das, Yoshua Bengio