Jie Fu

Visiting Scholar

Hong Kong University of Science and Technology (HKUST)

Jie Fu (付杰) is a happy and funny visiting scholar at Hong Kong University of Science and Technology (HKUST), chasing his human-friendly big AI dream.

He was a postdoctoral fellow (funded by Microsoft Research Montreal) supervised by Yoshua Bengio at University of Montreal, Quebec AI Institute (Mila). He was an IVADO postdoctoral fellow supervised by Chris Pal at Polytechnique Montreal, Quebec AI Institute (Mila). He worked as a researcher (PI) at BAAI (智源人工智能研究院). He obtained his PhD from National University of Singapore under the supervision of Tat-Seng Chua. He received ICLR 2021 Outstanding Paper Award.

He is currently working towards safe, scalable system-2, mixed-modal interactive LLMs that are capable of observing, acting, and receiving feedback iteratively from external entities. More concretely:

Towards Safe and Scalable System-2 Language Models: Human-AI Alignment, Reinforcement learning, Meta learning, Lifelong learning, Reasoning, Modular neural architectures, etc
Applications: Multi-modal learning (e.g., NLP, CV), Multi-modal embodied intelligence, AI for science, etc

Current Key Research Projects

Large Language Model and Beyond

Towards safe system-2 deep learning

Drug Discovery

In-silico + in-vitro approaches

AI for Social Good

Focusing on climate change, environment protection, humanities

Meta Learning

Making it scalable

Selected Publications

I’m (still…) training myself (slowly…)

My full publications can be found in Google Scholar.

When Do Graph Neural Networks Help with Node Classification: Investigating the Homophily Principle on Node Distinguishability

NeurIPS 2023

Sitao Luan, Chenqing Hua, Minkai Xu, Qincheng Lu, Jiaqi Zhu, Xiao-Wen Chang, Jie Fu, Jure Leskovec, Doina Precup

When Do Graph Neural Networks Help with Node Classification: Investigating the Homophily Principle on Node Distinguishability

MARBLE: Music Audio Representation Benchmark for Universal Evaluation

NeurIPS 2023

Ruibin Yuan, Yinghao Ma, Yizhi Li, Ge Zhang, Xingran Chen, Hanzhi Yin, Le Zhuo, Yiqi Liu, Jiawen Huang, Zeyue Tian, Binyue Deng, Ningzhi Wang, Wenhu Chen, Gus Xia, Wei Xue, Si Liu, Shi Wang, Ruibo Liu, Yike Guo, Jie Fu

MARBLE: Music Audio Representation Benchmark for Universal Evaluation

Running Ahead of Evolution - AI based Simulation for Predicting Future High-risk SARS-CoV-2 Variants

IJHPCA, ACM Gordon Bell COVID Finalist 2022

Jie Chen, Zhiwei Nie, Yu Wang, Kai Wang, Fan Xu, Zhiheng Hu, Bin Zheng, Zhennan Wang, Guoli Song, Jingyi Zhang, Jie Fu, Xiansong Huang, Zhongqi Wang, Zhixiang Ren, Qiankun Wang, Daixi Li, Dongqing Wei, Chao Yang, Yonghong Tian

ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory

Symposium on Large Language Models, IJCAI 2023

Chenxu Hu, Jie Fu, Chenzhuang Du, Simian Luo, Junbo Zhao, Hang Zhao

ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory

MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

arXiv

Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge, Roger Dannenberg, Ruibo Liu, Wenhu Chen, Gus Xia, Yemin Shi, Wenhao Huang, Yike Guo, Jie Fu

MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

Think Before You Act: Decision Transformers with Internal Working Memory

arXiv

Jikun Kang, Romain Laroche, Xingdi Yuan, Adam Trischler, Xue Liu, Jie Fu

Interactive Natural Language Processing

arXiv

Zekun Wang, Ge Zhang, Kexin Yang, Ning Shi, Wangchunshu Zhou, Shaochun Hao, Guangzheng Xiong, Yizhi Li, Mong Yuan Sim, Xiuying Chen, Qingqing Zhu, Zhenzhu Yang, Adam Nik, Qi Liu, Chenghua Lin, Shi Wang, Ruibo Liu, Wenhu Chen, Ke Xu, Dayiheng Liu, Yike Guo, Jie Fu

MUDiff: Unified Diffusion for Complete Molecule Generation

arXiv

Chenqing Hua, Sitao Luan, Minkai Xu, Rex Ying, Jie Fu, Stefano Ermon, Doina Precup

MUDiff: Unified Diffusion for Complete Molecule Generation

Chinese Open Instruction Generalist: A Preliminary Release

arXiv

Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang, Chenghua Lin, Wenhao Huang, Jie Fu

Learning Multi-Objective Curricula for Robotic Policy Learning

CoRL 2022

Jikun Kang, Miao Liu, Abhinav Gupta, Chris Pal, Xue Liu, Jie Fu

Learning Multi-Objective Curricula for Robotic Policy Learning

Biological Sequence Design with GFlowNets

ICML 2022

Moksh Jain, Emmanuel Bengio, Alex-Hernandez Garcia, Jarrid Rector-Brooks, Bonaventure Dossou, Chanakya Ekbote, Jie Fu, Tianyu Zhang, Micheal Kilgour, Dinghuai Zhang, Lena Simine, Payel Das, Yoshua Bengio