Zheqing (Bill) Zhu
朱哲清
Senior Staff Research Lead Manager
Head of Applied Reinforcement Learning
Facebook (Meta) AI
LinkedIn: https://www.linkedin.com/in/zheqingzhubill/
Twitter: https://twitter.com/ZheqingZhu
Facebook (Meta) AI Profile: https://ai.facebook.com/people/zheqing-bill-zhu
Contact:
billzhu@meta.com (For industry-related inquiries)
zheqzhu@stanford.edu (For academic-related inquiries)
Zheqing (Bill) Zhu is a Senior Staff Research Lead Manager at Meta AI, where he serves as the Head of Applied Reinforcement Learning. His main interest lies in bringing state-of-the-art reinforcement learning technologies to real-life and bridging the gap between theoretical reinforcement learning and real-world systems. Prior to serving as Head of Applied Reinforcement Learning, he was the engineering manager and tech lead for Meta’s Ads Growth Machine Learning team, where he built the team from scratch and enabled exponential growth in business market for Meta.
Bill earned his PhD degree in Reinforcement Learning at Stanford University, advised by Professor Benjamin Van Roy, while working full-time at Meta AI leading the Applied Reinforcement Learning team. His main research focus is to understand theoretical and practical gaps in existing reinforcement learning algorithms in a real-world context. He received Master of Science in Computer Science from Stanford University (also while full-time at Meta AI) and Bachelor of Science in Computer Science with a Minor in Finance, summa cum laude, from Duke University. He has been the recipient of the Alex Vasilos Memorial Award, the Highest Distinction Graduate Award from Duke University and Ericsson BUSS Shanghai Quarterly Technical Award.
Professional Experience
Engineering Manager - Head of Applied Reinforcement Learning, Facebook (Meta) AI, 2021 - now
Engineering Manager / Tech Lead, Ads Growth Machine Learning, Facebook (Meta), 2018 - 2021
Machine Learning Engineer, Ads Growth Machine Learning, Facebook (Meta), 2017 - 2018
Selected Publicly Available Research
Pearl - A Production-ready Reinforcement Learning Agent
ArXiv Link, Website, Github Repo. Submitting to JMLR
Zheqing Zhu (First author and PI), Rodrigo de Salvo Braz, Jalaj Bhandari, Daniel Jiang, Yi Wan, Yonathan Efroni, Ruiyang Xu, Liyuan Wang, Hongbo Guo, Alex Nikulkov, Dmytro Korenkevych, Urun Dogan, Frank Cheng, Zheng Wu, Wanqiao XuNon-Stationary Contextual Bandit Learning via Neural Predictive Ensemble Sampling
ArXiv Link, Submitted to ICLR 2024 and Also Presented at INFORMS 2023
Zheqing Zhu, Yueyang Liu, Xu Kuang, Benjamin Van RoyOffline Reinforcement Learning for Optimizing Production Bidding Policies
ArXiv Link, Submitted to WWW 2024
Dmytro Korenkevych, Frank Cheng, Artsiom Balakir, Alex Nikulkov, Zhihao Cen, Zuobing Xu, Zheqing Zhu (PI)Optimizing Long-term Value for Auction-Based Recommender Systems via On-Policy Reinforcement Learning.
ArXiv Link, RecSys 2023 (Also presented at KDD Workshop 2023)
Ruiyang Xu*, Jalaj Bhandari*, Dmytro Korenkevych, Fan Liu, Yuchen He, Alex Nikulkov, Zheqing Zhu (PI)Deep Exploration for Recommendation Systems.
ArXiv Link, RecSys 2023
Zheqing Zhu, Benjamin Van RoyScalable Neural Contextual Bandit for Recommender Systems.
ArXiv Link, CIKM 2023 (also presented at KDD Workshop 2023)
Zheqing Zhu, Benjamin Van RoyLearning to Bid and Rank Together in Recommendation Systems.
(ArXiv Coming Soon), Springer Machine Learning Journal
Geng Ji, Wentao Jiang, Jiang Li, Fahmid Morshed Fahid, Zhengxing Chen, Yinghua Li, Jun Xiao, Chongxi Bao, Zheqing Zhu (PI)IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Control.
ArXiv Link, ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023
Rohan Chitnis*, Yingchen Xu*, Bobak Hashemi, Lucas Lehnert, Urun Dogan, Zheqing Zhu (Secondary PI), Olivier Delalleau
Evaluating Online Bandit Exploration In Large-Scale Recommender System.
ArXiv Link, KDD Workshop on Multi-Armed Bandits and Reinforcement Learning: Advancing Decision Making in E-Commerce and Beyond, 2023
Hongbo Guo, Ruben Naeff, Alex Nikulkov, Zheqing Zhu (PI)Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation via Deep Reinforcement Learning.
ArXiv Link, Submitted to CoNext, 2023
Chang-Lin Chen, Hanhan Zhou, Jiayu Chen, Mohammad Pedramfar, Vaneet Aggarwal, Tian Lan, Zheqing Zhu (RL PI), Chi Zhou, Tim Gasser, Pol Mauri Ruiz, Vijay Menon, Neeraj Kumar, Hongbo DongMulti-Agent Safe Planning with Gaussian Processes.
ArXiv Link, IROS 2020
Zheqing Zhu, Erdem Biyik, Dorsa Sadigh
Education
PhD, Reinforcement Learning, Stanford University, Advisor: Benjamin Van Roy, 2023 (Completed while leading the Applied Reinforcement Learning team at Meta AI)
MS, Computer Science, Stanford University, 2020
BS, Computer Science, summa cum laude, Duke University, Advisor: Ronald Parr, 2017
Honors
CMO's Highlight Launch List, Facebook (Meta), 2021
Win of Month / Win of Quarter, Ads Growth, Facebook (Meta), 2017-2021 (Multi-time Winner)
Alex Vasilos Memorial Award, Duke University, 2017
Gradudate with Highest Distinction, Duke University, 2017
Ericsson BUSS Shanghai Quarterly Technical Award, 2015
Community Services
Workshop Chair of AAAI 2023 Reinforcement Learning Ready for Production Workshop
Reviewer: NeurIPS, AAAI, MLJ
Invited Talks
Reinforcement Learning for Recommender Systems, Neflix Research, 2023
Reinforcement Learning for Recommender Systems, DataFunSummit, 2023
Deep Exploration for Recommendation Systems, at University of Chinese Academy of Sciences, 2023
Deep Machine Learning Panel, at ML Summit San Francisco, 2019
Deep Reinforcement Learning Applications, at Shanshu.ai, 2019