Zheqing (Bill) Zhu
Head of Applied Reinforcement Learning
Facebook (Meta) AI
Zheqing (Bill) Zhu is an Engineering Manager at Facebook (Meta) AI, where he serves as the Head of Applied Reinforcement Learning. His main interest lies in bringing state-of-the-art reinforcement learning technologies to real-life and bridging the gap between theoretical reinforcement learning and real-world systems. Prior to serving as Head of Applied Reinforcement Learning, he was the engineering manager and tech lead for Facebook (Meta)'s Ads Growth Machine Learning team, where he built the team from scratch and enabled exponential growth in business market for Facebook (Meta).
While working full-time at Facebook (Meta), he is pursuing a PhD degree in Reinforcement Learning at Stanford University, advised by Professor Benjamin Van Roy. His main research focus is to understand theoretical and pratical gaps in existing reinforcement learning algorithms when integrated with real-life recommendation systems. He received Master of Science in Computer Science from Stanford University in 2019, which was also completed while working full-time at Facebook (Meta). He received Bachelor of Science in Computer Science with a Minor in Finance, summa cum laude, from Duke University in 2017 within 3 years. He has been the recipient of the Alex Vasilos Memorial Award and the Highest Distinction Graduate Award from Duke University and Ericsson BUSS Shanghai Quarterly Technical Award.
Engineering Manager - Head of Applied Reinforcement Learning, Facebook (Meta) AI, 2021 - now
Engineering Manager / Tech Lead, Ads Growth Machine Learning, Facebook (Meta), 2018 - 2021
Machine Learning Engineer, Ads Growth Machine Learning, Facebook (Meta), 2017 - 2018
Selected Publicly Available Research
Optimizing Long-term Value for Auction-Based Recommender Systems via On-Policy Reinforcement Learning.
ArXiv Link, RecSys 2023 (Also presented at KDD Workshop 2023)
Ruiyang Xu*, Jalaj Bhandari*, Dmytro Korenkevych, Fan Liu, Yuchen He, Alex Nikulkov, Zheqing Zhu (PI)
Deep Exploration for Recommendation Systems.
ArXiv Link, RecSys 2023
Zheqing Zhu, Benjamin Van Roy
Scalable Neural Contextual Bandit for Recommender Systems.
ArXiv Link, CIKM 2023 (also presented at KDD Workshop 2023)
Zheqing Zhu, Benjamin Van Roy
IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Control.
ArXiv Link, ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023
Rohan Chitnis*, Yingchen Xu*, Bobak Hashemi, Lucas Lehnert, Urun Dogan, Zheqing Zhu (Secondary PI), Olivier Delalleau
Evaluating Online Bandit Exploration In Large-Scale Recommender System.
ArXiv Link, KDD Workshop on Multi-Armed Bandits and Reinforcement Learning: Advancing Decision Making in E-Commerce and Beyond, 2023
Hongbo Guo, Ruben Naeff, Alex Nikulkov, Zheqing Zhu (PI)
Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation via Deep Reinforcement Learning.
ArXiv Link, Submitted to CoNext, 2023
Chang-Lin Chen, Hanhan Zhou, Jiayu Chen, Mohammad Pedramfar, Vaneet Aggarwal, Tian Lan, Zheqing Zhu (RL PI), Chi Zhou, Tim Gasser, Pol Mauri Ruiz, Vijay Menon, Neeraj Kumar, Hongbo Dong
Learning to Bid and Rank Together in Recommendation Systems.
(ArXiv Coming Soon), Submitted to MLJ, 2023
Geng Ji, Wentao Jiang, Jiang Li, Fahmid Morshed Fahid, Zhengxing Chen, Yinghua Li, Jun Xiao, Chongxi Bao and Zheqing Zhu (PI)
Multi-Agent Safe Planning with Gaussian Processes.
ArXiv Link, IROS 2020
Zheqing Zhu, Erdem Biyik, Dorsa Sadigh
PhD, Reinforcement Learning, Stanford University, 2023 (expected), Advisor: Benjamin Van Roy
MS, Computer Science, Stanford University, 2019
BS, Computer Science, summa cum laude, Duke University, 2017, Advisor: Ronald Parr
CMO's Highlight Launch List, Facebook (Meta), 2021
Win of Month / Win of Quarter, Ads Growth, Facebook (Meta), 2017-2021 (Multi-time Winner)
Alex Vasilos Memorial Award, Duke University, 2017
Gradudate with Highest Distinction, Duke University, 2017
Ericsson BUSS Shanghai Quarterly Technical Award, 2015
Workshop Chair of AAAI 2023 Reinforcement Learning Ready for Production Workshop
Reviewer: NeurIPS, AAAI, MLJ
Reinforcement Learning for Recommender Systems, DataFunSummit, 2023
Deep Exploration for Recommendation Systems, at University of Chinese Academy of Sciences, 2023
Deep Machine Learning Panel, at ML Summit San Francisco, 2019
Deep Reinforcement Learning Applications, at Shanshu.ai, 2019