Chengshuai Shi

Postdoctoral Fellow Princeton Language and Intellegence, Princeton University Email: chengshuai at princeton dot edu Phone: 434-218-9860 Google Scholar Profile

I am currently a Postdoctoral Fellow at the Princeton Language and Intelligence (PLI) initiative at Princeton University, where I work closely with Professor Chi Jin, Professor Karthik Narasimhan, and Professor Danqi Chen. Prior to joining PLI, I worked for a year as a Senior Machine Learning Research Engineer in the AI group at Bloomberg, New York City.

I received my Ph.D. in Electrical Engineering from the University of Virginia in 2024, where I was advised by Professor Cong Shen. During my Ph.D. (2021–2024), I was honored to be supported by the Bloomberg Data Science Ph.D. fellowship.

My research centers on intelligent decision-making, with a growing focus on integrating reinforcement learning and large language models. I develop principled methods grounded in reinforcement learning, multi-armed bandits, game theory, and multi-agent systems, and apply them to emerging problems in wireless communications, recommender systems, and language-model-based agents. My broader goal is to build adaptive, reliable, and scalable intelligent systems capable of learning and making decisions in complex, interactive environments.

News

• 09/2026: Three papers accepted to NeurIPS 2026!

  • “LeAct: Learning to Reason from Expert Actions”: LeAct teaches foundation models to reason from expert actions without annotated reasoning by generating and selecting explanations that improve action prediction, turning “silent” expert systems into reasoning teachers and outperforming direct imitation in games and simulated robotics.

  • “MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI”: MLS-Bench introduces a 140-task benchmark across 12 ML domains to test whether AI agents can invent generalizable and scalable machine-learning methods, finding that current agents are much better at engineering/tuning than at genuine methodological discovery.

  • “Skill-Coupled Policy Optimization with Calibrated Group-Wise Advantage Estimation”: SCPO improves RLVR by estimating advantage baselines using skill-related task groups rather than prompt-local or global averages, yielding more accurate credit assignment and more stable, higher-performing reasoning-model training.

• 05/2026: Project Odysseus is now released!

  • “Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning”, with the project website here

  • We study how to make reinforcement learning stable and effective for training VLM agents in long-horizon, visually grounded environments, using Super Mario Land as a testbed. Successful play in Super Mario Land often requires 100+ turns of closed-loop control, whereas existing VLM-RL work has mostly focused on shorter-horizon settings, typically around 20–30 turns. We propose Odysseus, a framework that combines lightweight SFT initialization with multi-task RL. The resulting trained model outperforms the base model by 5× and the strongest frontier model we evaluated by 3.6× in game performance. It also shows clear generalization to unseen levels and cross-game transfer to Super Mario Bros., while preserving the base model’s capabilities on general-purpose multimodal benchmarks.

  • 07/2026: Odysseus is accepted to COLM 2026!

• 04/2026: Two papers accepted to ICML 2026!

  • “f-Divergence Regularized RLHF: Two Tales of Sampling and Unified Analyses”: This work provides the first unified theory and provably efficient online RLHF algorithms for general f-divergence regularization, covering alternatives beyond reverse KL such as forward KL and chi-squared divergence.

  • “SMILE: Extended Deep Submodular Function-Based Instruction and In-context Learning Demonstration Selection”: This work proposes a framework SMILE to jointly optimize instructions and ICL demonstrations by modeling their interactions with a submodular surrogate, yielding more robust prompt optimization than separately tuning each component.

• 01/2026: One paper accepted to ICLR 2026!

  • “Efficient Multi-objective Prompt Optimization via Pure-exploration Bandits”: This work studies multi-objective prompt optimization under a budget constraint using tools from pure-exploration bandits. It extends the framework introduced in our previous work (NeurIPS 2024), which established a connection between bandit algorithms and prompt optimization.

• 10/2025: Will attend the 2025 EAS Trailblazers Symposium at Caltech as one of the seven trailblazers!