Jiajun Fan

Jiajun Fan

Open to Research Internships — Fall 2026 / 2027

RL Post-Training for Generative Models Multimodal Reasoning LLMs Superhuman Deep RL Agentic RL Recursive Self-Improvement

CV Scholar Email OpenReview GitHub LinkedIn

↓ Jump to: News Featured Publications North Star Research Impact Vision Awards Deadlines Blog CV Contact ⌘K

I am a CS Ph.D. student at UIUC, where I study how to train multimodal large generative/agentic models with reinforcement learning in a way that is stable (e.g., lifelong continual learning) — avoiding the diversity and performance collapse that RL fine-tuning tends to cause — and progressively autonomous (e.g., recursive self-improvement, RSI), step by step taking the human out of the loop: first human data collection, then human labelling and hyper-parameter tuning, and finally hand-designed rewards. That work spans these domains:

A ONE-MINUTE FILM From RL to AGI The research above as one story: taking the human out of the loop, step by step, toward an AI scientist.

Latest News

• Oct 2026 TalkGave a talk at the IGB (Lunch with CNRG) on building LLM post-training projects with an AI coding agent. Always happy to give talks on LLM post-training with RL and on AI scientists — agentic methods for automated scientific discovery — just reach out. Event
• Oct 2026 CodeProteinZero is accepted at TMLR and now open source — feel free to use it, for example as a tool in your own agent or design pipeline, and I am always happy to discuss collaborations. Project page · Code
• Sep 2026 AcceptMultiple papers accepted at NeurIPS 2026. See you in Atlanta 
• Sep 2026 AcceptPaper accepted at IEEE TMM (Transactions on Multimedia).
• Aug 2026 FindingVoice agents could be measured, never improved — so we closed the loop and trained one in native audio. Blog · Paper
• Aug 2026 FindingFor the first time, we can see the brain map of an Audio LLM — its global workspace, the “consciousness space”. Blog · Paper
• Jul 2026 AcceptPaper accepted at TMLR — first online-RL framework for self-improving protein generation.
• Jul 2026 FeatureCESAR is now featured on Amazon Science.
• Jul 2026 AcceptPapers accepted at COLM 2026 (San Francisco, Oct 6–9). See you in San Francisco 
• Apr 2026 AcceptSeveral papers accepted at ICML 2026 (Seoul, Jul 6–11). See you in Seoul 
• Apr 2026 TalkPresented CESAR & SP-VLA at ICLR 2026, Rio de Janeiro, Apr 23–27 
• Jan 2026 Accept2 papers at ICLR 2026 — CESAR & SP-VLA. See you in Rio 
• Sep 2025 Accept2 papers at NeurIPS 2025 — ADRPO & VarCon. See you in San Diego 
• Jun 2025 AcceptPaper accepted at IEEE TPAMI: PRANCE.
• Feb 2025 AcceptPaper accepted at ICLR 2025: ORW-CFM-W2 (Flow Matching self-evolution).
• Jan 2025 ServiceReviewer: ICLR 2024–26, NeurIPS 2022–25, ICML 2023–26, CVPR 2026, AAAI 2025, AISTATS 2025, KDD 2024.
• Aug 2024 Started Ph.D. at UIUC CS (GPA 4.0/4.0).
• Jan 2023 Oral · Top 5%LBC at ICLR 2023, ranked 5/4176 — broke 24 Atari world records.

▼ Show more news

Featured Research

ICLR 2026

CESAR: Process Rewards for Audio LLM Reasoning

First to show test-time inverse scaling in Audio LLMs — and fix it with process rewards via online RL.

SOTA MMAU · Beats Gemini 2.5 Pro

NeurIPS 2025

ADRPO: Adaptive Regularization for Generative Model RLHF

Sample-level adaptive divergence regularization — 2B model outperforms 12B competitors.

2B SD3 > FLUX 12B & SANA 4.8B

ICLR 2025

ORW-CFM-W2: Online RLHF for Flow Matching

First collapse-free online RL framework for flow matching models with W2 regularization.

First online RLHF for flow matching

ICLR 2023 · Oral

LBC: Superhuman Atari — 24 World Records

Breaking 24 Atari human world records with 78× less data than prior SOTA (Agent57).

Rank 5/4176 · 10,077% mean human performance

Selected Publications

* = first/co-first author  ·  Full list on Google Scholar  /  Publications page

ICLR 20262026

Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards ▾ Abstract

CESAR resolves test-time inverse scaling in Audio LLMs by rewarding the reasoning process via GRPO, achieving SOTA on MMAU — outperforming Gemini 2.5 Pro and GPT-4o Audio.

J. Fan*, R. Ren, J. Li, R. Pandey, P.G. Shivakumar, I. Bulyko, A. Gandhe, G. Liu, Y. Gu

CESAR: process-reward RL (GRPO) resolving test-time inverse scaling in Audio LLMs — models produce hallucinatory reasoning without proper guidance; CESAR fixes that.

SOTA on MMAU Test-mini · Outperforms Gemini 2.5 Pro & GPT-4o Audio

Paper Project arXiv Amazon Science

ICLR 20262026

SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration ▾ Abstract

SP-VLA introduces action-aware model scheduling and spatio-semantic token pruning for VLA model acceleration, achieving 1.5× lossless speedup on LIBERO and 2.4× speedup on SimplerEnv.

Y. Li, Y. Meng, Z. Sun, K. Ji, C. Tang, J. Fan, X. Ma, S.-T. Xia, Z. Wang, W. Zhu

Action-aware model scheduling + spatio-semantic token pruning for VLA acceleration.

1.5× lossless speedup (LIBERO) · 2.4× speedup (SimplerEnv)

Paper

NeurIPS 20252025

Adaptive Divergence Regularized Policy Optimization for Fine-tuning Generative Models ▾ Abstract

ADRPO introduces sample-level adaptive divergence regularization for RLHF — high-value samples get more freedom, poor samples get stronger constraints. Plug-and-play on any RL method.

J. Fan*, T. Wei, C. Cheng, Y. Chen, G. Liu

ADRPO: sample-level adaptive divergence regularization — high-value samples get more freedom, poor samples get stronger constraint. Plug-and-play on top of any RLHF method.

2B SD3 surpasses 4.8B & 12B models · Generalizes to LLMs & audio reasoning

Paper Project arXiv

NeurIPS 20252025

Variational Supervised Contrastive Learning ▾ Abstract

VarCon reformulates supervised contrastive learning as variational inference, achieving SOTA 79.36% Top-1 accuracy on ImageNet-1K with ResNet-50.

Z. Wang, J. Fan, T. Nguyen, H. Ji, G. Liu

VarCon: supervised contrastive learning as variational inference — posterior-weighted ELBO replaces pairwise comparisons.

SOTA 79.36% Top-1 on ImageNet-1K with ResNet-50

Paper

ICLR 20252025

Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularization ▾ Abstract

ORW-CFM-W2 is the first online RLHF method for flow matching — no human data, no likelihood estimation. Wasserstein regularization maintains generation diversity.

J. Fan*, S. Shen, C. Cheng, Y. Chen, C. Liang, G. Liu

ORW-CFM-W2: first online RLHF for flow matching — no human data, no likelihood, no collapse. W2 regularization keeps generation diverse.

First online RLHF for flow matching · Collapse-free W2 regularization

Paper Project arXiv

Preprint2025

Fine-tuning Flow Matching Generative Models with Intermediate Feedback ▾ Abstract

AC-Flow introduces actor-critic with intermediate feedback for flow matching — reward shaping + dual-stability mechanism + Wasserstein regularization enables robust SD3 fine-tuning without collapse.

J. Fan*, C. Cheng, S. Shen, X. Zhou, G. Liu  ·  Under Review

AC-Flow: actor-critic with intermediate feedback for flow matching — reward shaping + dual-stability + Wasserstein regularization. Robust fine-tuning on SD3 without collapse.

Actor-critic with step-level reward · Stable SD3 fine-tuning without collapse

Project arXiv

TPAMI 20262026

PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inference ▾ Abstract

PRANCE jointly optimizes token pruning and structural channel pruning for adaptive ViT inference, achieving significant speedup while maintaining accuracy.

Y. Li, C. Tang, Y. Meng, J. Fan, Z. Chai, X. Ma, Z. Wang, W. Zhu  ·  IEEE TPAMI

Joint token + channel pruning · Adaptive ViT inference · IEEE TPAMI 2026

Paper arXiv

ICLR 2023
Oral2023

Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selection ▾ Abstract

LBC introduces a learnable hybrid behavior mapping and bandit meta-controller for exploration control in deep RL, breaking 24 Atari human world records with 78× less data than prior SOTA.

J. Fan*, Y. Zhuang, Y. Liu, J. Hao, B. Wang, J. Zhu, H. Wang, S.-T. Xia

LBC: learnable hybrid behavior mapping + bandit meta-controller. Unified framework for exploration control in deep RL.

Ranked 5/4176 · 10,077% mean human score · 24 world records · 78× data efficiency

Paper Project arXiv

ICML 20222022

Generalized Data Distribution Iteration ▾ Abstract

GDI shows that optimizing the training data distribution is the key lever for superhuman RL efficiency. Provides a unified framework that subsumes diverse RL algorithms as special cases.

J. Fan*, C. Xiao

GDI: optimizing the data distribution is the key to superhuman RL efficiency. Unified framework for diverse RL algorithms.

Agent57 beaten with 500× less data & 2× avg performance

Paper Project arXiv

No publications match your search. Browse the full list →

Research Paper Network

Hover a node to highlight connections. Papers are grouped by research theme.

Research North Star

My long-term goal is AI that improves itself with minimal human supervision. Everything I build ladders into three autonomous capabilities:

Auto Self-Evolution

Models that improve themselves

Generative models and reasoning LLMs that self-improve from their own outputs and environment feedback — no human-labeled data. Reward-weighted, collapse-free, process-supervised online RL.

ORW-CFM-W2ADRPOAC-FlowCESAR

Auto Exploration

Agents that explore on their own

Sample-efficient, self-directed exploration that enlarges what an agent can discover — learnable behavior control and data-distribution optimization that reached superhuman play with orders of magnitude less data.

LBCGDI

Auto Research

AI that does research

The horizon: agents that run the full scientific loop — hypothesize, experiment, analyze, refine — autonomously. Agentic RL that turns self-evolution and self-exploration into self-driven discovery.

Agentic RLOngoing

Research Interests

RL Post-Training for Generative Models

Collapse-free online RLHF for flow/diffusion models. No human-collected preference data needed — models improve from their own generations (ORW-CFM-W2, ADRPO, AC-Flow).

Multimodal Reasoning LLMs

Process-reward RL for audio/visual LLMs — fixing test-time inverse scaling so reasoning actually helps, not hurts (CESAR).

Superhuman Deep RL

Sample-efficient RL that exceeds human performance. Broke 24 Atari world records with 78× less data (LBC), and beat Agent57 with 500× less data (GDI).

Recursive Self-Improvement (RSI)

Models that keep improving themselves: each improved policy becomes the critic and teacher of its successor, removing human scaffolding from RL post-training step by step.

Impact at a Glance

Research Vision

Making AI Systems That Improve Themselves

Today's AI is frozen after training. I work to change that: AI that never stops getting better, with progressively less human scaffolding.

Awards & Academic Service

Conference Deadlines

Key AI/ML venue deadlines I track — for the full list see ccfddl.com.

Contact

Happy to discuss research, internships, or collaborations. Best reached by email.
jiajunf3@illinois.edu  ·  Siebel Center for CS, UIUC  ·  CV  ·  LinkedIn  ·  ORCID

Email Scholar OpenReview GitHub CV Projects

© 2026 Jiajun Fan · CS Ph.D. @ UIUC · Built with and curiosity

↑

Ask my AI agent