Researchers led by Zheng Jiang and Lifeng Sun posted Frontier-OPD on October 23, 2026, a way to train small AI agents from a stronger teacher by spending the teacher’s help only where it is likely to pay off. On three agent benchmarks, the authors report that it beats five earlier distillation methods; with a Qwen3-32B teacher, a 1.7B student reached a 45.5% success rate on ALFWorld, versus 22.1% for plain on-policy distillation. It is a preprint and has not been peer reviewed.

The problem: where should a teacher step in?

Distillation trains a small “student” model to imitate a larger “teacher”. In on-policy distillation (OPD), the student acts on its own, and the teacher scores what the student did. For a single answer this is simple. For an agent that takes many turns, such as navigating a household simulator or shopping on a website, an early mistake can wreck everything after it, so the question becomes which moments deserve teacher guidance. Our explainer on knowledge distillation covers the basic idea, and our guide to AI agents covers multi-step systems.

Disagreement between student and teacher is an obvious signal, but the authors note it does not show whether the student can carry on productively after a correction. They define a “learning frontier”: situations the student has not mastered but can continue from after brief teacher guidance.

How Frontier-OPD works

The method follows a loop:

  1. Collect several student attempts at each task.
  2. Merge repeated situations across attempts into a shared graph, keyed by what the agent observed and which actions were available.
  3. Score each response by how much more the student favors it than the teacher does, then average per situation so that no single attempt dominates.
  4. Pick the highest-scoring situations, replay the environment up to that point and let the teacher act for one turn, a “bridge”.
  5. Let the student continue for three turns and compare its agreement with the teacher against an equally long stretch of the original attempt. Keep the bridge only if agreement did not get worse.
  6. Train on the student’s ordinary responses plus the accepted bridges, then collect fresh attempts, because the frontier moves as the student improves.

Reference settings were four attempts per task and four candidate situations, with training for 200 steps. The test students were Qwen3-1.7B and Qwen3-4B with thinking mode off, and teachers were Qwen3-32B and an RL-trained Qwen3-8B. Results are means over three seeds.

Results with a Qwen3-32B teacher

Success rates (SR) in percent, against plain OPD and the strongest earlier method, FutureBridge-OPD:

  • 1.7B, ALFWorld: OPD 22.1, FutureBridge-OPD 40.0, Frontier-OPD 45.5.
  • 1.7B, WebShop: OPD 33.7, FutureBridge-OPD 57.7, Frontier-OPD 58.3.
  • 1.7B, ScienceWorld: OPD 6.2, FutureBridge-OPD 10.2, Frontier-OPD 14.5.
  • 4B, ALFWorld: OPD 42.8, FutureBridge-OPD 48.5, Frontier-OPD 49.5.
  • 4B, ScienceWorld: OPD 8.2, FutureBridge-OPD 13.5, Frontier-OPD 16.2.

For scale, the zero-shot 1.7B student scored 8.2 on ALFWorld and the Qwen3-32B teacher 47.8. Some margins are within the reported spread: on 4B WebShop the gain over FutureBridge-OPD was 0.6 points with standard deviations around 1.5.

With the stronger Qwen3-8B-RL teacher, which scored 82.1 on ALFWorld, the 1.7B student reached 65.9 versus 55.2 for OPD, and the 4B student 74.6 versus 67.9. The paper reports Frontier-OPD had the best mean on every metric in that table.

Which pieces matter

In ablations with the 8B-RL teacher, removing the bridge data cut ALFWorld SR by about 3.7 points at both student sizes. Choosing situations at random did worst of all: 58.7 at 1.7B, against 65.9 for the full method. Dropping the filter that rejects unhelpful bridges was a smaller loss (63.7 at 1.7B).

Rows of server racks in a data center
Distillation runs like Frontier-OPD are trained on GPU servers; the paper used four NVIDIA B300 GPUs for its timing test. Photo: Carl Lender from Sunrise, USA / Wikimedia Commons, CC BY 2.0

On cost, the method has the highest time per training step on four NVIDIA B300 GPUs, since it collects extra data. The authors report that in one ALFWorld setting it still reached a 55% solve rate in the least elapsed time.

Limits

The authors list several. Disagreement is a proxy and may not reflect a missing skill. There were three seeds and no significance tests. The two teachers differ in size and training history, so the comparison cannot isolate either factor. The method needs a replayable environment and access to teacher token probabilities, and only three benchmarks were tested, so it is not shown to work for unrestricted real-world agents.

What to watch next

  • Replication by other groups, and tests on more realistic agent tasks.
  • Whether the approach works where environments cannot be replayed.
  • Significance testing across more seeds.
  • Whether the work is accepted at a peer-reviewed venue.

Key takeaways

  • Frontier-OPD spends teacher help only on situations the student can learn from and verifies each short intervention.
  • With a Qwen3-32B teacher, ALFWorld success for a 1.7B student rose from 22.1% (OPD) to 45.5%.
  • Random selection of intervention points did worst in ablations, so choosing where to help matters.
  • Extra data collection raises per-step cost, though one test reached a target solve rate sooner.
  • This is a non-peer-reviewed preprint on three simulated benchmarks.

Frequently asked questions

What is on-policy distillation?

It is a way to train a small model on its own attempts, with a stronger teacher model scoring them. This keeps the training data close to what the student actually does.

Is the Frontier-OPD paper peer reviewed?

No. It is an arXiv preprint submitted on October 23, 2026, and the page lists no conference or journal.

Does it need a bigger teacher model?

It needs a teacher that outperforms the student and exposes token probabilities. The paper used Qwen3-32B and an RL-trained Qwen3-8B.