PILOT: Guiding Long-Horizon World-Model Planning for Robot Manipulation with Visual Subgoals and Policy Priors

Yang Liu^*, Hanchen Cui^*, and Karthik Desingh

* denotes equal contribution

University of Minnesota Twin Cities

68.8%Real-robot successFour tabletop pick-and-place tasks

+30ppLong Push-T improvementOver direct execution of the same policy

60%Fewer CEM iterationsSame planning cost: 4 vs. 10 on Push-T

Visual subgoals meet policy-guided search

A distant goal tells the robot where to finish. Visual subgoals reveal how to get there,
and a policy prior gives each local search a useful starting point.

Enlarge

Temporal guidance breaks a distant objective into local targets. Behavioral guidance makes the search toward each target more efficient.

See the intermediate steps

A video model predicts visual subgoals from the current observation and task instruction.

Start with a policy prior

A learned policy initializes CEM around behaviorally plausible action sequences.

Refine before execution

An action-conditioned world model evaluates candidates. The robot executes a short prefix and replans.

Abstract

Latent world-model planning has shown promising results in goal-conditioned robotic manipulation, but remains challenging for long-horizon planning due to limited temporal guidance and inefficient action search. A distant visual goal provides little guidance for preparatory actions, while uninformed action sampling makes local optimization inefficient. We propose PILOT (Policy-Initialized Local Optimization with Temporal subgoals), which uses a video model to generate intermediate visual subgoals and a learned policy to initialize the cross-entropy method (CEM). CEM refines candidate action sequences toward each subgoal using an action-conditioned world model in a receding-horizon control loop.

In our primary real-robot evaluation across four UR5e manipulation tasks, PILOT achieves 68.8% success, compared with 55.0% without policy initialization, 13.8% for policy-only execution, and 0.0% for direct final-goal planning. Simulation results show that these gains persist over longer horizons: PILOT exceeds direct execution of the same policy by 30 percentage points on long Push-T tasks and by 22 percentage points on LIBERO-10.

Policy initialization also accelerates local optimization: on Push-T, CEM matches the cost reached by generic initialization after 10 iterations in only 4 iterations and reduces the final planning cost by 24.1% under the same sampling budget. These results show that combining temporal subgoals with policy-guided action search improves both long-horizon task execution and local planning efficiency.

PILOT framework

Three complementary models, connected by a closed-loop planner.

PILOT architecture: a video model generates visual subgoals, a policy proposes an action prior, and CEM refines actions using predicted latent states and the active visual target. — The video model proposes task progression; the policy proposes where to search. The action-conditioned world model evaluates and refines actions toward the active target.

Implementation details

Wan2.1-1.3B

Predict temporal subgoals

Generate a future video from the initial observation and language instruction. Select uniformly spaced frames as intermediate planning targets.

π₀ + CEM

Initialize and optimize

Use the policy’s action proposal as the initial CEM mean, then refine candidates against the current subgoal rather than executing the policy directly.

JEPA-WM + MPC

Execute and replan

Minimize terminal latent distance to the target. Execute the first action, observe the new state, and repeat before advancing to the next subgoal.

Real-world control: 6 visual targets · prediction horizon h = 2 · 1 action executed per update · 2 planning–execution cycles per target.

Four Real-World Tasks

Long-horizon tabletop manipulation on a UR5e with a fixed RGB camera.

×4

Cup

11/20 successes

×4

Pencil Case

13/20 successes

×4

Bowl-Box

13/20 successes

×4

Bowl-Bowl

18/20 successes

Real robot demonstrations shown at 4× speed. Select Enlarge for a larger player.

Four real-world task panels. In each panel, the top row shows the initial observation and six predicted subgoals; the bottom row shows the robot’s actual observations as it executes the plan. — **Predicted subgoals → robot execution.** The top row of each panel shows video-predicted targets; the bottom row shows the robot observation when each subgoal expires. Swipe sideways to explore the figure on smaller screens.

Real-World Experimental Results

20 trials per task, 80 trials per method. All world-model variants share the same short prediction horizon.

Average task success Higher is better ↑

Flat WM

0.0%

Policy only

13.8%

PILOT w/o prior

55.0%

PILOT

68.8%

Visual subgoals provide temporal guidance. Policy initialization adds a further 13.8 percentage points in average success.

Real-world task success. Average pools all 80 trials; intervals are 95% Wilson confidence intervals.
Method	Cup	Pencil Case	Bowl-Box	Bowl-Bowl	Average	95% CI
Flat WM	0/20	0/20	0/20	0/20	0.0%	[0.0, 4.6]
Policy only	4/20	3/20	2/20	2/20	13.8%	[7.9, 23.0]
PILOT w/o prior	8/20	5/20	14/20	17/20	55.0%	[44.1, 65.4]
PILOT Ours	11/20	13/20	13/20	18/20	68.8%	[57.9, 77.8]

Planning efficiency

Policy initialization improves performance under limited planning budgets
and provides a lower-cost starting point for CEM refinement.

Real-world success versus planning time: planning with a policy prior achieves higher success under constrained online planning budgets. — **Success vs. planning time.** Policy-guided initialization reaches strong performance with less search.

Real-world planning cost across CEM iterations: policy-prior initialization starts at a lower cost and maintains lower cost during optimization. — **Cost during CEM refinement.** A useful behavioral prior improves local optimization.

Complementary gains across environments

Controlled studies of task length, policy initialization, and the source of visual subgoals.

Push-T: scaling with task horizon

30 held-out initial states per length bucket. The prediction horizon stays fixed as task duration increases.

+30 pp on long tasks vs. policy only

Success by task length. PILOT uses the same 30-demonstration policy as the corresponding Policy only baseline.
Method	Short	Medium	Long
Flat WM	33.3%	0.0%	0.0%
Flat WM + prior	46.7%	0.0%	0.0%
Policy only 5 demos	0.0%	0.0%	0.0%
Policy only 10 demos	13.3%	3.3%	6.7%
Policy only 20 demos	26.7%	16.7%	10.0%
Policy only 30 demos	40.0%	33.3%	23.3%
PILOT w/o prior	43.3%	36.7%	40.0%
PILOT Ours	80.0%	63.3%	53.3%
PILOT (oracle) Expert subgoals	90.0%	93.3%	83.3%

Push-T predicted visual subgoals and simulator observations as the robot pushes a T-shaped block toward the target pose. — Video-predicted visual subgoals (top) and simulator observations when each subgoal expires (bottom). Swipe sideways to explore the figure on smaller screens.

Planning efficiency and the strength of the policy prior

Even a weak prior improves the success–planning-time trade-off. The 30-demonstration prior exceeds 60% medium-task success at 8 seconds per planning call. Policy initialization reaches in four iterations the cost that generic initialization reaches in ten, and finishes 24.1% lower under the same sampling budget.

Push-T medium-task success versus planning time for generic initialization and policy priors trained on 10, 20, and 30 demonstrations. Curves are saturating-exponential fits. — Stronger priors reach higher success with less planning time; curves are saturating-exponential fits.

Normalized CEM cost on identical planning problems from 90 held-out Push-T initial states, comparing policy initialization with generic initialization. Shading shows standard errors. — Normalized objective on 90 held-out initial states; shading indicates standard errors.

Short: expert episode length < 100 steps. Medium: 100–129. Long: ≥ 130. Wan uses 100 Push-T video demonstrations; the main policy prior uses 30 action demonstrations.

LIBERO-10: long-horizon manipulation

Ten language-conditioned, two-stage tasks on a Franka arm. Five held-out initial states per task, 50 trials per method.

+22 pp over direct policy execution

Aggregate success across the ten LIBERO tasks.
Method	Success rate
Flat WM	0.0%
Flat WM + prior	0.0%
Policy only	38.0%
PILOT w/o prior	22.0%
PILOT Ours	60.0%
PILOT (oracle) Expert subgoals	88.0%

Predicted subgoals and simulator observations for the LIBERO-10 instruction: put the black bowl in the bottom drawer of the cabinet and close it. — “Put the black bowl in the bottom drawer of the cabinet and close it.” Predicted subgoals above, execution below. Swipe sideways to explore the figure on smaller screens.

About the oracle reference. PILOT (oracle) replaces generated targets with frames from a held-out expert trajectory while keeping the policy, world model, and controller unchanged. It measures the effect of subgoal source and is not assumed to be a strict upper bound.

BibTeX

If you find PILOT useful for your research, please cite our paper using the following BibTeX entry:

@article{liu2026pilot,
  title={PILOT: Guiding Long-Horizon World-Model Planning for Robot Manipulation with Visual Subgoals and Policy Priors},
  author={Liu, Yang and Cui, Hanchen and Desingh, Karthik},
  journal={arXiv preprint arXiv:2610.14335},
  year={2026}
}