Multi-step agentic planning with internal world models
Multi-step agentic planning with internal world models
Want to keep this search?
Sign in to save it to your profile and mark papers as good, seen or bookmarked.
-
98 MatchArXiv 2026
Agentic World Modeling for Agent Test-time Scaling without Environment Replica
Wendi Li; Huseyin A. Inan; Wei-Ning Chen; Yanzhi Chen; Chetan Bansal; Saravan Rajmohan; Sharon Li
- TOPIC
- This paper investigates test-time scaling for autonomous agents using internal world models to perform look-ahead planning.
- NOVELTY
- It introduces an Agentic World Model (AWM) that enables search-based reasoning, like MCTS, to occur entirely within a latent state-space rather than relying on an external simulator.
- INSIGHT
- The key takeaway is that increasing the compute budget for internal "imagination" follows a scaling law that significantly boosts success rates in long-horizon tasks.
- RELEVANCE
- Since you are focused on multi-step agentic planning with internal world models, this work is directly applicable as it treats the world model as a searchable reasoning engine for inference.
Found Nov 2 -
98 MatchArXiv 2026
Train An Imaginative Agent
Jiaxin Ge; Junyi Zhang; Tianning Chai; Hanjun Yoo; Chenyang Wang; Xiuyu Li; Weijia Shi; XuDong Wang; Sewon Min; Luke Zettlemoyer; Trevor Darrell; Haiwen Feng
- TOPIC
- This paper presents Agima, a framework for autonomous agents that integrates a world model directly into an omni-model to enable multi-step planning via visual imagination.
- NOVELTY
- It unifies policy and world modeling by training the agent to generate interleaved sequences of action proposals and predicted visual outcomes, optimized further using GRPO.
- INSIGHT
- The key takeaway is that internalizing the planning process allows the agent to adaptively trigger counterfactual branching and long-horizon forward thinking only when it predicts a failure in its primary plan.
- RELEVANCE
- This is directly applicable to your work as it implements a scalable architecture for agentic planning using an internal world model to navigate environments with sparse feedback and irreversible actions.
Found Oct 21 -
95 MatchArXiv 2026
H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
Wancong Zhang; Basile Terver; Michael Rabbat; Yann LeCun; Randall Balestriero
- TOPIC
- This paper investigates long-horizon visual planning using a hierarchical Joint-Embedding Predictive Architecture (H-JEPA) for robotics.
- NOVELTY
- It introduces a temporal hierarchy where each level maintains its own distinct latent space to perform selective abstraction of the environment.
- INSIGHT
- By decomposing complex tasks into a top-down sequence of subgoals, the model creates smoother cost gradients for gradient-based planning compared to flat world models.
- RELEVANCE
- This is directly applicable to your work on multi-step agentic planning as it provides a concrete method for implementing internal world models that scale across different temporal abstractions.
Found Nov 3 -
95 MatchArXiv 2026
Beyond Next-Observation Prediction: Agent-Authored World Modeling for Sequential Decision Making
Guangfeng Cai; Kaibing Yang; Shuo He; Yu Li; Shengtian Yang; Jiaqi Lv; Lei Feng
- TOPIC
- This paper investigates world modeling for LLM agents in partially observed text-based environments to improve sequential decision-making.
- NOVELTY
- It introduces Agent-Authored World Modeling (AAWM), which replaces standard next-observation prediction with decision-driven targets generated from the agent's own beliefs and uncertainties.
- INSIGHT
- By using self-probing and cross-trajectory retrieval, the model learns to synthesize internal representations that specifically resolve the knowledge gaps needed for optimal action selection.
- RELEVANCE
- This is directly applicable to your work on multi-step agentic planning as it provides a concrete framework for training an internal world model that prioritizes task-relevant dynamics over raw environment reconstruction.
Found Nov 3 -
95 MatchArXiv 2026
Reinforcement World Model Learning for LLM-based Agents
Xiao Yu; Baolin Peng; Ruize Xu; Yelong Shen; Pengcheng He; Suman Nath; Nikhil Singh; Jiangfeng Gao; Zhou Yu
- TOPIC
- This paper investigates how to improve agentic intelligence in long-horizon environments by training LLMs to function as internal world models.
- NOVELTY
- The authors introduce Reinforcement World Model Learning (RWML), which uses GRPO to optimize for semantic equivalence of predicted next states rather than exact token matching.
- INSIGHT
- They show that this self-supervised mid-training approach allows the agent to better anticipate environment dynamics without needing expert trajectories or sparse rewards.
- RELEVANCE
- Since it directly addresses multi-step planning via a learned world model that reduces invalid actions, it is a direct match for your work on agentic planning.
Found Nov 3 -
95 MatchArXiv 2026
Beyond One-Step Accuracy: State-Affine Latent Transition for Reliable Visual Planning
Boyuan Zhang; Yingjun Du; Xiantong Zhen; Ling Shao
- TOPIC
- This paper investigates visual planning using joint-embedding world models to avoid the overhead of pixel-level reconstruction.
- NOVELTY
- They introduce SALT, a transition model with a state-affine architecture that ensures the Jacobian is independent of the latent state.
- INSIGHT
- The key takeaway is that restricting dynamics to be affine prevents the compounding of errors during multi-step rollouts, even if one-step prediction accuracy is lower.
- RELEVANCE
- Since you are working on multi-step agentic planning with internal world models, this approach to stabilizing recursive latent predictions is directly applicable to your work.
Found Oct 30 -
95 Match
- TOPIC
- This paper presents WEVA, a framework for autonomous agents that uses a predictive world model to handle long-horizon interactive planning.
- NOVELTY
- It introduces a modular architecture that maps interactions to quasi-symbolic states, allowing the agent to perform verified lookahead by rolling out and pruning action sequences based on predicted state transitions.
- INSIGHT
- The key takeaway is that using structured abstractions instead of free-text predictions enables the agent to explicitly detect and reject invalid action sequences before execution.
- RELEVANCE
- This is directly applicable to your work as it implements a concrete mechanism for multi-step agentic planning grounded in an evolving internal world model.
Found Oct 28 -
95 MatchEMNLP 2026
MemWM: Memory-Augmented Text-Based World Model
Yujun Wang; Tao Zhang; Jinhe Bi; Aniri; Wenxuan Ye; Boliang Liu; Sikuan Yan; Shuning Wang; Xuebing Zhou; Soren Pirk; Hinrich Schuetze; Yunpu Ma
- TOPIC
- This paper tackles the state-fidelity bottleneck in text-based world models used for sequential decision-making agents.
- NOVELTY
- It introduces MemWM, a memory-augmented architecture that uses a dual-channel system of world memory and world skills to condition next-state imagination on retrieved transition rules and state caches.
- INSIGHT
- The key takeaway is that augmenting the world model with explicit memory of environment dynamics significantly reduces hallucinations of behavior-critical facts during lookahead planning.
- RELEVANCE
- Since you are working on multi-step agentic planning with internal world models, this approach to improving state prediction fidelity without fine-tuning the policy is directly applicable to your work.
Found Oct 28 -
95 MatchEMNLP 2026
Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models
Youwei Liu; Jian Wang; Hanlin Wang; Beichen Guo; Wenjie Li
- TOPIC
- This paper investigates the problem of shallow grounding in LLM agents by implementing a textual world model for long-horizon task planning.
- NOVELTY
- They introduce the Imagine-then-Plan (ITP) framework, which uses a POIMDP formulation to generate imagined trajectories in a mental sandbox before executing actions.
- INSIGHT
- A key takeaway is the discovery of a bell curve relationship between lookahead horizon and success rate, meaning the agent must dynamically balance foresight against compounding world-model errors.
- RELEVANCE
- This is directly applicable to your work as it provides a formal method for multi-step agentic planning using an internal world model with an adaptive lookahead mechanism.
Found Oct 28 -
95 MatchEMNLP 2026
Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning
Xuan Zhang; Zhijian Zhou; Lingfeng Qiao; Yulei Qin; Ke Li; Xing Sun; Xiaoyu Tan; Chao Qu; Yuan Qi
- TOPIC
- This paper investigates how to move LLM agents from reactive policies to foresight-conditioned planning by internalizing a world model within a single autoregressive model.
- NOVELTY
- They introduce a three-stage training pipeline that uses mid-training, SFT, and GRPO-based RL to enable the agent to verbalize prospective state rollouts and calibrated success estimates (Q-values) before taking actions.
- INSIGHT
- The key takeaway is that predicting future states alone is insufficient; the agent needs calibrated confidence scores to effectively prune paths and guide multi-step planning.
- RELEVANCE
- This is directly applicable to your work as it provides a concrete method for implementing internal world models for long-horizon agentic planning without needing external simulators.
Found Oct 28 -
95 MatchICLR 2025
Empowering LLM Agents with Zero-Shot Optimal Decision-Making through Q-learning
Jiajun Chai; Sicheng Li; Yuqian Fu; Dongbin Zhao; Yuanheng Zhu
- TOPIC
- This paper investigates optimal decision-making for LLM agents in long-horizon MDP tasks using a model-based reinforcement learning approach.
- NOVELTY
- It introduces MLAQ, which combines a Q-Planner with an LLM-based imagination space that acts as a world model to predict next states and actions.
- INSIGHT
- The key takeaway is that using an internal world model to simulate transitions and a UCB-guided planner allows the agent to derive optimal policies with minimal environmental interaction.
- RELEVANCE
- This is directly relevant to your work as it implements a concrete architecture for multi-step agentic planning driven by an internal world model.
Found Oct 25 -
95 MatchEMNLP 2026
CoMAP: Co-Evolving World Models and Agent Policies for LLM Agents
Youwei Liu; Jian Wang; Hanlin Wang; Wenjie Li
- TOPIC
- This paper investigates the co-evolution of LLM agent policies and textual world models for long-horizon planning in interactive environments.
- NOVELTY
- It introduces COMAP, a framework that uses a closed-loop system where the world model is updated via on-policy self-distillation while the agent performs future-aware reflection to refine actions.
- INSIGHT
- The key takeaway is that static world models fail as agent policies evolve, and dynamically aligning the world model with the agent's current state-action distribution is critical for maintaining planning fidelity.
- RELEVANCE
- This is directly applicable to your work on multi-step agentic planning, as it provides a concrete mechanism for integrating an evolving internal world model into the decision-making loop.
Found Oct 25 -
95 MatchArXiv 2026
Steering Black-Box Large Language Model Agents with Pitfall Anticipation World Model
Jiongxiao Wang; Han Ding; Youzhi Luo; Yawei Wang; Yun Zhou; Linbo Liu; Panpan Xu
- TOPIC
- This paper investigates steering black-box LLM agents in long-horizon environments to avoid execution failures through predictive state-space modeling.
- NOVELTY
- It introduces the Pitfall Anticipation World Model (PAWM), an external cognitive layer that simulates future trajectories to identify and bypass critical failure points before they occur.
- INSIGHT
- The core takeaway is that explicitly modeling the "negative space" of outcomes provides a more effective steering mechanism for multi-step planning than standard self-reflection or CoT.
- RELEVANCE
- Since you are focused on agentic planning with internal world models, this approach to decoupling execution from anticipation via a predictive world model is directly applicable to your work.
Found Oct 23 -
95 MatchArXiv 2026
Self-Improving Agents with World Model Predictive Paths
Haori Lu; Tianyu Gao; Yaoyao Liu; Yingying Li
- TOPIC
- This paper investigates decision-making for LLM agents in long-horizon text environments using a predict-select-replan loop.
- NOVELTY
- It introduces an MPPI-inspired framework that uses a language-based world model to imagine multiple candidate outcomes and weights them via a Gibbs distribution to select the optimal action.
- INSIGHT
- The key takeaway is that separating the world model's predictive accuracy from the planner's action proposal through a dual-branch memory significantly reduces per-step regret.
- RELEVANCE
- This is directly applicable to your work as it implements a concrete architecture for multi-step agentic planning driven by an internal world model.
Found Oct 23 -
95 MatchArXiv 2026
Agentic vs. Parametric World Models: Hybrid Planning for Reliable Language Agents
Xinyuan Song; Zekun Cai
- TOPIC
- This paper investigates reliability in multi-step planning for language agents operating in graph-structured environments.
- NOVELTY
- It introduces Hybrid-WM, a framework that pairs a semantic LLM planner with a lightweight parametric world model to detect and correct hallucinated state transitions.
- INSIGHT
- The key takeaway is that using a parametric model as a grounding mechanism to check consistency against the LLM's imagined states prevents error propagation over long horizons.
- RELEVANCE
- This is directly applicable to your work on agentic planning with internal world models, specifically regarding how to mitigate the drift between an agent's internal beliefs and actual environment dynamics.
Found Oct 23 -
95 MatchArXiv 2026
World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems
Runze Li; Hongyin Zhang; Junxi Jin; Qixin Zeng; Zifeng Zhuang; Yiqi Tang; Shangke Lyu; Donglin Wang
- TOPIC
- The paper investigates long-horizon decision-making for embodied agents using a Vision-Language-Action (VLA) framework.
- NOVELTY
- It introduces the World-Value-Action (WAV) model, which replaces direct action prediction with iterative latent-space trajectory optimization using a Diffusion Transformer.
- INSIGHT
- The core takeaway is that planning in a learned latent manifold avoids the "Curse of Feasibility" associated with high-dimensional action spaces, allowing for more robust multi-step reasoning.
- RELEVANCE
- This is directly applicable to your work as it implements a concrete internal world model via video generation coupled with a value function to drive agentic planning.
Found Oct 14 -
95 MatchArXiv 2026
LeWAM: End-to-End World and Action Modeling with JEPAs
Minghao Fu; Tavis Siebert; Eryk Halicki; Randall Balestriero
- TOPIC
- This paper investigates goal-conditioned visual planning and robotic manipulation using a Joint-Embedding Predictive Architecture (JEPA).
- NOVELTY
- It introduces LeWAM, which jointly trains a latent world model and a stochastic policy via flow matching to enable both direct execution and gradient-based planning.
- INSIGHT
- The key takeaway is that using a learned action distribution to seed candidates, followed by gradient refinement through the latent dynamics, drastically reduces planning latency compared to CEM while improving success in contact-rich tasks.
- RELEVANCE
- This is directly applicable to your work as it implements a multi-step agentic planning loop powered by an internal world model that avoids the pitfalls of pixel-reconstruction.
Found Oct 12 -
95 MatchArXiv 2026
SAGE: Subgoal-Conditioned Action Generation for Latent World Model Planning
Letian Cheng; Qi Zhang; Yisen Wang
- TOPIC
- This paper investigates long-horizon goal-conditioned planning in robotic manipulation using latent world models.
- NOVELTY
- It introduces SAGE, a framework that couples a multi-horizon latent subgoal generator with a subgoal-conditioned action prior to guide the search process.
- INSIGHT
- The key takeaway is that decomposing a distant goal into intermediate latent subgoals prevents the proposal distribution from collapsing in high-dimensional action spaces.
- RELEVANCE
- Since you are focused on multi-step agentic planning with internal world models, this approach to structured latent subgoal decomposition is directly applicable to your work.
Found Oct 11 -
95 MatchACL 2026
Model-Based Imaginative Planning for Embodied Agents
Junru Song; Hengzhe Jin; Yucong Huang; Tingsong Jiang; Weien Zhou; Feifei Wang; Yang Yang; Ying Wen; Wen Yao
- TOPIC
- This paper investigates long-horizon embodied sequential decision-making in POMDPs using a model-based reasoning framework called IMPLEMENT.
- NOVELTY
- It introduces a symbolic world model that decouples perception from prediction, allowing a frozen LLM to perform 'imagine-evaluate-refine' loops via Monte Carlo rollouts.
- INSIGHT
- The use of Meta In-Context Learning to adapt the world model to environment-specific dynamics online provides a scalable way to handle epistemic uncertainty without parameter updates.
- RELEVANCE
- This is directly applicable to your work as it implements a concrete architecture for multi-step agentic planning driven by an internal world model.
Found Oct 10 -
95 Match
- TOPIC
- This paper explores model-based planning in goal-conditioned MDPs using a pretrained latent world model for visual-control tasks.
- NOVELTY
- It introduces RP1, which replaces hand-designed search algorithms with a learned neural planner that iteratively refines action sequences based on value gradients from a critic.
- INSIGHT
- The core takeaway is that learning a plan-improvement operator allows the agent to outperform traditional optimizers like CEM while reducing world-model rollouts by three orders of magnitude.
- RELEVANCE
- This is directly applicable to your work as it implements a multi-step agentic planning loop that leverages an internal world model to optimize trajectories.
Found Oct 8 - Loading more results…