PaperManta Sign in
Active · from 2025 onwards (ongoing) · Shared by Marek

Multi-step agentic planning with internal world models

Multi-step agentic planning with internal world models

Want to keep this search?

Sign in to save it to your profile and mark papers as good, seen or bookmarked.

Sign in to save
564 results
  • ArXiv 2026

    Agentic World Modeling for Agent Test-time Scaling without Environment Replica

    Wendi Li; Huseyin A. Inan; Wei-Ning Chen; Yanzhi Chen; Chetan Bansal; Saravan Rajmohan; Sharon Li

    98 Match
    TOPIC
    This paper investigates test-time scaling for autonomous agents using internal world models to perform look-ahead planning.
    NOVELTY
    It introduces an Agentic World Model (AWM) that enables search-based reasoning, like MCTS, to occur entirely within a latent state-space rather than relying on an external simulator.
    INSIGHT
    The key takeaway is that increasing the compute budget for internal "imagination" follows a scaling law that significantly boosts success rates in long-horizon tasks.
    RELEVANCE
    Since you are focused on multi-step agentic planning with internal world models, this work is directly applicable as it treats the world model as a searchable reasoning engine for inference.
    Found Nov 2
  • ArXiv 2026

    Train An Imaginative Agent

    Jiaxin Ge; Junyi Zhang; Tianning Chai; Hanjun Yoo; Chenyang Wang; Xiuyu Li; Weijia Shi; XuDong Wang; Sewon Min; Luke Zettlemoyer; Trevor Darrell; Haiwen Feng

    98 Match
    TOPIC
    This paper presents Agima, a framework for autonomous agents that integrates a world model directly into an omni-model to enable multi-step planning via visual imagination.
    NOVELTY
    It unifies policy and world modeling by training the agent to generate interleaved sequences of action proposals and predicted visual outcomes, optimized further using GRPO.
    INSIGHT
    The key takeaway is that internalizing the planning process allows the agent to adaptively trigger counterfactual branching and long-horizon forward thinking only when it predicts a failure in its primary plan.
    RELEVANCE
    This is directly applicable to your work as it implements a scalable architecture for agentic planning using an internal world model to navigate environments with sparse feedback and irreversible actions.
    Found Oct 21
  • ArXiv 2026

    H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning

    Wancong Zhang; Basile Terver; Michael Rabbat; Yann LeCun; Randall Balestriero

    95 Match
    TOPIC
    This paper investigates long-horizon visual planning using a hierarchical Joint-Embedding Predictive Architecture (H-JEPA) for robotics.
    NOVELTY
    It introduces a temporal hierarchy where each level maintains its own distinct latent space to perform selective abstraction of the environment.
    INSIGHT
    By decomposing complex tasks into a top-down sequence of subgoals, the model creates smoother cost gradients for gradient-based planning compared to flat world models.
    RELEVANCE
    This is directly applicable to your work on multi-step agentic planning as it provides a concrete method for implementing internal world models that scale across different temporal abstractions.
    Found Nov 3
  • ArXiv 2026

    Beyond Next-Observation Prediction: Agent-Authored World Modeling for Sequential Decision Making

    Guangfeng Cai; Kaibing Yang; Shuo He; Yu Li; Shengtian Yang; Jiaqi Lv; Lei Feng

    95 Match
    TOPIC
    This paper investigates world modeling for LLM agents in partially observed text-based environments to improve sequential decision-making.
    NOVELTY
    It introduces Agent-Authored World Modeling (AAWM), which replaces standard next-observation prediction with decision-driven targets generated from the agent's own beliefs and uncertainties.
    INSIGHT
    By using self-probing and cross-trajectory retrieval, the model learns to synthesize internal representations that specifically resolve the knowledge gaps needed for optimal action selection.
    RELEVANCE
    This is directly applicable to your work on multi-step agentic planning as it provides a concrete framework for training an internal world model that prioritizes task-relevant dynamics over raw environment reconstruction.
    Found Nov 3
  • ArXiv 2026

    Reinforcement World Model Learning for LLM-based Agents

    Xiao Yu; Baolin Peng; Ruize Xu; Yelong Shen; Pengcheng He; Suman Nath; Nikhil Singh; Jiangfeng Gao; Zhou Yu

    95 Match
    TOPIC
    This paper investigates how to improve agentic intelligence in long-horizon environments by training LLMs to function as internal world models.
    NOVELTY
    The authors introduce Reinforcement World Model Learning (RWML), which uses GRPO to optimize for semantic equivalence of predicted next states rather than exact token matching.
    INSIGHT
    They show that this self-supervised mid-training approach allows the agent to better anticipate environment dynamics without needing expert trajectories or sparse rewards.
    RELEVANCE
    Since it directly addresses multi-step planning via a learned world model that reduces invalid actions, it is a direct match for your work on agentic planning.
    Found Nov 3
  • ArXiv 2026

    Beyond One-Step Accuracy: State-Affine Latent Transition for Reliable Visual Planning

    Boyuan Zhang; Yingjun Du; Xiantong Zhen; Ling Shao

    95 Match
    TOPIC
    This paper investigates visual planning using joint-embedding world models to avoid the overhead of pixel-level reconstruction.
    NOVELTY
    They introduce SALT, a transition model with a state-affine architecture that ensures the Jacobian is independent of the latent state.
    INSIGHT
    The key takeaway is that restricting dynamics to be affine prevents the compounding of errors during multi-step rollouts, even if one-step prediction accuracy is lower.
    RELEVANCE
    Since you are working on multi-step agentic planning with internal world models, this approach to stabilizing recursive latent predictions is directly applicable to your work.
    Found Oct 30
  • EMNLP 2026

    World-Model Evolving Agents

    Leonardo Ranaldi

    95 Match
    TOPIC
    This paper presents WEVA, a framework for autonomous agents that uses a predictive world model to handle long-horizon interactive planning.
    NOVELTY
    It introduces a modular architecture that maps interactions to quasi-symbolic states, allowing the agent to perform verified lookahead by rolling out and pruning action sequences based on predicted state transitions.
    INSIGHT
    The key takeaway is that using structured abstractions instead of free-text predictions enables the agent to explicitly detect and reject invalid action sequences before execution.
    RELEVANCE
    This is directly applicable to your work as it implements a concrete mechanism for multi-step agentic planning grounded in an evolving internal world model.
    Found Oct 28
  • EMNLP 2026

    MemWM: Memory-Augmented Text-Based World Model

    Yujun Wang; Tao Zhang; Jinhe Bi; Aniri; Wenxuan Ye; Boliang Liu; Sikuan Yan; Shuning Wang; Xuebing Zhou; Soren Pirk; Hinrich Schuetze; Yunpu Ma

    95 Match
    TOPIC
    This paper tackles the state-fidelity bottleneck in text-based world models used for sequential decision-making agents.
    NOVELTY
    It introduces MemWM, a memory-augmented architecture that uses a dual-channel system of world memory and world skills to condition next-state imagination on retrieved transition rules and state caches.
    INSIGHT
    The key takeaway is that augmenting the world model with explicit memory of environment dynamics significantly reduces hallucinations of behavior-critical facts during lookahead planning.
    RELEVANCE
    Since you are working on multi-step agentic planning with internal world models, this approach to improving state prediction fidelity without fine-tuning the policy is directly applicable to your work.
    Found Oct 28
  • EMNLP 2026

    Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models

    Youwei Liu; Jian Wang; Hanlin Wang; Beichen Guo; Wenjie Li

    95 Match
    TOPIC
    This paper investigates the problem of shallow grounding in LLM agents by implementing a textual world model for long-horizon task planning.
    NOVELTY
    They introduce the Imagine-then-Plan (ITP) framework, which uses a POIMDP formulation to generate imagined trajectories in a mental sandbox before executing actions.
    INSIGHT
    A key takeaway is the discovery of a bell curve relationship between lookahead horizon and success rate, meaning the agent must dynamically balance foresight against compounding world-model errors.
    RELEVANCE
    This is directly applicable to your work as it provides a formal method for multi-step agentic planning using an internal world model with an adaptive lookahead mechanism.
    Found Oct 28
  • EMNLP 2026

    Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning

    Xuan Zhang; Zhijian Zhou; Lingfeng Qiao; Yulei Qin; Ke Li; Xing Sun; Xiaoyu Tan; Chao Qu; Yuan Qi

    95 Match
    TOPIC
    This paper investigates how to move LLM agents from reactive policies to foresight-conditioned planning by internalizing a world model within a single autoregressive model.
    NOVELTY
    They introduce a three-stage training pipeline that uses mid-training, SFT, and GRPO-based RL to enable the agent to verbalize prospective state rollouts and calibrated success estimates (Q-values) before taking actions.
    INSIGHT
    The key takeaway is that predicting future states alone is insufficient; the agent needs calibrated confidence scores to effectively prune paths and guide multi-step planning.
    RELEVANCE
    This is directly applicable to your work as it provides a concrete method for implementing internal world models for long-horizon agentic planning without needing external simulators.
    Found Oct 28
  • ICLR 2025

    Empowering LLM Agents with Zero-Shot Optimal Decision-Making through Q-learning

    Jiajun Chai; Sicheng Li; Yuqian Fu; Dongbin Zhao; Yuanheng Zhu

    95 Match
    TOPIC
    This paper investigates optimal decision-making for LLM agents in long-horizon MDP tasks using a model-based reinforcement learning approach.
    NOVELTY
    It introduces MLAQ, which combines a Q-Planner with an LLM-based imagination space that acts as a world model to predict next states and actions.
    INSIGHT
    The key takeaway is that using an internal world model to simulate transitions and a UCB-guided planner allows the agent to derive optimal policies with minimal environmental interaction.
    RELEVANCE
    This is directly relevant to your work as it implements a concrete architecture for multi-step agentic planning driven by an internal world model.
    Found Oct 25
  • EMNLP 2026

    CoMAP: Co-Evolving World Models and Agent Policies for LLM Agents

    Youwei Liu; Jian Wang; Hanlin Wang; Wenjie Li

    95 Match
    TOPIC
    This paper investigates the co-evolution of LLM agent policies and textual world models for long-horizon planning in interactive environments.
    NOVELTY
    It introduces COMAP, a framework that uses a closed-loop system where the world model is updated via on-policy self-distillation while the agent performs future-aware reflection to refine actions.
    INSIGHT
    The key takeaway is that static world models fail as agent policies evolve, and dynamically aligning the world model with the agent's current state-action distribution is critical for maintaining planning fidelity.
    RELEVANCE
    This is directly applicable to your work on multi-step agentic planning, as it provides a concrete mechanism for integrating an evolving internal world model into the decision-making loop.
    Found Oct 25
  • ArXiv 2026

    Steering Black-Box Large Language Model Agents with Pitfall Anticipation World Model

    Jiongxiao Wang; Han Ding; Youzhi Luo; Yawei Wang; Yun Zhou; Linbo Liu; Panpan Xu

    95 Match
    TOPIC
    This paper investigates steering black-box LLM agents in long-horizon environments to avoid execution failures through predictive state-space modeling.
    NOVELTY
    It introduces the Pitfall Anticipation World Model (PAWM), an external cognitive layer that simulates future trajectories to identify and bypass critical failure points before they occur.
    INSIGHT
    The core takeaway is that explicitly modeling the "negative space" of outcomes provides a more effective steering mechanism for multi-step planning than standard self-reflection or CoT.
    RELEVANCE
    Since you are focused on agentic planning with internal world models, this approach to decoupling execution from anticipation via a predictive world model is directly applicable to your work.
    Found Oct 23
  • ArXiv 2026

    Self-Improving Agents with World Model Predictive Paths

    Haori Lu; Tianyu Gao; Yaoyao Liu; Yingying Li

    95 Match
    TOPIC
    This paper investigates decision-making for LLM agents in long-horizon text environments using a predict-select-replan loop.
    NOVELTY
    It introduces an MPPI-inspired framework that uses a language-based world model to imagine multiple candidate outcomes and weights them via a Gibbs distribution to select the optimal action.
    INSIGHT
    The key takeaway is that separating the world model's predictive accuracy from the planner's action proposal through a dual-branch memory significantly reduces per-step regret.
    RELEVANCE
    This is directly applicable to your work as it implements a concrete architecture for multi-step agentic planning driven by an internal world model.
    Found Oct 23
  • TOPIC
    This paper investigates reliability in multi-step planning for language agents operating in graph-structured environments.
    NOVELTY
    It introduces Hybrid-WM, a framework that pairs a semantic LLM planner with a lightweight parametric world model to detect and correct hallucinated state transitions.
    INSIGHT
    The key takeaway is that using a parametric model as a grounding mechanism to check consistency against the LLM's imagined states prevents error propagation over long horizons.
    RELEVANCE
    This is directly applicable to your work on agentic planning with internal world models, specifically regarding how to mitigate the drift between an agent's internal beliefs and actual environment dynamics.
    Found Oct 23
  • ArXiv 2026

    World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems

    Runze Li; Hongyin Zhang; Junxi Jin; Qixin Zeng; Zifeng Zhuang; Yiqi Tang; Shangke Lyu; Donglin Wang

    95 Match
    TOPIC
    The paper investigates long-horizon decision-making for embodied agents using a Vision-Language-Action (VLA) framework.
    NOVELTY
    It introduces the World-Value-Action (WAV) model, which replaces direct action prediction with iterative latent-space trajectory optimization using a Diffusion Transformer.
    INSIGHT
    The core takeaway is that planning in a learned latent manifold avoids the "Curse of Feasibility" associated with high-dimensional action spaces, allowing for more robust multi-step reasoning.
    RELEVANCE
    This is directly applicable to your work as it implements a concrete internal world model via video generation coupled with a value function to drive agentic planning.
    Found Oct 14
  • ArXiv 2026

    LeWAM: End-to-End World and Action Modeling with JEPAs

    Minghao Fu; Tavis Siebert; Eryk Halicki; Randall Balestriero

    95 Match
    TOPIC
    This paper investigates goal-conditioned visual planning and robotic manipulation using a Joint-Embedding Predictive Architecture (JEPA).
    NOVELTY
    It introduces LeWAM, which jointly trains a latent world model and a stochastic policy via flow matching to enable both direct execution and gradient-based planning.
    INSIGHT
    The key takeaway is that using a learned action distribution to seed candidates, followed by gradient refinement through the latent dynamics, drastically reduces planning latency compared to CEM while improving success in contact-rich tasks.
    RELEVANCE
    This is directly applicable to your work as it implements a multi-step agentic planning loop powered by an internal world model that avoids the pitfalls of pixel-reconstruction.
    Found Oct 12
  • 95 Match
    TOPIC
    This paper investigates long-horizon goal-conditioned planning in robotic manipulation using latent world models.
    NOVELTY
    It introduces SAGE, a framework that couples a multi-horizon latent subgoal generator with a subgoal-conditioned action prior to guide the search process.
    INSIGHT
    The key takeaway is that decomposing a distant goal into intermediate latent subgoals prevents the proposal distribution from collapsing in high-dimensional action spaces.
    RELEVANCE
    Since you are focused on multi-step agentic planning with internal world models, this approach to structured latent subgoal decomposition is directly applicable to your work.
    Found Oct 11
  • ACL 2026

    Model-Based Imaginative Planning for Embodied Agents

    Junru Song; Hengzhe Jin; Yucong Huang; Tingsong Jiang; Weien Zhou; Feifei Wang; Yang Yang; Ying Wen; Wen Yao

    95 Match
    TOPIC
    This paper investigates long-horizon embodied sequential decision-making in POMDPs using a model-based reasoning framework called IMPLEMENT.
    NOVELTY
    It introduces a symbolic world model that decouples perception from prediction, allowing a frozen LLM to perform 'imagine-evaluate-refine' loops via Monte Carlo rollouts.
    INSIGHT
    The use of Meta In-Context Learning to adapt the world model to environment-specific dynamics online provides a scalable way to handle epistemic uncertainty without parameter updates.
    RELEVANCE
    This is directly applicable to your work as it implements a concrete architecture for multi-step agentic planning driven by an internal world model.
    Found Oct 10
  • ArXiv 2026

    Reinforced Planning with Latent World Models

    Armin Sommer; Jannik Schilling

    95 Match
    TOPIC
    This paper explores model-based planning in goal-conditioned MDPs using a pretrained latent world model for visual-control tasks.
    NOVELTY
    It introduces RP1, which replaces hand-designed search algorithms with a learned neural planner that iteratively refines action sequences based on value gradients from a critic.
    INSIGHT
    The core takeaway is that learning a plan-improvement operator allows the agent to outperform traditional optimizers like CEM while reducing world-model rollouts by three orders of magnitude.
    RELEVANCE
    This is directly applicable to your work as it implements a multi-step agentic planning loop that leverages an internal world model to optimize trajectories.
    Found Oct 8
  • Loading more results…