AgentFounder, a groundbreaking AI agent developed by Alibaba's Tongyi Lab, introduces Agentic Continual Pre-training (Agentic CPT) to revolutionize AI agent development. This novel three-stage approach incorporates an intermediate training phase focused on agentic behaviors, addressing limitations in current two-stage models. Using innovative data synthesis methods like First-order Action Synthesis and Higher-order Action Synthesis, AgentFounder-30B achieves state-of-the-art performance across multiple benchmarks, surpassing existing open-source models and some proprietary systems. The research demonstrates logarithmic scaling patterns for agentic capabilities and improved fine-tuning efficiency. This work paves the way for more capable, general-purpose AI agents that can autonomously navigate complex information environments and solve multi-step problems.
The groundbreaking research paper “Scaling Agents via Continual Pre-training” introduces a transformative approach to developing more capable AI agents by fundamentally redesigning how these systems learn to use tools and perform complex tasks. Published by researchers from Alibaba’s Tongyi Lab in September 2025, this paper presents AgentFounder, a deep research agent that achieves state-of-the-art performance across multiple benchmarks by implementing a novel training stage called Agentic Continual Pre-training (Agentic CPT).
The research demonstrates that current approaches to building AI agents are fundamentally limited because they attempt to teach general-purpose language models complex agentic behaviors during post-training, creating an inherent tension where models must simultaneously learn capabilities and alignment. By introducing an intermediate training phase focused specifically on developing agentic behaviors before fine-tuning, the researchers have achieved remarkable improvements in agent performance, particularly for deep research capabilities that involve web search, browsing, and multi-step reasoning.
At its core, the paper challenges the traditional two-stage training paradigm for large language models, which typically consists of pre-training on diverse text data followed by post-training that includes Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Instead, the authors introduce a three-stage approach that incorporates Agentic CPT between pre-training and post-training. This additional phase enables the model to develop foundational agentic capabilities like tool invocation patterns and multi-step reasoning chains before being fine-tuned for specific tasks.
The researchers identify a critical bottleneck in existing approaches: even leading agentic models like WebSailor (12.0), GLM-4.5 (26.4), and DeepSeek-V3.1 (30.0) exhibit substantial performance gaps compared to proprietary systems like OpenAI’s Deep Research (51.5) on challenging benchmarks. This performance disparity stems from the limitations of post-training methods when applied to foundation models that lack inherent agentic inductive biases.
To address this challenge, AgentFounder employs two innovative data synthesis methods. First-order Action Synthesis (FAS) transforms static knowledge into diverse training contexts that simulate real-world agent scenarios, using an entity-anchored open-world knowledge memory approach. This method generates planning and reasoning actions without requiring expensive API calls, making it highly scalable. Higher-order Action Synthesis (HAS) remodels trajectories as multi-step decision-making processes, expanding potential exploration paths at each step to enhance the model’s exploration and decision-making capabilities.
The training process follows a two-stage strategy: the first stage primarily uses FAS data and short HAS data within a 32K context window, while the second stage focuses on high-quality HAS data with an extended 128K context length. This progressive approach allows the model to efficiently absorb different types of agentic data.
Evaluation results are impressive, with AgentFounder-30B achieving 39.9% on BrowseComp-en, 43.3% on BrowseComp-zh, 72.8% on GAIA, 31.5% on HLE, and 73.0% on xbench-DeepSearch. These scores represent significant improvements over existing open-source models and even surpass some proprietary systems. Notably, AgentFounder-30B becomes the first open-source model to exceed the 30-point threshold on the challenging Humanity’s Last Exam (HLE) benchmark.
The research also explores scaling laws for agentic capabilities, confirming that both model size and training data volume follow logarithmic scaling patterns. Performance improvements continue steadily as training data increases from 0B to 315B tokens, with the most substantial gains occurring within the initial 15B tokens. This finding suggests that agentic behaviors can be efficiently acquired through targeted pre-training.
Moreover, the authors demonstrate that Agentic CPT significantly enhances fine-tuning efficiency. Models that undergo Agentic CPT before post-training show consistently lower loss values during supervised fine-tuning compared to baseline models, indicating that pre-aligned agentic foundation models adapt more effectively to downstream agent tasks.
Perhaps most promising is the finding that AgentFounder maintains strong general tool-use capabilities alongside its specialized deep research skills. This suggests the potential for developing general-purpose agents that can perform well across a wide range of tasks, rather than being limited to specific domains.
The implications of this research extend beyond the specific model presented. By redefining the training pipeline for agentic alignment, the authors have opened a new path toward developing more capable AI systems that can autonomously navigate complex information environments, reason through multi-step problems, and provide more reliable answers. As AI continues to evolve toward more agentic systems capable of autonomous problem-solving, this work represents a significant milestone in aligning these powerful tools with human intent and expectations.
The paper stands as an important contribution to the field of AI research, demonstrating how continual pre-training can bridge the gap between foundation models and specialized agents, potentially accelerating progress toward more general and capable AI assistants that can help humans navigate increasingly complex information landscapes. With Tongyi Lab’s commitment to open-source development, these advancements may soon become accessible to researchers and developers worldwide, fostering further innovation in agentic AI systems.
