LLM Agent Survey: Methodology, Applications, and Challenges in 2025

Understanding the LLM Agent Revolution

The emergence of LLM agents marks a pivotal transformation in artificial intelligence. Unlike traditional AI systems that passively respond to user inputs, large language model agents actively perceive their environments, reason about complex goals, and execute multi-step actions autonomously. This comprehensive survey by researchers from Peking University, University of Illinois at Chicago, Nanyang Technological University, and other leading institutions provides the most systematic examination of LLM agent methodology published to date.

The survey introduces a novel Build-Collaborate-Evolve framework that unifies fragmented research across agent construction, multi-agent collaboration, and evolutionary learning. Commercial systems like DeepResearch, DeepSearch, and Manus have already demonstrated the practical viability of autonomous agents that perform complex tasks previously requiring human expertise — from in-depth research to computer operation. This represents not merely a technological advancement but a fundamental reimagining of human-machine relationships.

As explored in our analysis of large language model capabilities and limitations, the convergence of unprecedented reasoning capabilities, advanced tool manipulation, and sophisticated memory architectures has transformed theoretical constructs into production-ready systems. The boundary between AI assistants and genuine collaborators is rapidly dissolving.

LLM Agent Architecture: The Four Pillars of Construction

The survey identifies four interdependent pillars that form the foundation of every LLM agent system. Understanding these components is essential for anyone building, evaluating, or deploying agent technology in 2025 and beyond.

Profile Definition establishes an agent’s operational identity through two approaches. Human-curated static profiles ensure domain-specific consistency through manual specification — systems like MetaGPT and ChatDev coordinate predefined technical roles such as product managers and programmers. Batch-generated dynamic profiles employ parameterized initialization to create diverse agent populations that simulate realistic social dynamics, enabling applications from behavior studies to emergent group intelligence.

Memory Mechanisms equip agents with temporal information management across three paradigms: short-term memory for transient context (dialog histories and environmental feedback), long-term memory for persistent knowledge (skill libraries, experience repositories, and tool synthesis frameworks), and knowledge retrieval as memory through RAG techniques that expand accessible information boundaries. Systems like Voyager’s skill discovery in Minecraft, Reflexion’s trial-optimized memory, and MemGPT’s tiered architecture demonstrate the power of structured long-term storage.

Planning Capability enables agents to decompose complex tasks into manageable sub-tasks through strategies like Plan-and-Solve prompting, Tree of Thoughts reasoning, and Monte Carlo Tree Search. Feedback-driven iteration allows agents to refine plans based on execution outcomes, creating adaptive planning loops.

Action Execution bridges the gap between reasoning and real-world impact through tool utilization frameworks (GPT4Tools, EASYTOOL) and physical interaction capabilities for embodied agents operating in simulated or real environments.

Multi-Agent Collaboration Frameworks and Paradigms

One of the survey’s most significant contributions is its systematic analysis of how multiple LLM agents collaborate to solve problems beyond the reach of individual systems. The research identifies three fundamental collaboration architectures that are reshaping how we design intelligent systems.

Centralized Control architectures employ a coordinator agent that orchestrates the activities of specialized sub-agents. Systems like MetaGPT use structured role orchestration where a central planner assigns tasks, monitors progress, and synthesizes outputs. This approach excels in scenarios requiring strict workflow management and quality control — similar to how a project manager coordinates a software development team.

Decentralized Collaboration enables peer-to-peer interaction among autonomous agents without hierarchical oversight. The CAMEL framework pioneered role-playing communication between agents, while approaches like Multi-Agent Debate (MAD) leverage diverse perspectives to improve reasoning quality. MedAgents demonstrates how decentralized collaboration among specialist agents can enhance medical diagnosis accuracy.

Hybrid Architectures combine centralized coordination with decentralized flexibility. Systems like KnowAgent integrate external knowledge bases with agent reasoning, while TextGrad enables gradient-based optimization across multi-agent workflows. These hybrid approaches are increasingly favored in production deployments where both reliability and adaptability are critical.

Our analysis of agent skills for large language models provides additional context on how individual agent capabilities map to collaborative system design.

Agent Evolution: Self-Learning and Continuous Improvement

Perhaps the most forward-looking dimension of the survey examines how LLM agents evolve over time through three mechanisms that mirror biological and social evolution processes.

Autonomous Self-Learning allows individual agents to improve through iterative refinement. The SELF-REFINE framework enables agents to critique and improve their own outputs without external supervision. STaR (Self-Taught Reasoner) and its successor V-STaR demonstrate how agents can bootstrap reasoning capabilities through self-generated training data. Self-Rewarding Language Models eliminate the need for human preference labels by having the model judge its own outputs.

Multi-Agent Co-Evolution enables populations of agents to improve collectively through competitive or cooperative dynamics. Red-Team approaches pit adversarial agents against defensive ones, driving mutual improvement in safety and robustness. ProAgent explores how agents can proactively infer teammates’ intentions and adapt their strategies in cooperative games, demonstrating emergent collaborative intelligence.

Evolution via External Resources leverages outside knowledge to drive agent improvement. CRITIC enables agents to verify and correct their outputs using external tools like search engines and code interpreters. KnowAgent and WKM integrate structured world knowledge to enhance planning and reasoning capabilities, demonstrating how external knowledge augmentation accelerates agent development.

LLM Agent Evaluation: Benchmarks and Assessment Methods

The survey provides a comprehensive overview of evaluation methodologies, highlighting the gap between existing benchmarks and the complexity of real-world agent deployment. Current evaluation approaches span three categories.

General Assessment benchmarks test fundamental agent capabilities across diverse tasks. These include reasoning benchmarks, code generation challenges, and general-purpose task completion metrics. However, the survey notes that many benchmarks fail to capture the emergent behaviors that arise when agents operate in open-ended environments over extended periods.

Domain-Specific Evaluation targets specialized agent applications in fields like software engineering (SWE-bench), scientific research, and medical diagnosis. These benchmarks provide more realistic assessment but often remain narrow in scope.

Collaboration Evaluation measures how effectively multiple agents work together, assessing metrics like task completion rates in multi-agent scenarios, communication efficiency, and the quality of negotiated solutions. This emerging evaluation dimension is critical as multi-agent systems become more prevalent in enterprise applications.

The researchers emphasize that robust evaluation must capture both individual agent competence and systemic properties like scalability, fault tolerance, and emergent behavior — areas where current benchmarks remain insufficient.

Tools and Development Ecosystems for LLM Agents

The relationship between LLM agents and tools operates bidirectionally, creating a powerful development ecosystem. The survey categorizes this relationship into three interaction patterns that define modern agent development.

LLM Use Tools — agents leverage existing tools like search engines, code interpreters, calculators, and APIs to extend their capabilities beyond text generation. Frameworks like ToolFormer and Gorilla have demonstrated how agents can autonomously select and invoke appropriate tools for specific tasks, dramatically expanding the action space available to language models.

LLM Create Tools — agents generate new tools by writing code, creating APIs, or composing existing tools into novel combinations. Voyager’s automatic skill discovery exemplifies this pattern, where the agent creates reusable JavaScript functions that serve as persistent tools for future tasks. This capability represents a qualitative leap in agent autonomy.

Tools Develop LLM — specialized tools and frameworks facilitate the training, fine-tuning, and deployment of language models that power agents. This includes synthetic data generation pipelines, automated evaluation harnesses, and reinforcement learning environments designed specifically for agent training.

As highlighted in our exploration of the Accenture Technology Vision 2025, the maturation of agent tooling ecosystems is a critical enabler for enterprise adoption of autonomous AI systems.

Security, Privacy, and Ethical Challenges in Agent Deployment

The survey dedicates significant attention to the security challenges of LLM agents, recognizing that as agents gain more autonomy and tool access, the potential for harm scales proportionally. The analysis covers three critical dimensions.

Agent-Centric Security addresses threats directed at the agent itself, including prompt injection attacks, jailbreaking techniques, and adversarial inputs designed to manipulate agent behavior. The survey documents how attackers can exploit the reasoning chain of thought to inject malicious instructions, potentially causing agents to execute unauthorized actions, leak sensitive data, or produce harmful outputs.

Data-Centric Security encompasses vulnerabilities in the training data and knowledge bases that agents rely upon. Training data poisoning can introduce persistent biases or backdoors that activate under specific conditions, while contaminated retrieval sources can lead agents to incorporate and propagate misinformation.

Privacy Risks emerge from agents’ ability to memorize and potentially regurgitate sensitive training data, including personal information, proprietary code, or confidential business data. The survey highlights intellectual property exploitation concerns, where agents may inadvertently reproduce copyrighted content or proprietary algorithms, raising significant legal and ethical questions.

For financial institutions navigating these challenges, our analysis of banking risk management in 2025 provides practical frameworks for deploying AI agents while managing operational risk.

Real-World Applications Across Industries

The survey maps an impressive breadth of LLM agent applications across diverse industries, demonstrating the technology’s versatility and transformative potential in 2025.

Healthcare and Medical Research — agents are assisting in medical diagnosis through multi-specialist collaboration (MedAgents), drug discovery through automated hypothesis generation, and clinical trial optimization through patient matching and protocol design. The ability to reason across vast medical literature while maintaining patient safety protocols makes LLM agents particularly valuable in this domain.

Scientific Research — from chemistry (Coscientist for autonomous experiment design) to materials science and astronomy, LLM agents are accelerating the scientific method by automating literature review, hypothesis generation, experimental design, and data analysis. These agents can process and synthesize information from thousands of papers simultaneously, identifying patterns human researchers might miss.

Software Development — systems like ChatDev and MetaGPT demonstrate how multi-agent teams can autonomously handle the complete software development lifecycle, from requirements analysis through coding, testing, and deployment. These systems achieve this through role-based agent coordination that mirrors real development team structures.

Social Science and Simulation — generative agents capable of simulating human behavior enable researchers to study social dynamics, test policy interventions, and model economic systems at unprecedented scale, opening new avenues for social science research as noted by Stanford’s generative agents research.

Financial Services — LLM agents are being deployed for market analysis, risk assessment, regulatory compliance monitoring, and customer service automation. The ability to process and reason over complex financial documents, regulations, and market data makes these agents particularly impactful in finance, as we explore in our financial services regulatory outlook for 2026.

Key Research Findings and Data Points

The survey synthesizes several critical findings that shape the trajectory of LLM agent research heading into the second half of the decade.

First, the convergence of three capabilities — reasoning, tool use, and memory — has created a qualitative leap in agent sophistication. No single capability alone produces the emergent behaviors observed in modern systems; it is their integration that enables truly autonomous operation. The survey documents how systems combining all three dimensions consistently outperform those missing any single component.

Second, multi-agent systems demonstrate emergent capabilities that exceed the sum of their parts. Through debate, negotiation, and role specialization, agent collectives achieve reasoning quality and task completion rates significantly higher than individual agents. This finding has profound implications for system design — investing in collaboration infrastructure may yield greater returns than improving individual agent capability.

Third, the evolution mechanisms catalogued in the survey suggest that agent systems are approaching a form of continuous self-improvement. Self-rewarding models, co-evolutionary dynamics, and external resource integration create positive feedback loops that accelerate capability development. However, this same dynamic raises important safety concerns about uncontrolled capability growth.

The researchers identify several open challenges: grounding agents in physical reality, achieving robust long-horizon planning, ensuring alignment with human values during autonomous operation, and developing evaluation frameworks that capture real-world complexity. These challenges represent the frontier of LLM agent development and the focus areas for the next generation of research as documented by the original survey paper.

Implications for Enterprise AI Strategy

For organizations evaluating LLM agent technology, this survey provides essential strategic guidance. The Build-Collaborate-Evolve framework offers a practical lens for assessing agent maturity and planning deployment roadmaps.

Organizations should prioritize agent construction fundamentals before pursuing complex multi-agent deployments. Establishing robust memory architectures, reliable tool integration, and clear agent profiles creates the foundation upon which collaborative and evolutionary capabilities can be built. Premature deployment of multi-agent systems without solid individual agent foundations leads to compounding errors and unpredictable behavior.

The survey’s analysis of collaboration architectures suggests that hybrid approaches — combining centralized oversight with decentralized agent autonomy — offer the best balance of reliability and flexibility for enterprise use cases. This mirrors the organizational patterns that have proven successful in human teams and suggests that agent system design can benefit from decades of management science research.

Security considerations should be addressed from the architectural level, not bolted on after deployment. The survey’s documentation of attack vectors — from prompt injection to training data poisoning — provides a comprehensive threat model that security teams should integrate into their agent deployment planning. As highlighted in our analysis of AI alignment taxonomy, ensuring agents remain aligned with organizational objectives requires proactive design choices, not reactive patches.

Finally, organizations should invest in evaluation infrastructure that goes beyond simple benchmarks. The survey emphasizes that real-world agent performance depends on emergent properties like fault tolerance, graceful degradation, and adaptive behavior under novel conditions — qualities that standard benchmarks frequently fail to measure.