AI Agent Tool Use: MCP, CLI, Skills, and Code Execution

Viacheslav Dubrov

2026-03-24 · Updated 2026-09-06 · 22 min · AI Agents

Article update

Originally published on 24 March 2026. Reviewed and updated on 6 September 2026. The update covers the revised MCP specification, programmatic tool calling, and newer evidence on tool-use costs and limitations.

An agent needs a way to act: a JSON tool call, an MCP service, a CLI command, or code in a sandbox. It may also need instructions for choosing and using that mechanism. Skills provide those instructions. The harness is the ordinary program around the model: it builds prompts, checks a proposed call, runs an approved call, and decides when the task is done.

This third article in the series adds the action layer to Part 1’s reasoning loops and Part 2’s memory. Part 4 examines the policy check before execution, and Part 6 examines the harness that runs both the call and that check.

The tooling story changed in 2025–2026. MCP, the Model Context Protocol, gave vendors one shared way to expose external services. Code-executing agents showed that a model can sometimes compose a small program more efficiently than it can issue a long sequence of JSON calls. Anthropic reported a 98.7% token reduction for one Google Drive-to-Salesforce workflow, and the CodeAct paper reported task-success gains of up to 20 percentage points across its benchmark setup. Those results describe their tasks and harnesses, not a universal advantage for code execution.

I compare JSON tool calling, MCP, CLI tools, and code execution, then show where Skills fit across them. A later section applies Agent-Computer Interface (ACI) design principles to the Market Analyst Agent, a small LangGraph research agent I built for Part 1 that fetches market data and writes an analyst report.

For the short interface decision, see AI Agent Tool Interfaces.

Execution surfaces and procedural guidance

The reasoning loop proposes a call. The harness checks its arguments and whether the call is allowed, then sends it to a tool or sandbox and returns the result. JSON schemas, MCP transport, CLI wrappers, and code runners can constrain inputs, but none decides whether the requested action is allowed. Skills supply instructions for that path. These execution surfaces trade token cost, flexibility, and enforcement differently.

1. JSON tool calling: the baseline

The original pattern: you define tool schemas as JSON, the LLM emits structured function calls, your code executes them. It is well-understood and works fine for small toolsets.

# Schema cost depends on its text, structure, and the model tokenizer
tools = [
    {
        "name": "get_stock_price",
        "description": "Get the current stock price for a ticker symbol",
        "input_schema": {
            "type": "object",
            "properties": {
                "ticker": {"type": "string", "description": "Stock ticker (e.g., NVDA)"}
            },
            "required": ["ticker"]
        }
    }
]

Count the tokens in your actual schemas. Cost depends on their length and how many the host loads; a compact price lookup and a deeply nested API contract are not equal units. Deferred discovery can avoid loading the whole registry.

2. MCP for shared integrations

MCP is the standard most vendors converged on. An MCP server is a process that advertises a list of tools over a defined wire protocol — stdio for a local process, HTTP for a remote one. Your agent runs an MCP client that connects, asks the server what tools it has, and forwards the model’s calls to it, so the same server works with any client that speaks the protocol. Anthropic donated the protocol to the Linux Foundation in December 2025, under the Agentic AI Foundation it co-founded with OpenAI and Block. Google, Microsoft, and AWS back the foundation as platinum members. OpenAI added MCP support in its Responses API. As of Anthropic’s December 2025 donation announcement, the ecosystem counted 10,000+ active public MCP servers and 97M+ monthly SDK downloads across the Python and TypeScript SDKs.

MCP fits cross-vendor SaaS integration (Figma, Notion, Salesforce), services without CLI equivalents, and environments that need OAuth orchestration. Its value is a shared discovery and transport layer. Governance still depends on the server’s authentication, authorization, logging, and deployment controls.

Protocol version is now a practical migration decision. The 2026-07-28 revision changes behavior that older tutorials assume:

The same revision deprecates Roots, Sampling, Logging, and OAuth Dynamic Client Registration; deprecation is not immediate removal. Current client registration favors Client ID Metadata Documents. Existing integrations may still use an older revision, so inspect the installed SDK and the server contract before adopting a new feature.

The production story is messier than the headline numbers suggest.

The Vulnerable MCP Project collects reports involving prompt injection, input validation, authentication, and network controls. Such a collection helps identify test cases; without an exposure denominator it cannot rank MCP against shell or direct API calls.

Tool poisoning is the attack class I worry about most. Invariant Labs demonstrated that poisoned MCP tools can exfiltrate data even when they are never invoked. The model just reading the tool’s metadata is enough to trigger the attack. MCPTox benchmarks testing 20 LLM agents against 45 real-world MCP servers reported 72.8% average attack success for o1-mini under its tool-poisoning setup. That is one model’s benchmark result, not an average across the 20 agents or a real-world incident rate.

Token overhead is the operational problem. One team running MCP servers for GitHub, Slack, and Sentry (~40 tools total) found 55,000 tokens of schema definitions injected before a user asks anything. Another reported 143,000 of 200,000 available tokens (72%) consumed by tool definitions alone.

Anthropic’s Tool Search Tool report reported approximate context endpoints of 77,000 tokens before work begins and 8,700 after deferred discovery with about 72,000 tokens of tool definitions in the traditional setup. It loads only the three to five tools a request needs, but adds a discovery step before invocation; it is less useful for small, compact toolsets whose tools are used frequently in every session.

3. Skills package expertise, not execution

Agent skills are an open format for packaging instructions and supporting files. Tools provide capabilities (what agents can do), and skills provide expertise (what agents know about how to accomplish complex tasks).

The SKILL.md format defines a skill as a markdown file with YAML frontmatter. The open standard requires only name and description; the example below also uses two Claude Code extensions, argument-hint and user-invocable, plus its $0 positional-argument placeholder:

---
name: deploy
description: Deploy the application to production
argument-hint: "[environment]"
user-invocable: true
---
Deploy the application to the $0 environment (default: staging).
Steps:
1. Run the test suite
2. Build the production bundle
3. Deploy using the deploy script
4. Verify the deployment health check

Skills use progressive disclosure. At startup, the agent gets about 100 tokens of name and description. It loads the full SKILL.md only when it needs the skill, then loads referenced scripts, documents, or assets as needed. That startup cost is much smaller than the roughly 55,000 tokens that about 40 MCP tools can consume before reasoning begins. An active skill still adds its instructions and resources to context.

Use skills for domain knowledge, multi-step procedures, and recurring work such as database migrations or payment integrations. They fit tasks where the agent needs instructions for how to use an existing capability.

4. CLI and shell tools

CLI interfaces can be much cheaper in context when the model already knows the command. Scalekit reported a 4-32x token difference between its CLI and MCP paths across 75 runs. That case study measures its tools and tasks; it does not replace a comparison on your own tool definitions and command output.

Widely documented commands such as git, docker, kubectl, gh, curl, and jq often need little introductory schema text. Less common or internal CLIs still need discoverable help, examples, and stable machine-readable output.

Ugo Enyioha’s guide “Writing CLI Tools That AI Agents Actually Want to Use” codified eight design rules:

1. Structured output is mandatory — support --json
2. Exit codes are control flow — use distinct codes for different error types
3. Commands should be idempotent
4. Self-documenting --help with realistic examples
5. Design for composability — --quiet for bare values, stdin support
6. Provide --dry-run and --yes flags
7. Support version introspection
8. Handle auth via environment variables

CLI has no protocol-level discovery. JSON tool calling can carry typed schemas, while MCP standardizes tool discovery and, for HTTP transports, an authorization model. Neither one supplies governance by itself: the host, server, or harness must enforce policy and record the calls it needs to audit. A practical default is CLI for development and local operations, and MCP for shared external-service integration when cross-client discovery or OAuth orchestration is worth the server overhead.

5. Code execution for multi-step work

This is the change in agent tooling I find most consequential. Instead of emitting structured JSON to invoke predefined functions one at a time, the agent writes a Python or bash script. The script calls multiple tools, processes results with loops and conditionals, and returns only the final summary to the model context.

Anthropic introduced Programmatic Tool Calling (PTC) in beta. The current API guide uses the regular Messages API with code_execution_20260120 or later; the original beta launch is historical context. The academic foundation is the CodeAct paper (Wang et al., ICML 2024), which tested across 17 LLMs and found code actions achieved up to 20 percentage points higher task success and 30% fewer actions than JSON alternatives.

Three first-party case studies show where the pattern can help: Vercel and Cloudflare below, then Anthropic’s expense-analysis example. Treat them as vendor evidence and re-run the comparison on your own tasks.

• Vercel rebuilt d0, its natural-language-to-SQL data agent. Its old code example names 17 tools; its new code example exposes ExecuteCommand and ExecuteSQL. Vercel frames the redesign as removing 80% of its tools, but that statement is Vercel’s headline, not a percentage that follows from the examples’ named tools. Across five representative queries, Vercel reports task success went from 4/5 to 5/5, average execution time dropped 3.5x (274.8 s to 77.4 s), and average token use fell 37% (~102k to ~61k). Their phrasing: “The best agents might be the ones with the fewest tools.”

• Cloudflare developed “Code Mode,” letting agents write TypeScript to call their API rather than defining tool schemas, which reduces context overhead. Their reasoning: “LLMs have an enormous amount of real-world TypeScript in their training set, but only a small set of contrived examples of tool calls.”

Here is the pattern from Anthropic’s PTC documentation. In Anthropic’s sequential expense-analysis illustration, traditional tool calling requires 20+ separate inference passes, with intermediate data flowing through context. After the team lookup, a host that supports parallel tool calls can batch the independent expense requests; the 20+ figure does not make that impossible. Anthropic reports that generated code answering the same question cuts what reaches the context from 200KB of raw expense rows — over 2,000 line items — down to 1KB of results. The script below illustrates that control flow using custom async Python adapters: they accept positional arguments and return decoded lists and dictionaries. It is not the native PTC wrapper contract.

# Custom decoded Python adapters, not native Claude PTC wrappers.
import asyncio
import json

async def main() -> None:
    team = await get_team_members("engineering")
    levels = list(set(member["level"] for member in team))
    budgets = dict(zip(
        levels,
        await asyncio.gather(*(get_budget_by_level(level) for level in levels)),
    ))
    expenses = await asyncio.gather(
        *(get_expenses(member["id"], "Q3") for member in team)
    )
    over_budget = []
    for member, employee_expenses in zip(team, expenses):
        total = sum(expense["amount"] for expense in employee_expenses)
        limit = budgets[member["level"]]["travel_limit"]
        if total > limit:
            over_budget.append(
                {"name": member["name"], "spent": total, "limit": limit}
            )
    # Only this final summary returns to the LLM context
    print(json.dumps(over_budget))

asyncio.run(main())

To use native Claude PTC wrappers, pass each tool one argument dictionary, decode its returned JSON string, and use top-level await in the managed execution environment rather than starting an event loop with asyncio.run. The custom-adapter example above assumes an ordinary Python script runtime; it is also not a drop-in native MCP connector example. The current PTC API constraints exclude strict: true tools and native MCP connector tools from programmatic calling and restrict recursive schemas. A custom code-to-MCP bridge is a separate integration. allowed_callers guides how Claude calls a tool; it is not an authorization boundary. The host must validate every returned invocation, including an unexpected direct call.

The LLM sees only the final JSON summary, not the thousands of expense line items processed in the sandbox. The saving is not specific to expense reports: Anthropic’s separate code-execution write-up puts the sharpest number on the pattern, a Google Drive-to-Salesforce workflow that fell from ~150,000 tokens to ~2,000, a 98.7% reduction.

The current PTC guide also reports a counterexample: on tau2-bench’s airline, retail, and telecom tasks, PTC left scores unchanged and cost roughly 8% more. On a separate 75-tool project-management benchmark, it cut billed input tokens by roughly 38% without changing accuracy. These internal evaluations name only a production Claude model, not an exact ID. Small sequential workflows may not save enough to offset container and code-generation overhead.

Token efficiency is a potential gain. Loops and conditionals come for free, and code execution can handle errors with explicit handlers instead of making the model reason about failures in natural language. A code-execution path can keep sensitive intermediate data out of model context, but that is not confidentiality: isolation, egress controls, scoped credentials, and logging need separate enforcement.

When JSON tool calling still makes sense: single atomic operations, environments without sandboxing infrastructure, smaller models with weak code generation, or audit requirements that need every individual tool invocation logged.

AI agent tool-execution comparison

JSON tool calling, MCP, CLI, and code execution are execution surfaces. Skills are instructions that guide one of those surfaces, so their latency, context, and setup depend on the selected mechanism. “Meta-tools” means the few generic entry points a code-executing agent needs—Vercel’s ExecuteCommand and ExecuteSQL, for example—instead of one schema for every operation. The composability and latency rows describe calls whose later arguments depend on earlier results. JSON tool calling and MCP can issue independent calls together, but dependent calls usually need another model turn. PTC moves that dependent control flow and filtering into a script, then returns a summary to the model. The token and task-success cells summarize cited examples, not one controlled benchmark across all five columns.

The Agent-Computer Interface (ACI) for AI agent tools

The term “Agent-Computer Interface” (ACI) was coined by John Yang, Carlos E. Jimenez, and colleagues at Princeton in their SWE-agent paper (NeurIPS 2024). Human interface quality gets a whole discipline devoted to it — human-computer interaction, or HCI. The paper argues that language-model agents deserve the same treatment: they are “a new category of end users with their own needs and abilities, and would benefit from specially-built interfaces.”

Their ablation results put a number on that. Using the same GPT-4 Turbo base model, the paper’s SWE-bench Lite ablation reached 18.0% with SWE-agent’s full ACI on 300 tasks, versus 7.3% for the shell-only condition without a worked demonstration and 11.0% with one. The comparison shows that the interface and demonstration conditions materially changed performance in this setup; it does not isolate interface design from every other difference or show that the model did no work. Within the same interface ablation, enabling linting raised the edit condition from 15.0% to 18.0%; across the full SWE-bench test set, 51.7% of SWE-agent’s runs hit at least one edit the linter rejected before it could propagate.

Anthropic adopted ACI as a foundational concept in their “Building Effective Agents” guide, listing it as one of three core principles: “Carefully craft your agent-computer interface through thorough tool documentation and testing.” Their practical guidance: “One rule of thumb is to think about how much effort goes into human-computer interfaces, and plan to invest just as much effort in creating good agent-computer interfaces.”

Four ACI principles in practice

1. Actions should be simple and easy to understand. The most common mistake is wrapping API endpoints one-to-one. Instead of list_users, list_events, create_event, implement schedule_event that finds availability and schedules in one call. Instead of read_logs, implement search_logs that returns only the relevant lines with context.

2. Actions should be compact and efficient. Consolidate important operations into as few actions as possible. In the Market Analyst Agent, I combine price fetching with basic metrics into a single get_stock_snapshot tool rather than requiring separate calls for price, volume, market cap, and PE ratio.

3. Environment feedback should be informative but concise. Avoid returning raw HTML or full API payloads. Resolve cryptic IDs to semantic names. Anthropic’s testing added a response_format enum so the agent can ask for a concise (~72 tokens) or a detailed (~206 tokens) response, roughly a 3x difference in token cost.

4. Validation should mitigate error propagation. Automatic error detection helps agents recognize and correct mistakes quickly. In SWE-agent, a custom file editor with integrated linting automatically rejects syntax errors — the validation step behind the 51.7% figure above. This is validation on a tool’s inputs and outputs, not the content filtering around a model call that the guardrail products in Part 4 do; the same word gets used for both. I apply the same principle in the Market Analyst Agent by validating tool arguments with Pydantic schemas before execution:

from pydantic import BaseModel, Field, field_validator

from market_analyst.utils import normalize_ticker

class StockQuery(BaseModel):
    """Validated input for stock queries.

    Pydantic catches malformed tickers before the API call,
    preventing error propagation through the reasoning loop.
    """
    ticker: str = Field(description="Stock ticker symbol (e.g., NVDA)")

    @field_validator("ticker")
    @classmethod
    def validate_ticker(cls, v: str) -> str:
        return normalize_ticker(v)

class StockHistoryQuery(StockQuery):
    """Validated input for price history queries."""

    period: str = Field(default="1mo", description="Time period: 1d, 5d, 1mo, 3mo, 6mo, 1y")

    @field_validator("period")
    @classmethod
    def validate_period(cls, v: str) -> str:
        valid = {"1d", "5d", "1mo", "3mo", "6mo", "1y"}
        if v not in valid:
            raise ValueError(f"Invalid period: {v}. Must be one of {valid}")
        return v

The shared normalizer trims and uppercases the value, then accepts ticker digits and dotted or hyphenated suffixes such as BRK.B and BF-B; StockHistoryQuery, not StockQuery, owns period.

AI agent tool design patterns that work

Anthropic’s “Writing effective tools for agents” guide frames tools as “a new kind of software which reflects a contract between deterministic systems and non-deterministic agents.”

Treat tool descriptions as prompt engineering

Descriptions should run at least three or four sentences, covering when to use the tool, required versus optional parameters, output format, and edge cases. Anthropic reports that the choice between prefix- and suffix-based namespacing (asana_search versus search_asana) had “non-trivial effects” on its own tool-use evaluations. It does not say which scheme wins, so test both on your toolset rather than assuming prefixes. Anthropic also fed the transcripts from its evaluation agents back into Claude Code and let it rewrite the tools. On held-out test sets that loop found further improvements “even beyond what we achieved with ‘expert’ tool implementations” — whether those tools were hand-written by its researchers or generated by Claude.

# Bad: vague, no context for when to use
tools = [{
    "name": "search",
    "description": "Search for items",
}]

# Good: specific, with input examples and edge cases
tools = [{
    "name": "search_news",
    "description": (
        "Search for recent news articles about a specific stock or company. "
        "Use this tool when the user asks about recent events, earnings, "
        "announcements, or market-moving news for a specific ticker. "
        "Returns up to 10 articles sorted by relevance. "
        "For company competitors rather than news, use search_competitors instead."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "Search query. Examples: 'NVDA earnings Q3 2025', 'Tesla delivery numbers'"
            },
            "max_results": {
                "type": "integer",
                "description": "Max articles to return (1-10, default 5)",
                "default": 5
            }
        },
        "required": ["query"]
    }
}]

Anthropic’s internal testing showed that adding an input_examples field lifted accuracy on complex parameter handling from 72% to 90%.

Return high-signal, machine-readable output

Use semantic labels instead of low-level identifiers (uuid, mime_type) in the default response. Keep an ID when a later tool needs it, or offer a detailed response that includes it. For example, a search result for Jane can be concise to read, while a detailed result includes the ID that send_message needs. Structure the response so the agent can reason about it without parsing boilerplate:

# Bad: raw API response dumped to agent
def get_stock_snapshot(ticker: str) -> dict:
    response = api.get(f"/v1/quotes/{ticker}")
    return response.json()  # 500+ tokens of nested JSON

# Good: high-signal summary the agent can immediately reason about
def get_stock_snapshot(ticker: str) -> dict:
    data = api.get(f"/v1/quotes/{ticker}").json()
    return {
        "ticker": ticker,
        "price": data["regularMarketPrice"],
        "change_pct": round(data["regularMarketChangePercent"], 2),
        "volume": data["regularMarketVolume"],
        "market_cap_b": round(data["marketCap"] / 1e9, 1),
        "pe_ratio": data.get("trailingPE"),
        "summary": f"{ticker} at ${data['regularMarketPrice']:.2f} "
                   f"({'up' if data['regularMarketChangePercent'] > 0 else 'down'} "
                   f"{abs(data['regularMarketChangePercent']):.1f}%)"
    }

Return errors the loop can act on

Error handling needs four separate mechanisms, because they handle different failure classes:

1. Retry with exponential backoff for transient errors
2. Model fallback chains for provider outages
3. Error classification routing — transient errors retry, LLM-recoverable errors return to the agent with context, human-required errors escalate
4. Checkpoint recovery for crash survival

Anthropic’s “Writing effective tools for agents” argues for clear tool errors and evaluation-driven tool design, but it puts no universal number on what these four mechanisms recover. Measure recovery rate, retries, and escalation on your own task suite.

import httpx
from tenacity import retry, retry_if_exception, stop_after_attempt, wait_exponential

def is_transient_error(error: BaseException) -> bool:
    if isinstance(error, (httpx.TimeoutException, httpx.NetworkError)):
        return True
    if isinstance(error, httpx.HTTPStatusError):
        return error.response.status_code == 429 or 500 <= error.response.status_code < 600
    return False

@retry(
    retry=retry_if_exception(is_transient_error),
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=2, max=10),
    reraise=True,
)
def call_stock_api(ticker: str) -> dict:
    """Fetch stock data with automatic retry on transient failures.

    Mechanism 1 of the four above: exponential backoff for rate limits
    and network blips.
    This only retries transient transport failures. If attempts are exhausted,
    Tenacity re-raises the original httpx exception.
    """
    response = httpx.get(
        f"https://api.example.com/v1/quotes/{ticker}",
        timeout=10.0,
    )
    response.raise_for_status()
    return response.json()

The caller or harness still needs the next step: turn that exception into a stable result that says which operation failed, whether to retry, and what to do next. A non-retryable 4xx skips this decorator and needs the same treatment. Retrying a request does not classify its error or recover a checkpoint.

Applying the patterns to the Market Analyst Agent

The Market Analyst Agent from Part 1 makes the effect of the interface visible.

Tool consolidation

The original tool modules defined get_stock_price, get_company_metrics, get_price_history, two search tools, and execute_trade. For a basic analysis, the agent had to choose both the price and metrics calls; market cap and P/E were fields of get_company_metrics, not standalone tools. The pre-consolidation source shows that earlier surface.

I reshaped the market-data surface into 5 high-level tools, following the ACI principle of compact, efficient actions. The repo’s ReAct tool list carries four more alongside them — a skill loader, two CLI wrappers, and a restricted in-process Python evaluator (an AST allowlist, not a sandbox; Part 4 takes that up) — covering three of the five modalities above. MCP shows up as a sidecar rather than as a tool in this list:

This moves price and valuation into one task-shaped definition and adds financial statements as an explicit action. Whether that improves tool selection is a claim to test against representative requests and traces.

Structured outputs for tool results

The stock and news tools return Pydantic-validated responses. The CLI and code-execution wrappers return str, so the models below describe the structured tool results rather than every wrapper in the repository:

from pydantic import BaseModel

class StockSnapshot(BaseModel):
    """Structured tool response — the agent never sees raw API noise."""
    ticker: str
    price: float
    change_pct: float
    volume: int
    market_cap_b: float
    pe_ratio: float | None
    summary: str  # Human-readable one-liner for direct use in reports

class NewsItem(BaseModel):
    """One news item pre-processed for agent consumption."""
    headline: str
    source: str
    date: str
    relevance_score: float  # Pre-ranked so the agent doesn't waste tokens sorting
    key_points: list[str]  # Extracted by the tool, not the agent

class NewsSearchResult(BaseModel):
    query: str
    results: list[NewsItem]
    summary: str

The summary field gives the agent a ready-to-use string for a report. NewsItem.key_points save the model from parsing article bodies. If a later action needs an ID, keep it in the detailed response or offer concise and detailed modes; do not remove it everywhere.

Trade-offs and considerations

Beyond the caveats specific to each pattern above, a few cross-cutting concerns shape the choice:

• Operational cost varies by dimension. Code execution saves tokens but adds sandbox cold-start latency. MCP saves development time for SaaS integrations but adds server deployment overhead. CLI is free to start but harder to govern at scale. Optimize for your actual bottleneck, whether that is token cost, latency, or operational complexity.

• Team skills matter. Code execution assumes your agents (and the models behind them) can generate reliable Python or TypeScript. CLI assumes familiarity with Unix conventions. MCP requires understanding transport protocols and OAuth flows. Match the modality to your team’s strengths.

• Tool consolidation can go too far. If one tool accumulates unrelated modes and arguments, the agent faces a different selection problem inside the schema. Use tool-selection and task-success evaluations to find the right surface for your workload.

• Skills are prompt-based, not enforced. A skill is instructions the agent should follow, not guardrails it must follow. A skill bundle can include arbitrary files and executable scripts, so trust its source, review the bundle, and have the host enforce the permissions for every resource it can read, change, or execute. For critical workflows, combine skills with deterministic validation.

• Audit requirements shape the choice. Structured MCP and JSON calls are convenient events to log, but neither protocol creates a complete audit trail out of the box. The host, server, or harness must record invocations and results, then enforce authorization, policy, retention, and review. Code execution needs the same instrumentation around the sandbox; its script and output alone are not a compliance record.

Three directions for AI agent tooling at scale

The first is tool RAG for scaling. Before the model chooses a tool, retrieve the few tool descriptions that match the request and let it choose from that subset instead of the full registry. In RAG-MCP’s benchmark tasks and MCP stress test, its baseline tool-selection accuracy was 13.62%; retrieval raised it to 43.13%, a 3.2x improvement, while reducing prompt tokens from 2,133.84 to 1,084 (about 49.2%). The paper’s abstract says “over 50%,” and its generator/evaluator descriptions differ across sections; those inconsistencies limit interpretation. The result is evidence for that evaluation setup, not a universal rate for naive selection as toolsets grow.

The second is agents creating their own tools. The LATM framework (“LLMs As Tool Makers”) established a two-phase paradigm where a powerful LLM creates reusable Python functions and a lightweight LLM uses them. On ToolMaker’s 15-task benchmark of papers with public code repositories, supplied as GitHub URLs and short task descriptions, it correctly implemented 12 of 15 tasks; the benchmark contains more than 100 tests in total. That small repository-task benchmark does not establish production reliability. Both point past tool use toward tool creation, and then toward managing a library of generated tools.

The third is the A2A + MCP dual-protocol stack. Google transferred A2A to the Linux Foundation in June 2025. The A2A protocol documentation separates their responsibilities: MCP connects an agent to tools and resources, while A2A lets independent agents discover each other, negotiate interactions, manage shared tasks, and delegate work.

Compare interfaces on identical tasks and permitted operations. Record discovery tokens, cached and uncached input, execution output, retries, latency, and final-state success. Test missed discovery as well as token savings; for generated programs, count syntax errors, runtime errors, and partial completion.

Key takeaways

1. Choose the execution surface from the action: JSON calls for small typed operations, MCP for shared services, CLI for established commands, and sandboxed code for local composition. Use Skills to document how to choose and use that surface.
2. Keep benchmark conditions attached to the result. CodeAct, Anthropic, Vercel, Cloudflare, Apideck, and Scalekit measured different models, tasks, tools, and harnesses.
3. ACI quality survives protocol changes. Clear actions, compact feedback, validation, and useful errors help every modality.
4. Consolidate overlapping tools only when evaluations show that the smaller surface improves selection or task success.
5. Security moves with execution power. Shell and code interfaces need sandboxing; MCP needs scoped identity and server policy; Skills remain instructions rather than enforcement.

The next layer is policy

Part 4, AI Agent Security, covers the harness check between a proposed tool call and execution. Part 5 puts the tool and its sandbox inside a recoverable runtime. Part 6 adds a contract the model does not see: an effect category, retry rule, and structured result that an acceptance check can read without parsing prose.

References

Papers

• Executable Code Actions Elicit Better LLM Agents (CodeAct) — Wang et al., ICML 2024 — Code-based actions report up to 20 percentage points higher task success in the paper’s tasks
• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering — Yang, Jimenez et al., NeurIPS 2024 — ACI design principles and SWE-bench evaluation (18.0% SWE-agent vs 7.3% shell-only without a demonstration in the paper’s 300-task SWE-bench Lite ablation; interface and demonstration conditions differ)
• RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection — Its benchmark tasks and MCP stress test improved selection accuracy from 13.62% to 43.13%
• LLMs As Tool Makers (LATM) — Cai et al., 2023 — Two-phase paradigm for agent tool creation
• ToolMaker: LLM Agents Making Agent Tools — Wolflein et al., ACL 2025 — 80% on its 15-task benchmark of papers with public code repositories
• MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers — 72.8% average attack success for o1-mini in a benchmark covering 20 agents and 45 servers

Anthropic engineering

• Advanced Tool Use / Programmatic Tool Calling — Tool Search (approximate context endpoints ~77K and ~8.7K), PTC, and tool use examples
• Code Execution with MCP — 98.7% token reduction (150K to 2K tokens) through code-based tool orchestration
• Writing Effective Tools for Agents — Tool description engineering; input examples improve accuracy; response_format enum (72 vs 206 tokens)
• Building Effective Agents — ACI as a foundational design principle

Protocol specifications

• Google Cloud donates A2A to Linux Foundation — June 23, 2025 announcement of the protocol, SDK, and tooling transfer
• A2A and MCP: Detailed Comparison — A2A protocol documentation on the complementary agent-to-agent and agent-to-tool responsibilities

Industry case studies

• Vercel: We Removed 80% of Our Agent’s Tools — old example names 17 tools; new example exposes ExecuteCommand and ExecuteSQL; Vercel frames the redesign as removing 80%; 4/5 to 5/5 success across five representative queries, 3.5x faster, 37% fewer tokens
• Cloudflare: Code Mode — TypeScript-driven API calls replacing tool schemas
• Apideck: MCP Server Eating Your Context Window — 550-1,400 tokens/tool, 55K tokens for ~40 MCP tools, 143K/200K context consumed
• Scalekit: MCP vs CLI Token Benchmark — 4-32x token overhead for MCP vs CLI across 75 benchmark runs

Security

• Vulnerable MCP Project — Vulnerability reports for constructing threat-model tests
• AuthZed: Timeline of MCP Breaches — 9 major MCP security incidents (April-October 2025)
• Invariant Labs: MCP Tool Poisoning Attacks — Tool poisoning, rug pulls, and cross-origin escalation
• Pivot Point Security: MCP Security Analysis — 43% command injection, 43% OAuth authentication flaws

CLI design

• Writing CLI Tools That AI Agents Actually Want to Use — Ugo Enyioha — Eight design rules for agent-friendly CLIs

Demo project

• Market Analyst Agent — Full implementation with tool consolidation and ACI patterns

The complete Market Analyst Agent code, including the tool designs described in this post, is on GitHub.