Skip to content

Blog

The Hands — Tool Ergonomics and the Agent-Computer Interface

Part 3 of the Engineering the Agentic Stack series

Part 1 covered reasoning loops, Part 2 covered memory. This post is about tools — how agents interact with the world, and why tool design matters more than most teams realize.

The tool landscape shifted dramatically in 2025–2026. MCP won the standards war — but production teams are discovering its security gaps and token overhead the hard way. The alternative gaining traction: agents that write and execute code instead of calling JSON-defined tools, achieving 98.7% token reductions and 20% higher task success rates.

This post covers all five tool modalities, the ACI design principles that govern them, and practical patterns I apply in the Market Analyst Agent.

The LLM Engineering Field Guide: 45 Concepts Every Practitioner Needs

Every production LLM system sits at the intersection of GPU physics, systems engineering, and ML theory. Whether you are optimizing Time to First Token for a chatbot or configuring DeepSpeed ZeRO for a fine-tuning run, the same interconnected set of concepts keeps surfacing. This guide distills them into a single practical reference.

TL;DR: I organized the 45 essential concepts every LLM practitioner encounters into eight parts — hardware foundations, inference fundamentals, inference optimizations, model architecture, training and alignment, scaling and deployment, applications, and production operations. Each entry covers what the concept is, why it matters, the key numbers, and how it connects to everything else. All data reflects the 2024–early 2026 landscape with verified references.

This guide assumes familiarity with basic ML concepts (backpropagation, gradient descent, softmax) and some systems knowledge (memory hierarchies, networking basics).

The Definitive Guide to OCR in 2026: From Pipelines to VLMs

The model that tops the most popular OCR benchmark ranks near-last when real users judge quality. That single fact captures the state of OCR in 2026: the field has shifted so fundamentally that our evaluation methods haven't caught up. And it matters more than ever — OCR is now the front door to every RAG pipeline and AI assistant.

Vision-Language Models (VLMs) now dominate text extraction from complex documents, achieving 3–4× lower Character Error Rate (CER) than traditional engines on noisy scans, receipts, and distorted text. The OCR Arena leaderboard (early 2026) tells the definitive story: Gemini 3 Flash leads, followed by Gemini 3 Pro, Claude Opus 4.6, and GPT-5.2. These general-purpose frontier models are outperforming dedicated OCR systems. Meanwhile, open-source models like dots.ocr (1.7B params, 100+ languages) and Qwen3-VL (2B–235B) deliver remarkable quality at near zero cost.

Yet traditional engines remain the fastest, cheapest option for clean printed documents, and no single approach dominates every scenario. Let's break down the entire OCR ecosystem today: the benchmarks, the models, the metrics, and how to actually deploy this in production.

🔧 Companion repo: The OCR Gauntlet — a runnable single-notebook demo comparing 5 document types across 3 model tiers (Tesseract → dots.ocr → Gemini 3 Flash) to expose the quality cliffs and cost trade-offs.

Benchmarking Modern Data Processing Engines

Historically, data processing often relied on Pandas for in-memory workloads and Apache Spark for distributed processing. This approach was highly effective for structured tabular data.

However, modern workloads increasingly involve multimodal data, such as images, audio, and video frames. In these scenarios, traditional challenges like the JVM's garbage collection overhead and Python's Global Interpreter Lock can become performance bottlenecks, particularly when integrating GPU inference. This shift has accelerated the adoption of engines built on Rust and Apache Arrow.

Having transitioned many of my own single-instance pipelines to Polars, I wanted to objectively evaluate how it compares to other modern engines. This article presents a comparison of several Rust-based compute frameworks using real-world datasets to provide a practical guide for choosing the right tool.

The entire benchmark suite and runnable code can be found in the engine-comparison-demo repository. I encourage you to clone it, run the benchmarks on your own hardware, and verify the results yourself.

The Cortex — Architecting Memory for AI Agents

Part 2 of the Engineering the Agentic Stack series

State is what separates a chatbot from an agent. Without memory, every interaction starts from zero — the agent cannot pause and resume, cannot learn from past sessions, cannot personalize. In Part 1, I covered the cognitive engine that decides how an agent thinks. This post tackles the infrastructure that determines what it remembers.

I'll walk through the memory architecture of the Market Analyst Agent, showing how hot and cold memory layers work together to support checkpointing, pause/resume workflows, and cross-session learning — and why a third tier of document-based memory is becoming essential for agents that manage their own knowledge.

Building a Modern Search Ranking Stack: From Embeddings to LLM-Powered Relevance

Search is no longer a string-matching problem. A query for "wireless headphones" on a product search engine is not just about finding items containing those two words — it is about surfacing the best result based on semantic relevance, product quality, user preferences, and real-time availability. The gap between BM25 keyword matching and what users actually expect has forced a complete rethinking of search architecture.

This post walks through the anatomy of a modern search ranking stack: a multi-stage pipeline that combines sparse lexical retrieval, dense semantic embeddings, reciprocal rank fusion, cross-encoder reranking, and LLM-powered listwise ranking. I built a working demo that benchmarks each stage on the Amazon ESCI product search dataset — proving the value of every layer with real numbers.

The Cognitive Engine: Choosing the Right Reasoning Loop

Part 1 of the Engineering the Agentic Stack series

Building production AI agents is no longer about prompt engineering—it's about system engineering. The difference between a demo that impresses and a product that delivers comes down to one critical decision: how your agent thinks.

This post introduces three reasoning loop architectures and shows you how to choose between them. I'll use a production-grade Market Analyst Agent as the running example, with code you can use today.

mHC: How DeepSeek Scaled Residual Connections Without Breaking Training

The success of modern deep learning rests on a deceptively simple idea: the residual connection. Yet after a decade of stacking layers deeper and deeper, researchers at DeepSeek asked a different question—what if we could scale width instead? Their answer, Manifold-Constrained Hyper-Connections (mHC), solves a fundamental instability problem that has blocked this path for years.

In this post, I'll break down the evolution from basic residuals to mHC, explaining why each step was necessary and how exactly DeepSeek's solution works at scale.

Enterprise RAG Challenge 3: Winning Approaches for Autonomous AI Agents

The Enterprise RAG Challenge 3 (ECR3) just concluded, and the results reveal powerful insights for building production-grade AI agents. Out of 524 teams and over 341,000 agent runs, only 0.4% achieved a perfect score—making this one of the most demanding benchmarks in the field. With the leaderboard and solution write-ups now public, I analyzed the winning approaches to distill the patterns that separated top performers from the rest.

This article breaks down what ECR3 is, what tasks were involved, and how the best teams solved them.

The Complete Guide to LLM Fine-Tuning in 2025: From Theory to Production

Fine-tuning has become the secret weapon for building specialized AI applications. While general-purpose models like GPT-4 and Claude excel at broad tasks, fine-tuning transforms them into laser-focused experts for your specific domain. This guide walks you through everything you need to know—from understanding when to fine-tune to deploying your custom model.