Series.

Multi-part deep dives on the practices that unlock what AI tools can actually do. Each one is written to be read in order.

series
4
parts
10
minutes of reading
104

Series 01

AI-assisted delivery

AI-assisted engineering is the practice of letting models write production code while keeping the controls that decide whether it is safe to ship. This is what the research actually found, why AI-written code fails differently, what to measure, and which gates have to exist before the tools go out.

Parts
1
Minutes to read
13

Latest part published

  1. Part 1: What is AI-assisted engineering? A guide to shipping faster without losing the quality

    AI-assisted engineering is the practice of letting models write production code while keeping the controls that decide whether it is safe to ship. This is what the research actually found, why AI-written code fails differently, what to measure, and which gates have to exist before the tools go out.

    14 min read

Series 02

AI Eval Discipline

Most teams adopting AI ask which eval framework to buy. Wrong question. Evals are not a benchmark suite — they are a habit of looking at your data, naming the failure modes, and writing assertions that fire when they recur. This series walks the discipline from foundations to operational practice: what evals actually are, the three gulfs LLM development has to bridge, how to build an eval set without burning a quarter, and why mutation testing is the only honest eval for AI-written tests.

Parts
5
Minutes to read
66

Latest part published

  1. Part 1: What is AI evaluation? A guide for teams putting AI into production

    AI evaluation is how you find out whether your AI system is right, on your data, before a customer does. This is the whole picture: what an eval is, how it differs from a benchmark, what to measure, how to build your first set, and where it sits in the pipeline.

    23 min read

  2. Part 2: Evals Aren't a Benchmark Suite. They're a Habit of Looking at Your Data.

    Most teams buy an eval tool, run it once, and call it done. They confuse benchmarks with evals — and ship AI that confidently produces wrong outputs no one will catch. The work the tool can't do for you is the work that matters: looking at your traces, naming the failure modes, and writing assertions that fire when they recur.

    12 min read

  3. Part 3: You Can't Look at Data You Don't Have: Building Your First Eval Set

    The first eval post said pull a hundred production traces and read them. But what if you haven't launched? Here's how to build your first eval set from dimensions, scenarios, and synthetic data — and why that set is scaffolding, not the building.

    11 min read

  4. Part 4: You Don't Have Evals. You Have One Kind of Eval.

    Most teams build one corner of the eval map — offline, end-to-end, on the inputs they imagined — and call it "evals," the way you'd say "we have tests." Eval is plural. Here's the map: Hamel Husain's three levels, and the three questions that decide which kind of eval you're actually writing.

    11 min read

  5. Part 5: Error Analysis Is the Eval Work. Here's How to Actually Do It.

    Everyone agrees you should "look at your data." Then they open a hundred traces, scroll for ten minutes, feel vaguely worried, and reach for a tool. The looking has a method — open coding, then axial coding into a ranked taxonomy — and the taxonomy, not your assumptions, is what decides which evals to write.

    12 min read

Series 03

Agents in production

Six AI systems, each live for a named client, each written from the engagement itself. What the client asked for, the part that was harder than it looked, what we built instead, and what we left out on purpose. Read them for the pattern: grounded, cited, judged by the client before it was trusted.

Parts
2
Minutes to read
10

Latest part published

  1. Part 1: Give the model buttons, not a database

    Ask an AI model for a figure it does not have and it will give you one anyway, fluently, and wrong. If your name goes on the numbers, that is not a risk you can prompt away. Here is the pattern we use so an assistant cannot invent a figure, explained from the start, and the climate data publisher it runs for today.

    6 min read

  2. Part 2: Before you touch the prompt, check what reached the model

    When an AI assistant answers wrongly, the first instinct is to rewrite the instructions. Half the time the instructions are fine and the model never saw the figure at all. Here is how retrieval works, the failure that looks like a model failure, how to tell them apart, and the funds firm where the returns turned out to be pictures.

    6 min read

Series 04

AI Won't Do This by Default — Until You Do

AI coding agents are incredibly powerful. But the teams getting unreal results combine them with practices most teams skip. This series walks the full delivery value stream — one practice per post — showing what AI tools do brilliantly, what they don't do by default, and what happens when you add the practice that unlocks their real power.

Parts
2
Minutes to read
15

Latest part published

  1. Part 1: AI Won't Do This by Default: Hypothesis-Driven Development

    AI coding agents are the most powerful tools engineering has ever had. But the teams getting unreal results aren't just prompting — they're combining AI with practices that unlock outcomes nobody else can explain. Hypothesis-driven development is the first unlock.

    6 min read

  2. Part 2: AI Won't Do This by Default: User Story Mapping & Shape

    Even a good hypothesis needs a map before it needs code. AI agents build whatever ticket lands in front of them — they can't see the journey the ticket sits inside. User story mapping is the practice that turns a backlog of features into a walking skeleton of the user's journey. It's the second unlock.

    11 min read

Want one of these practices in your team?