AI for product managers
You do not need to become an engineer to ship AI well
You need a way to tell whether the thing is working. That is what evals are, and this is the shortest path I know to being useful on an AI product: read the four pieces below in order, then run one of the tools against something you actually own.
Start here
Four pieces, in order. Together they take you from not being sure what an eval is to being able to write one for your own product.
What Is an AI Eval?
Start here if the word "eval" is still fuzzy.
An AI eval is a repeatable test that scores an AI's output against a standard of what good looks like. Here's what that means, what an eval is made of, and why it matters.
How to Design AI Evals: A Practical Guide
The practical version: how to write one for your own product.
An AI eval is a repeatable test that scores your AI's output against what good looks like. Here's how to design one that actually catches problems, from defining the task to choosing how you grade.
How to Test for AI Hallucinations
The failure mode everyone worries about, made testable.
An AI hallucination is output that sounds confident but isn't grounded in fact or source. Here's how to build an eval that catches them before your users do.
Eval-Driven Development for AI Products
What changes about how you build once evals exist.
Eval-driven development means writing the eval alongside the AI feature and using its score to guide every change. Here's the loop, and how to grade LLM output you can trust.
Try it on your own product
Reading about evals does not get you one. These are free, need no sign-up, and run on whatever you are working on now.
AI Eval Builder
Turn a vague quality worry into a scored, repeatable test. Produces evals you can hand to an engineer.
AI Health Check
A short diagnostic on whether your AI feature is actually working, or just demoing well.
AI Maturity Assessment
Where your team sits today, and the specific next step rather than a generic maturity ladder.
How I think about it
Longer pieces on strategy and on the toolkit I built and use daily.
How I Think About AI Product Strategy
AI product strategy is not about what the model can do. It is about what the user needs done. The distinction sounds obvious. It is not. Here is the framework I use to tell the difference.
How I Built a 340-Skill AI Toolkit (and the 3-Layer Framework Behind It)
I spent a year building a product leadership toolkit with 340+ agent skills, practice guides, and delivery templates. The prompts were easy. Making them findable, trustworthy, and useful at scale required a framework I didn't expect to need.
From the Google Sheet to the AI Toolkit
In 2014, I managed 24 engineers building a car dealership negotiation platform. In 2026, I ship comparable output solo with AI tooling. Here is what transferred and what did not.
Working together
Most AI pilots stall for the same few reasons
No clear use case, no guardrails, no operating model. I was the AI Practice Lead at Artium, where I built a 340-skill agent system that powered real consulting work rather than demos. If your team is stuck somewhere in that list, that is the work I do.