How I Think About AI Product Strategy
Most AI products fail because they lead with the model, not the job.
A PM I was coaching came to me with a product idea. The pitch was good: use an LLM to classify customer support tickets, route them to the right team, and suggest a first response. The demo was impressive. The model was fast. The accuracy was high.
I asked one question: "What does the support agent do after they see the suggestion?"
The PM paused. They had not mapped the workflow past the AI output. The model classified tickets beautifully. But no one had designed what happened next: how agents edited the suggestion, how confidence thresholds affected routing, what happened when the model was wrong, how the team would know if agents trusted it or ignored it.
The model worked. The product did not exist yet.
Capability vs. job
I wrote about the Fruit Tool problem -- the pattern where teams build one clever AI feature, wrap it in a UI, and call it a product. The diagnostic I use is simple: if you remove the AI, is there still a product? If not, you have a demo.
That diagnostic is the starting point for how I think about AI product strategy. The full framework has three layers.
Layer 1: Start with the job, not the model
Every AI product I have built started with the same question: what is the person trying to accomplish?
At FLUXX Health, we built a menopause education platform with AI-driven phase detection. The capability was impressive -- quiz responses mapped to hormonal phases using a scoring engine. But the job was not "detect my phase." The job was "help me understand what is happening to my body and what to do about it." Phase detection was infrastructure. The product was clarity.
When I built assessment tools for my own practice, the same principle applied. The AI Maturity Assessment does not just produce a score. It surfaces the specific dimension where a team is weakest and recommends the first action for that week. The output is a decision, not a number.
If your AI product's primary value proposition is "look what the model can do," you are building a demo. If the value proposition is "here is what you should do next," you are building a product.
Layer 2: Design for the moment the model is wrong
Every model is wrong sometimes. The question is not whether it will be wrong. The question is what happens when it is.
Most AI product teams spend 90% of their energy on the happy path -- the model is right, the user is delighted. The best teams spend equal energy on the failure path. What does the user see when confidence is low? How do they correct the output? Does the correction make the system better?
At FLUXX, we built uncertainty handling into the phase detection logic. When the model could not confidently assign a phase, it said so. It did not guess. That decision was hard -- it meant some users got a less satisfying result on their first use. But it also meant users trusted the results they did get. Trust compounded. Engagement stayed high.
The failure path is where AI products earn or lose trust. Design it as carefully as the happy path.
Layer 3: Measure the outcome, not the output
AI products have a unique measurement trap: the model metrics can be excellent while the product metrics are terrible.
A classification model with 95% accuracy sounds great. But if agents override the suggestion 40% of the time, the product is not working. If the time-to-resolution does not decrease, the classification is not helping. If agents stop looking at the suggestions after the first week, accuracy does not matter.
The metrics that matter are always downstream of the model:
- Did the user accomplish their goal faster?
- Did the user trust the result enough to act on it?
- Did the product reduce the work the user had to do?
At FLUXX, we measured quiz completion rates and return visits -- not just phase detection accuracy. A model that detected phases correctly but produced a quiz that 60% of people abandoned was a model that was not helping anyone. When we optimized the quiz for mobile and reduced abandonment by 38%, that was a product win. The model did not change. The experience around it did.
The framework in practice
When I evaluate an AI product opportunity, I run three questions in order:
1. Can I describe the user's job without mentioning AI? If the pitch only works because "AI" is in the sentence, the job is not real. "Help support agents resolve tickets faster" is a job. "Use AI to classify tickets" is a capability.
2. What happens when the model is wrong? If the answer is "it won't be" or "we'll improve accuracy," the failure path has not been designed. Every model will be wrong. The product needs to handle it gracefully.
3. What outcome am I measuring? If the primary metric is model accuracy, the team is optimizing the wrong thing. Find the user outcome that the model enables, and measure that.
The organizational problem
The hardest part of AI product strategy is not the framework. It is the organizational pressure to ship something that demos well.
Executives who have seen ChatGPT want an AI feature. Boards want an AI story. Sales wants an AI pitch. The pressure to wrap a model in a UI and ship it is enormous.
The best AI product leaders I know resist that pressure by reframing the conversation. They do not say "we should not use AI." They say "here is the user problem AI can solve, and here is how we will know it is working." They make the job the hero of the story, not the model.
That reframe is the entire strategy. Start with the job. Design for failure. Measure outcomes. Everything else is implementation.
I help teams build AI products that solve real problems, not demos that impress in a boardroom. If your team is figuring out where AI fits in your product strategy, let's talk.