Skip to main content

AI Skills

Build AI products that survive production

A curated track through the skills for shipping AI features that hold up under real use: specs, evals, agents, RAG, guardrails, testing, and observability. Each one names the hard part a generalist misses, not just the steps.

Two learning paths

Start here if you want a sequence rather than a menu. Same skill library, ordered for the job you actually have.

For engineers

Spec → Prototype → Eval → Deploy → Monitor

  1. 1
    SpecAI Product Spec
  2. 2
    PrototypeAI Prototype Guide
  3. 3
    EvalAI Eval Design
  4. 4
    DeployAI Agent Design
  5. 5
    HardenAI Guardrails Design
  6. 6
    MonitorLLM Observability Plan

For product managers

Market → Adoption → Health → Cost → Governance

  1. 1
    MarketMulti-Model Strategy
  2. 2
    AdoptionAI Adoption Evaluator
  3. 3
    HealthAI Health Check
  4. 4
    CostLLM Observability Plan
  5. 5
    GovernanceAI Maturity (Org)

The skills, by phase

The full AI-building set, grouped by where it sits in the lifecycle. Click any skill for the full process.

01

Spec & plan

Decide why AI, which model, and what good looks like before any code.

AI Product Spec
/ai-product-spec

Spec the feature: why-AI gate, model requirements, prompt architecture, cost, guardrails.

Multi-Model Strategy
/multi-model-strategy

Match models to jobs; route by cost, quality, and latency instead of defaulting to one.

02

Build

Get it working: prototype, the agent loop, and grounding in your own data.

AI Prototype Guide
/ai-prototype-guide

Stand up a working version fast with AI coding tools.

AI Agent Design
/ai-agent-design

Multi-step agents: tool surface, loop and termination, state, failure recovery.

RAG Implementation
/rag-implementation

Ground the model in your documents: RAG vs fine-tune vs long context, chunking, retrieval evals.

03

Evaluate & harden

Define quality, test the system, and defend it against misuse.

AI Eval Design
/ai-eval-design

Rubric, golden dataset, pass bar, and hard-fails. What good means, made measurable.

AI Testing Strategy
/ai-testing-strategy

The testing pyramid, regression suites, red-teaming, and deploy gates.

AI Guardrails Design
/ai-guardrails-design

Layered input validation, output filtering, and abuse detection.

04

Ship & monitor

Watch quality, cost, and risk in production, because none of them throw an error.

LLM Observability Plan
/llm-observability-plan

Quality metrics, logging, cost-reduction playbook, drift detection, A/B in prod.

Agent Eval Harness
/agent-eval-harness

Offline regression suites, trajectory scoring, and production sampling. Agent quality watched live, not just pre-ship.

AI Risk Register
/ai-risk-register

A living register across accuracy, bias, security, cost, and compliance, maintained after launch.

05

Assess & govern

Diagnose AI readiness and set the limits before scale forces them.

AI Adoption Evaluator
/ai-adoption-evaluator

Assess whether and how AI actually gets used.

AI Health Check
/ai-health-check

Run an AI Health Indicator assessment.

AI Maturity Assess
/ai-maturity-assess

Individual AI maturity self-assessment.

AI Maturity (Org)
/ai-maturity-org

Organizational AI maturity assessment.

Related reading

The thinking behind the skills: how to know your AI actually works, and the terms worth getting straight.

How I Think About AI Product Strategy

Start with the job, not the model. The three-layer framework.

What Is an AI Eval?

The plain-English definition, and the three parts every eval is made of.

How to Design AI Evals

A practical guide to building an eval that catches real regressions.

Eval-Driven Development for AI Products

Make evals part of the build loop, not a one-time check.

How to Test for AI Hallucinations

The failure mode evals catch most often, and how to test for it.

Glossary: AI & product terms

Plain-English definitions for evals, AI adoption, and fractional roles.

Looking for something else?

These are the AI-building skills. The full library covers product management, engineering, design, facilitation, and research too.

Browse all skills