Experiment-Driven Development

Most teams build features based on assumptions they've never tested. Someone important says "users want X," the team builds X, and three months later the data shows nobody uses it. Experiment-driven development is the discipline of testing assumptions before investing significant development time.

The goal is to maximize the odds that what you build will actually drive the outcomes you're chasing - for the business and for the people using the product.

Why experiments matter

Reduce time-to-learning - test assumptions quickly instead of guessing
Avoid over-investing in features that may not meet user needs
Separate problems from solutions - the problem worth solving (discovered through user-centered design) and the solution that works are two different questions
Prioritize based on evidence - build what the data supports, not what the loudest stakeholder wants

The experiment process

1. Identify business objectives

What are you trying to achieve? Expanding a customer segment? Improving retention? Increasing revenue? Every experiment should connect to a business objective - an outcome, not an output.

2. Establish a discovery cadence

Schedule discovery calls with real customers or prospects every week. Consistently document findings with a focus on actionable opportunities.

3. Surface ideas and assumptions worth testing

Use ideation techniques - crazy eights, opportunity scoring, the new venture question list below. The goal is to separate what you know from what you assume.

4. Define hypothesis, test method, and success metrics

Use this template:

We believe that [doing this] for [these people] will achieve [this outcome]. We'll know this is true when we see [qualitative or quantitative signal] that improves [this KPI].

5. Run the experiment

Focus on the smallest, cheapest way to validate. Usually you don't need to build anything in code (see MVP strategy and launch for how to take validated experiments to market):

Paper prototype or Figma prototype
Signup or landing page with engagement metrics
Prove demand by asking for something of value - prepayment, contact info, a commitment
Time-box the activity so it doesn't drag on indefinitely

For validated ideas, link the experiment results to backlog tickets and write stories with clear acceptance criteria. For invalidated ideas, document the learning - it's just as valuable. Killing a bad idea early saves weeks of wasted development.

7. Continuously evaluate released features

Don't assume the solution is "good forever." Continually evaluate released features and prune what's not working. Solutions evolve with the product.

Principles of good experiments

Principle	Why it matters
Test one thing at a time	Isolate the variable so results are interpretable
Define success metrics in advance	Avoid moving the goalposts after seeing results
Smallest, cheapest validation	Build the minimum needed to learn - usually not code
Time-box the activity	Set a deadline so experiments don't drag on
Trace everything to a customer need	If you can't trace a feature back to real user evidence, question whether to build it

The culture that makes experiments work

Process alone doesn't make a team experiment-driven. Two team values do, and without them the process quietly dies:

Share experiments. Run hypothesis-driven experiments and share every outcome - validated or not - so the whole team owns the learning instead of one person holding the context. A result that only the PM sees is a result the team can't act on.
Celebrate failure. A failed experiment is an invalidated hypothesis, not a mistake to bury. The sooner a team produces failures, the sooner it steers off a doomed path. Teams that treat failed experiments as embarrassing stop running the risky ones - which are exactly the ones worth running.

These are two of the four balanced-team values. The best product teams I've worked on discovered great products piece by piece through cheap experiments, and treated the ones that didn't pan out with curiosity rather than blame.

The new venture question list

For any new product or large feature set, answer each question below. Then rate your confidence (rock-solid fact or risky assumption?) and severity (if wrong, does it sink the project?). Sort by low confidence + high severity - those are your most dangerous assumptions. Run experiments on those first.

Question
My target customer will be:
The problem my customers want to solve is:
My customer need can be solved with this product/solution:
The measurable outcome my customer wants to achieve is:
My primary customer acquisition tactic will be:
My earliest adopters will be:
I will make my money by:
My primary competition will be:
I will beat my competitors primarily because of:
My biggest risk to financial viability is:
What other assumption, if wrong, could cause failure:
What other critical risk could cause failure:

How to use this list

Answer each question as best you can
Add a confidence level to each answer (fact vs. assumption)
Add a severity level (if you're wrong, how bad is it?)
Map on a 2x2 - identify the ones that are both low confidence and high severity
Run experiments on those first

Common pitfalls

Running experiments after the decision is made - if leadership has already decided to build it, the experiment is theater
Experiments without clear success criteria - "let's see what happens" isn't an experiment, it's hope
Over-building the test - if your experiment takes more than a week to set up, you're building a feature, not testing an assumption
Ignoring invalidating results - the whole point is to learn. If the experiment says no, listen

Try this today

Pick one feature your team is planning to build in the next quarter. Write a hypothesis using the template above. Then ask: what's the cheapest thing we could do in the next week to test this? If the answer is "nothing - we'd have to build the whole thing," the feature isn't well enough understood yet.

Sources

"Share experiments" and "celebrate failure" are two of the four Pivotal balanced-team values I coached from for four years. The broader case that continuous experimentation is how modern product teams reduce guesswork is well made in The Product Economy (Product School, 2020).