Experiment-Driven Development

Most teams build features based on assumptions they've never tested. Someone important says "users want X," the team builds X, and three months later the data shows nobody uses it. Experiment-driven development is the discipline of testing assumptions before investing significant development time.

The goal: maximize the odds that what you build will actually drive the outcomes you're chasing -- for the business and for the people using the product.

Why experiments matter

• Reduce time-to-learning -- test assumptions quickly instead of guessing
• Avoid over-investing in features that may not meet user needs
• Separate problems from solutions -- the problem worth solving (discovered through user-centered design) and the solution that works are two different questions
• Prioritize based on evidence -- build what the data supports, not what the loudest stakeholder wants

The experiment process

1. Identify business objectives

What are you trying to achieve? Expanding a customer segment? Improving retention? Increasing revenue? Every experiment should connect to a business objective -- an outcome, not an output.

2. Establish a discovery cadence

Schedule discovery calls with real customers or prospects every week. Consistently document findings with a focus on actionable opportunities.

3. Surface ideas and assumptions worth testing

Use ideation techniques -- crazy eights, opportunity scoring, the new venture question list below. The goal is to separate what you know from what you assume.

4. Define hypothesis, test method, and success metrics

Use this template:

We believe that [doing this] for [these people] will achieve [this outcome]. We'll know this is true when we see [qualitative or quantitative signal] that improves [this KPI].

5. Run the experiment

Focus on the smallest, cheapest way to validate. Usually you don't need to build anything in code (see MVP strategy and launch for how to take validated experiments to market):

• Paper prototype or Figma prototype
• Signup or landing page with engagement metrics
• Prove demand by asking for something of value -- prepayment, contact info, a commitment
• Time-box the activity so it doesn't drag on indefinitely

6. Document and share results

For validated ideas, link the experiment results to backlog tickets and write stories with clear acceptance criteria. For invalidated ideas, document the learning -- it's just as valuable. Killing a bad idea early saves weeks of wasted development.

7. Continuously evaluate released features

Don't assume the solution is "good forever." Continually evaluate released features and prune what's not working. Solutions evolve with the product.

Principles of good experiments

Principle	Why it matters
Test one thing at a time	Isolate the variable so results are interpretable
Define success metrics in advance	Avoid moving the goalposts after seeing results
Smallest, cheapest validation	Build the minimum needed to learn -- usually not code
Time-box the activity	Set a deadline so experiments don't drag on
Trace everything to a customer need	If you can't trace a feature back to real user evidence, question whether to build it

The new venture question list

For any new product or large feature set, answer each question below. Then rate your confidence (rock-solid fact or risky assumption?) and severity (if wrong, does it sink the project?). Sort by low confidence + high severity -- those are your most dangerous assumptions. Run experiments on those first.

Question
My target customer will be:
The problem my customers want to solve is:
My customer need can be solved with this product/solution:
The measurable outcome my customer wants to achieve is:
My primary customer acquisition tactic will be:
My earliest adopters will be:
I will make my money by:
My primary competition will be:
I will beat my competitors primarily because of:
My biggest risk to financial viability is:
What other assumption, if wrong, could cause failure:
What other critical risk could cause failure:

How to use this list

1. Answer each question as best you can
2. Add a confidence level to each answer (fact vs. assumption)
3. Add a severity level (if you're wrong, how bad is it?)
4. Map on a 2x2 -- identify the ones that are both low confidence and high severity
5. Run experiments on those first

Common pitfalls

• Running experiments after the decision is made -- if leadership has already decided to build it, the experiment is theater
• Experiments without clear success criteria -- "let's see what happens" isn't an experiment, it's hope
• Over-building the test -- if your experiment takes more than a week to set up, you're building a feature, not testing an assumption
• Ignoring invalidating results -- the whole point is to learn. If the experiment says no, listen

Try this today

Pick one feature your team is planning to build in the next quarter. Write a hypothesis using the template above. Then ask: what's the cheapest thing we could do in the next week to test this? If the answer is "nothing -- we'd have to build the whole thing," the feature isn't well enough understood yet.