Survey Design

Use this when you need quantitative data from a larger sample -- measuring attitudes, satisfaction, feature preferences, or behavior frequency. Produces a complete survey: objectives, question design, scale selection, bias checks, distribution plan, and analysis approach. Surveys answer "how many" and "how much" -- for "why," use interviews instead.

Related skills: Use /interview-plan for qualitative depth. Use /research-synthesize for post-survey analysis. Pairs with /screener-design if targeting a specific segment. References method selection in practices/user-centered-design/research-ops-method-chooser.md.

Process

Step 1: Gather inputs

Ask the user to provide:

1. Research questions -- what do you need to learn? (e.g., "How satisfied are users with onboarding?" or "Which features do enterprise users value most?")
2. Decisions this informs -- what will change based on the results? (If nothing changes, don't survey.)
3. Target audience -- who should respond? (Segment, sample size target, any quotas.)
4. Distribution channel -- how will you reach respondents? (In-product, email, panel, social.)
5. Existing data -- do you have qualitative findings that the survey should validate at scale? (Surveys are best when grounded in prior qualitative work.)
6. Constraints -- timeline, budget, tool (Typeform, Google Forms, SurveyMonkey, Qualtrics).

Step 2: Define survey objectives and structure

## Survey Design -- (Topic, date)

### Objectives
1. (Primary objective -- the most important thing to measure)
2. (Secondary objective)
3. (Tertiary objective -- nice-to-have)

### Survey structure
- **Estimated completion time:** (Target 3-5 minutes. Maximum 7 minutes.)
- **Total questions:** (Target 8-15. Maximum 20.)
- **Section flow:**
  1. Screening / demographics (2-3 questions)
  2. Core research questions (5-8 questions)
  3. Open-ended follow-up (1-2 questions)
  4. Closing / optional contact info (1 question)

Step 3: Write the questions

### Survey questions

**Section 1: Context (2-3 questions)**

Q1: (Role/segment qualifier)
- Type: Multiple choice
- Options: (List options)
- Purpose: Segmentation

Q2: (Usage frequency or experience level)
- Type: Multiple choice / Scale
- Options: (List options)
- Purpose: Context for interpreting later answers

---

**Section 2: Core questions (5-8 questions)**

Q3: (Satisfaction or attitude question)
- Type: Likert scale (1-5)
- Scale: 1 = Strongly disagree ... 5 = Strongly agree
- Statement: "(Clear, single-concept statement -- e.g., 'I can find what I need in the product quickly')"
- Purpose: (What this measures)

Q4: (Importance/priority ranking)
- Type: Rank order or MaxDiff
- Items: (List 4-6 items to rank)
- Purpose: (What this measures)

Q5: (Behavior frequency)
- Type: Multiple choice
- Options: Never / Rarely / Monthly / Weekly / Daily
- Purpose: (What this measures)

Q6: (Feature preference or satisfaction)
- Type: Matrix / Rating scale
- Items: (List items being rated)
- Scale: (1-5 or similar)
- Purpose: (What this measures)

---

**Section 3: Open-ended (1-2 questions)**

Q_open1: "What is the single biggest improvement we could make to [product/feature]?"
- Type: Free text
- Purpose: Surface themes not covered by closed questions. Provides quotes for readouts.

Q_open2: (Optional) "Is there anything else you'd like to share?"
- Type: Free text
- Purpose: Catch-all for unexpected insights.

---

**Section 4: Closing**

Q_close: "Would you be willing to participate in a follow-up interview? (15-20 minutes, [incentive])"
- Type: Yes/No + optional email field
- Purpose: Recruiting for qualitative follow-up

Step 4: Bias check

Review each question against this checklist:

### Bias check

| # | Question | Check | Pass? |
|---|---|---|---|
| Q3 | (Question text) | Single concept? (Not double-barreled) | Y/N |
| Q3 | | Neutral framing? (Not leading) | Y/N |
| Q3 | | Balanced scale? (Equal positive/negative options) | Y/N |
| Q3 | | Exhaustive options? (Respondent can always find their answer) | Y/N |
| Q3 | | Mutually exclusive? (Options don't overlap) | Y/N |
| Q4 | (Question text) | Same checks | |

Common bias traps to catch:

• Leading questions: "How much do you love our new feature?" → "How would you rate your experience with the new feature?"
• Double-barreled: "How satisfied are you with speed and reliability?" → Split into two questions
• Acquiescence bias: All questions phrased positively → Mix positive and negative statements
• Social desirability: "Do you read the documentation?" → "How often do you refer to documentation?" with "Never" as an option
• Missing options: No "Not applicable" or "I don't know" when these are legitimate answers

Step 5: Distribution and sample plan

### Distribution plan

| Item | Detail |
|---|---|
| Channel | (In-product, email, panel, social) |
| Target sample size | (See calculator below) |
| Expected response rate | (Email: 10-25%, In-product: 5-15%, Panel: 70-90%) |
| Invitations needed | (Sample size / response rate) |
| Timeline | (Launch date, reminder schedule, close date) |
| Incentive | (If any) |

### Sample size guidance
- For directional insights (not statistically rigorous): 50-100 responses
- For segment comparisons (comparing 2-3 groups): 100-200 per segment
- For statistical significance at 95% confidence, ±5% margin: ~385 responses
- For NPS benchmarking: minimum 200 responses

### Reminder schedule
- Day 0: Initial invitation
- Day 3: First reminder (to non-responders)
- Day 7: Final reminder
- Day 10: Close survey

Step 6: Analysis plan

### Analysis plan

| Question | Analysis method | What you're looking for |
|---|---|---|
| Q3 (Likert) | Mean + distribution | Central tendency and spread. Look for bimodal distributions (two camps). |
| Q4 (Ranking) | Frequency of top-3 placement | Which items consistently rank highest? |
| Q5 (Frequency) | Cross-tab by segment (Q1) | Does behavior differ by role/segment? |
| Q6 (Matrix) | Mean per item + segment comparison | Which items rate highest/lowest? Where do segments diverge? |
| Q_open1 | Thematic coding (manual or AI-assisted) | Group responses into themes. Count theme frequency. Pull representative quotes. |

### Reporting
- Use `/research-synthesize` for full synthesis of survey results
- Use `/research-readout` for stakeholder presentation of findings

Step 7: Review and validate

Ask the user:

• Can you complete this survey in under 5 minutes? (Time yourself.)
• Is every question earning its place? (If removing a question wouldn't change a decision, cut it.)
• Are the scales consistent? (Don't mix 5-point and 7-point scales without reason.)
• Will the target audience understand every question without context? (Test with 2-3 people before launching.)
• Is the analysis plan clear? (If you don't know what you'll do with the data, don't collect it.)

Survey design rules

• Every question must tie to a decision. If the answer won't change what you do, don't ask it.
• Shorter is always better. Every additional question reduces completion rates. 10 questions is the sweet spot.
• Closed questions measure, open questions discover. Use closed for quantitative data, open for unexpected insights.
• Scales need anchors. "1-5" means nothing. "1 = Never, 5 = Daily" means something.
• Don't survey when you should interview. If you don't yet know the right questions to ask, do qualitative research first.
• Pilot before launch. Send to 3-5 people and watch them take it. Fix confusion before it reaches 300 people.
• Plan the analysis before writing questions. If you don't know how you'll analyze the data, you're collecting the wrong data.

Output location

Present the survey design as formatted text in the conversation. The user copies questions into their survey tool and adapts formatting. Follow up with /research-synthesize after data collection.