Use this when the team has shortlisted 2-4 vendors for a technology component and needs a structured evaluation to make a defensible selection decision. Also use for periodic vendor reviews to assess whether to renew or switch.

Related skills: Use /build-vs-buy when evaluation includes a build option. Use /vendor-security-assessment for security-specific evaluation. Use /decision-brief for final recommendation format. Use /business-case-builder for investment justification.

Process

Step 1: Gather inputs

Collect from the evaluation lead:

• Capability being procured -- specific description, not just a category
• Non-negotiable requirements -- must-haves that disqualify a vendor if unmet
• Nice-to-have requirements -- valued but not dealbreakers
• Timeline -- when does the decision need to be made, when must the solution be in production
• Budget constraints -- annual and total budget available
• Shortlisted vendors -- 2-4 vendors to evaluate (more than 4 makes the process unwieldy)
• Existing relationships -- current contracts, partnerships, or history with any of the vendors

Step 2: Define weighted scorecard

Define criteria categories with weights summing to 100. Adjust weights based on what matters most for this decision:

Ask the user: "Which of these categories matters most for your decision? I'll adjust weights accordingly."

Step 3: Establish scoring methodology

Use a 1-5 scale with clear definitions:

Every score must include evidence -- a number without a reason is just an opinion.

Step 4: Score each vendor

Evaluate each vendor against every criterion using:

• Vendor documentation -- feature lists, architecture docs, compliance certifications
• Demos/POC results -- hands-on evaluation, not just sales demos
• Reference checks -- conversations with existing customers (see Step 7)
• Security questionnaire responses -- reference /vendor-security-assessment
• Pricing proposals -- not list price, actual negotiated pricing

For each criterion, capture:

• Numeric score (1-5)
• Evidence supporting the score
• Notes on gaps or concerns

Step 5: Generate comparison matrix

# Vendor Evaluation -- {{capability_name}}

## Evaluation date: {{date}}
## Evaluation lead: {{name/role}}
## Vendors evaluated: {{vendor_1}}, {{vendor_2}}, {{vendor_3}}

### Must-have requirements check
| Requirement | {{vendor_1}} | {{vendor_2}} | {{vendor_3}} |
|---|---|---|---|
| {{requirement_1}} | Pass/Fail | Pass/Fail | Pass/Fail |
| {{requirement_2}} | Pass/Fail | Pass/Fail | Pass/Fail |

**Disqualified vendors:** {{any vendor failing a must-have}}

### Weighted scorecard
| Criterion | Weight | {{vendor_1}} | {{vendor_2}} | {{vendor_3}} |
|---|---|---|---|---|
| Functionality/fit | {{w}} | {{score}} | {{score}} | {{score}} |
| Scalability/performance | {{w}} | {{score}} | {{score}} | {{score}} |
| Security/compliance | {{w}} | {{score}} | {{score}} | {{score}} |
| Integration/ecosystem | {{w}} | {{score}} | {{score}} | {{score}} |
| Pricing/TCO | {{w}} | {{score}} | {{score}} | {{score}} |
| Vendor viability | {{w}} | {{score}} | {{score}} | {{score}} |
| Support quality | {{w}} | {{score}} | {{score}} | {{score}} |
| Developer experience | {{w}} | {{score}} | {{score}} | {{score}} |
| **Weighted total** | **100** | **{{total}}** | **{{total}}** | **{{total}}** |

### Pricing comparison (3-year TCO)
| Cost component | {{vendor_1}} | {{vendor_2}} | {{vendor_3}} |
|---|---|---|---|
| Year 1 licensing | {{$}} | {{$}} | {{$}} |
| Year 2 licensing | {{$}} | {{$}} | {{$}} |
| Year 3 licensing | {{$}} | {{$}} | {{$}} |
| Implementation | {{$}} | {{$}} | {{$}} |
| Integration maintenance | {{$}} | {{$}} | {{$}} |
| **3-year total** | **{{$}}** | **{{$}}** | **{{$}}** |

### Key differentiators
{{Per-vendor summary of what makes each stand out, positive and negative}}

Step 6: Produce recommendation

## Recommendation

**Selected vendor:** {{vendor_name}}

**Rationale:** {{2-3 sentences on why this vendor wins}}

**Key risks:**
- {{Risk 1 and mitigation}}
- {{Risk 2 and mitigation}}

**Negotiation leverage:**
- {{Where you have alternatives that strengthen your position}}
- {{Pricing benchmarks or competitive pressure points}}

**Recommended contract terms:**
- Contract length: {{X}} years
- SLA requirements: {{key SLAs to include}}
- Exit clause: {{terms for early termination}}
- Price escalation cap: {{maximum annual increase}}
- Data portability: {{export requirements}}

Step 7: Generate reference check questions

Prepare these questions for speaking with each vendor's existing customers:

## Reference check questions -- {{vendor_name}}

1. How long have you been using {{vendor_name}}? What was your implementation timeline?
2. How did the implementation compare to what was promised during the sales process?
3. What's been your experience with their support? Can you give an example of a critical issue?
4. Have you encountered any unexpected costs or pricing changes?
5. How does the product handle {{your_specific_use_case}}?
6. What's the biggest limitation you've discovered?
7. If you were starting over, would you choose them again? Why or why not?
8. What would you negotiate differently in the contract?

Step 8: Design POC/trial plan

If the decision requires hands-on validation before committing:

## POC plan -- {{vendor_name}}

**Scope:** {{what to test, focused on highest-risk areas}}
**Duration:** {{1-4 weeks}}
**Team:** {{who participates}}

**Success criteria:**
- {{Criterion 1 with measurable target}}
- {{Criterion 2 with measurable target}}
- {{Criterion 3 with measurable target}}

**Evaluation rubric:**
- All criteria met = proceed with vendor
- 1-2 criteria missed = negotiate mitigations
- 3+ criteria missed = disqualify

Step 9: Review

Before finalizing, ask:

• Are the weights right for this specific decision?
• Did any vendor score surprisingly high or low? If so, check the evidence.
• Are there political or relationship factors that the scorecard doesn't capture? (Name them explicitly rather than letting them influence scores silently.)
• Does the recommendation need stakeholder buy-in before proceeding?
• Is the evaluation team confident enough to commit, or is a POC needed?

Voice/audio vendor addendum

When evaluating voice or audio AI vendors, add these criteria to the weighted scorecard:

Voice-specific reference check questions (add to Step 7):

• How does the voice agent handle accented or dialectal speech from your users?
• What's the actual end-to-end latency you see in production (not the vendor's benchmark)?
• How did you handle the regulatory disclosure requirement ("you are speaking with an AI")?
• What's your warm handoff experience -- does context transfer to human agents reliably?

Output location

Save to deliverables/vendor-evaluation-{{capability_name}}-{{date}}.md. Reference check notes and POC results can be appended as sections or linked as separate files.