Skip to main content
Assessment & Diagnostics/vendor-security-assessment

Vendor Security Assessment

You need to assess a vendor's security posture before procurement or during periodic review.

Use this when you're evaluating a SaaS vendor, API provider, or third-party integration and need to assess their security posture -- before signing a contract, during a periodic review, or after a vendor has disclosed a security incident.

Related skills: Use /compliance-map for framework knowledge when evaluating vendor certifications. Use /decision-brief for vendor selection decisions. Use /vendor-evaluation for broader vendor evaluation beyond security. Use /build-vs-buy when vendor selection is part of a build-vs-buy decision.

Process

Step 1: Gather context

Collect from the user:

  • Vendor name and what service they provide
  • What data the vendor will access, process, or store
  • Data sensitivity classification -- PII, PHI, financial, proprietary, public
  • Integration points with your systems -- API, SSO, data sync, webhook, file transfer
  • Contract timeline -- new procurement, renewal, or periodic review
  • Any known concerns or prior incidents with this vendor

The depth of assessment should be proportional to risk. A logging SaaS that ingests application metrics gets a lighter review than a payment processor handling credit card data.

Step 2: Generate tailored questionnaire

Based on vendor type and data sensitivity, generate questions across these domains:

For all vendors:

  • Data handling: Where is data stored? Is it encrypted at rest and in transit? What encryption algorithms and key lengths?
  • Access controls: How is access to customer data controlled? Is there role-based access? How are privileged accounts managed?
  • Incident response: What is the vendor's breach notification timeline? Do they have a documented incident response plan?
  • Business continuity: What is the RTO/RPO? Where are backups stored? Has DR been tested?
  • Compliance: What certifications does the vendor hold (SOC 2, ISO 27001, etc.)? When was the last audit? Can they share the report?

For vendors handling sensitive data (add):

  • Subprocessor management: Who are the subprocessors? How are they vetted?
  • Data residency: In which regions is data stored? Can data residency be contractually guaranteed?
  • Vulnerability management: What is the patching cadence? Is there a bug bounty program?
  • Employee security: Background checks? Security training frequency? Least-privilege access enforcement?
  • Data lifecycle: How is data deleted on contract termination? Is deletion certified?

For infrastructure/platform vendors (add):

  • Network security: Network segmentation? DDoS protection? WAF?
  • Logging and monitoring: What audit logs are available to customers? Retention period?
  • Multi-tenancy isolation: How is customer data isolated? Shared or dedicated infrastructure?

For vendors with AI/ML capabilities (add):

  • AI training data: Is customer data used to train or improve models? Can customers opt out? Is training data shared across customers?
  • AI failure transparency: Does the vendor document known failure modes, false positive/negative rates, and confidence thresholds? Can the customer tune sensitivity?
  • Baseline requirements: What telemetry quality, log coverage, and tag consistency does the vendor need from the customer's environment to build accurate AI baselines (e.g., for UEBA or anomaly detection)?
  • AI explainability: When the AI flags an anomaly or makes a recommendation, can it trace back to specific events, identities, and resources -- or is it a black-box score?
  • Model update cadence: How often do detection or analysis models update? Are updates tested against customer-specific baselines before deployment? Can the customer pin a model version?
  • AI-specific incident handling: If the AI produces a false positive that triggers an incident response, what's the vendor's process for tuning and preventing recurrence?

Questionnaire length should be proportional to risk. Don't send 200 questions to a low-risk vendor.

Step 3: Define scoring criteria

Build a weighted evaluation with clear pass/fail thresholds:

| Domain | Weight | Pass/Fail Items | Scored Items |
|---|---|---|---|
| Data Encryption | {{weight}}% | Encryption at rest (AES-256 or equivalent) -- MUST PASS | Key management maturity |
| Access Controls | {{weight}}% | MFA for admin access -- MUST PASS | RBAC granularity, access review cadence |
| Incident Response | {{weight}}% | Documented IR plan -- MUST PASS | Notification timeline, testing frequency |
| Compliance | {{weight}}% | SOC 2 Type II or equivalent -- MUST PASS (for sensitive data) | Audit recency, scope coverage |
| Business Continuity | {{weight}}% | None | RTO/RPO, DR testing frequency |

Pass/fail items are non-negotiable. If a vendor fails any pass/fail item, that's a disqualifier unless compensating controls exist.

Step 4: Score vendor responses

Evaluate each response with a risk-rated assessment:

| Domain | Question | Vendor Response | Rating | Notes |
|---|---|---|---|---|
| {{domain}} | {{question}} | {{response_summary}} | Red / Yellow / Green | {{risk_note}} |

Rating definitions:

  • Red flag -- immediate disqualifier or critical gap. Example: no encryption at rest for PII, no incident response plan, SOC 2 report expired 18+ months ago.
  • Yellow flag -- acceptable with conditions. Example: SOC 2 Type I only (not Type II), patching cadence is quarterly rather than monthly, no dedicated security team but outsourced to reputable firm.
  • Green -- meets or exceeds requirements.

Distinguish between what a vendor claims and what they can evidence. "We encrypt everything" without specifying algorithms, key management, or providing a SOC 2 report is a yellow flag.

Step 5: Generate risk assessment

Produce an overall risk profile:

## Vendor Risk Assessment -- {{vendor_name}} -- {{date}}

**Overall Risk Rating:** Low / Medium / High / Critical

**Red Flags:** {{count}}
{{list_red_flags}}

**Yellow Flags:** {{count}}
{{list_yellow_flags}}

**Residual Risks:**
- {{risk_description}} -- Likelihood: {{L}}, Impact: {{I}}

**Compensating Controls (your organization should implement):**
- {{control_description}} -- addresses {{residual_risk}}

Step 6: Produce recommendation

One of three outcomes:

  1. Approve -- vendor meets security requirements. Proceed with procurement.
  2. Approve with conditions -- vendor is acceptable if specific conditions are met. List each condition, the timeline for the vendor to meet it, and monitoring requirements.
  3. Reject -- vendor has critical gaps that cannot be mitigated. Document the specific disqualifiers.

For conditional approval, define:

  • Reassessment triggers (vendor incident, contract renewal, material change in data handling)
  • Monitoring cadence (quarterly check-ins, annual reassessment)
  • Conditions the vendor must meet and by when

Step 7: Generate contract security clauses

Based on findings, draft security-specific contract terms:

## Recommended Contract Clauses

### Data Processing Agreement
- Vendor shall process data only for the purposes specified in {{agreement}}
- Data residency: {{region_requirements}}

### Breach Notification
- Vendor shall notify Customer within {{hours}} hours of discovering a security incident
- Notification shall include: nature of the breach, data affected, remediation steps taken

### Audit Rights
- Customer may audit vendor's security controls {{frequency}} or upon reasonable notice
- Vendor shall provide updated SOC 2 reports within {{days}} days of issuance

### Security SLAs
- Uptime: {{percentage}}
- Critical vulnerability patching: within {{hours}} hours
- Incident response time: {{hours}} hours

### Data Deletion
- Upon contract termination, vendor shall delete all Customer data within {{days}} days
- Vendor shall provide written certification of deletion

Contract clauses should be specific and enforceable. "Vendor will maintain reasonable security" is not a useful clause.

Step 8: Review

Ask the user:

  • Are there industry-specific requirements we should add (HIPAA BAA, PCI SAQ)?
  • Does legal need to review the contract clauses before they go to the vendor?
  • What's the reassessment cadence -- annual, semi-annual, or triggered by vendor incidents?
  • Are there existing vendor management policies this assessment should align with?
  • Is there a vendor risk register where this assessment should be recorded?

Output location

Deliver as a markdown document. Suggested filename: vendor-assessment-{{vendor-name}}-{{date}}.md. The recommendation section should be extractable for procurement review.

Example Output

Input

  • Vendor name and service: Mosaic Analytics — cloud-based HR analytics and workforce planning SaaS; integrates with Workday and ADP to surface headcount, attrition, and compensation benchmarking dashboards
  • Data accessed: Full employee records including names, job titles, salary bands, performance ratings, demographic data (age, gender, ethnicity), and Social Security Numbers for payroll reconciliation — classified PII/sensitive HR data
  • Integration points: Bidirectional API sync with Workday (nightly), SSO via Okta, webhook callbacks for real-time attrition alerts, file transfer (SFTP) for monthly ADP payroll exports
  • Contract timeline: New procurement; legal has a 45-day window before the planned go-live for Meridian Financial Group's Q3 HR transformation program
  • Known concerns: Mosaic disclosed a data exposure incident 11 months ago involving misconfigured S3 buckets affecting 3 customers; they claim remediation is complete but no third-party validation was shared

Output (abbreviated)

Vendor Security Assessment — Mosaic Analytics — 2025-07-14

Prepared for: Meridian Financial Group — HR Technology Procurement Assessed by: Information Security & Vendor Risk Assessment type: New procurement — high sensitivity data


Executive Summary

Overall Risk Rating: HIGH

Mosaic Analytics handles some of Meridian's most sensitive employee data, including SSNs, compensation, and protected demographic attributes. The vendor's prior S3 misconfiguration incident — unvalidated by a third party — is a material concern in this context. The assessment identifies 3 red flags and 5 yellow flags. Conditional approval is possible if Mosaic meets specific remediation requirements within 30 days; outright approval is not warranted at this time.


Tailored Security Questionnaire

Data Handling

  1. Where is employee data stored, and in which AWS regions? Can data residency be contractually locked to US-East/US-West only?
  2. What encryption algorithm and key length is used for data at rest? Who manages encryption keys — Mosaic, AWS KMS, or a customer-managed key option?
  3. Following the 2024 S3 misconfiguration incident, what technical controls now prevent public bucket exposure? Has this been independently validated?
  4. Is SSN data tokenized or masked in analytics dashboards, or is it stored and displayed in plaintext?

Access Controls

  1. How is Mosaic employee access to customer data controlled? Is access granted per-tenant, and is it logged?
  2. What MFA requirements apply to Mosaic engineers accessing production systems?
  3. How often are privileged access rights reviewed and recertified?

Incident Response

  1. What is your contractual breach notification timeline? Your 2024 incident — how long elapsed between discovery and customer notification?
  2. Can you share your current incident response plan (redacted acceptable)?

Subprocessor Management

  1. Which subprocessors process Meridian data (e.g., analytics engines, email providers, support tooling)? How are they assessed?
  2. Does any subprocessor receive SSN or demographic data?

AI/ML Capabilities

  1. Mosaic's attrition prediction model — is it trained on data pooled across customers, or per-tenant? Can Meridian opt out of contributing training data?
  2. When the model flags an attrition risk, can it trace back to specific data inputs, or is it a composite score only?
  3. How are model updates tested before deployment? Can Meridian pin to a previous model version if an update degrades accuracy?

Data Lifecycle

  1. Upon contract termination, what is the deletion timeline and method? Is written certification of deletion provided?

Scoring Criteria

DomainWeightPass/Fail ItemsScored Items
Data Encryption25%Encryption at rest (AES-256) — MUST PASS; SSN not in plaintext — MUST PASSKey management maturity, customer-managed key option
Access Controls20%MFA for all production access — MUST PASSRBAC granularity, access review cadence
Incident Response20%Documented IR plan — MUST PASS; notification ≤ 72 hrs — MUST PASSNotification SLA, post-incident validation
Compliance20%SOC 2 Type II current (< 12 months) — MUST PASSScope covers HR data workloads, penetration test recency
Subprocessors10%No unvetted subprocessors with SSN accessSubprocessor security evidence
Business Continuity5%NoneRTO/RPO, DR test evidence

Vendor Response Scoring

DomainQuestionVendor ResponseRatingNotes
Data EncryptionEncryption at rest algorithmAES-256 via AWS KMS; key rotation annual🟢 GreenMeets requirement; customer-managed keys available on Enterprise tier
Data EncryptionSSN storage and displaySSNs stored encrypted; displayed in full in payroll reconciliation exports🔴 RedSSNs in plaintext in SFTP exports — no masking or tokenization
Access ControlsMFA for production accessMFA required for all engineers via Okta🟢 GreenEvidenced in SOC 2 report
Access ControlsPrivileged access review cadenceQuarterly access reviews🟡 YellowQuarterly is acceptable; semi-annual would be preferred given incident history
Incident ResponseBreach notification timeline72-hour notification per contract🟡 Yellow2024 incident: 6-day lag between discovery and notification — inconsistent with stated SLA
Incident ResponseIR plan documentationShared redacted IR plan🟢 GreenPlan is documented; tabletop exercise completed Q1 2025
ComplianceSOC 2 Type IISOC 2 Type II issued March 2025; covers core platform🟡 YellowReport does not explicitly cover SFTP/ADP integration pipeline — scope gap
CompliancePost-incident third-party validation"Internal audit confirmed remediation"🔴 RedNo third-party penetration test or configuration audit post-incident
SubprocessorsSubprocessors with SSN accessFivetran (data pipeline), Snowflake (analytics warehouse), Zendesk (support)🟡 YellowZendesk access to support tickets — confirm whether ticket data includes PII
AI/MLAttrition model training dataCross-customer training with anonymization; opt-out not available🔴 RedMeridian employee data contributes to shared model; no opt-out — unacceptable for demographic data
AI/MLModel explainabilityDashboard shows contributing factors (tenure, performance trend, manager score)🟢 GreenFactor-level attribution available; not black-box
Data LifecycleDeletion on termination30-day deletion; certification provided🟢 GreenAcceptable

Risk Assessment

Overall Risk Rating: HIGH

Red Flags: 3

  • SSN plaintext in SFTP exports — Payroll reconciliation files transferred to ADP contain unmasked SSNs with no tokenization. Exposure via compromised SFTP credential would give direct access to ~4,200 employee SSNs.
  • No third-party validation of 2024 S3 incident remediation — Mosaic's self-attestation is insufficient given that the original misconfiguration involved the same infrastructure now proposed for Meridian data.
  • Cross-customer AI training with no opt-out — Meridian's employee demographic data (age, gender, ethnicity) would be used to train models available to Mosaic's full customer base. This creates regulatory exposure under state biometric/demographic data laws and reputational risk.

Yellow Flags: 5

  • SOC 2 scope excludes SFTP/ADP pipeline
  • 2024 breach notification lag (6 days vs. 72-hour stated SLA)
  • Zendesk subprocessor may receive PII in support tickets — unconfirmed
  • Quarterly privileged access review cadence (prefer semi-annual or monthly given incident history)
  • No customer-managed key option on current contract tier (requires Enterprise upgrade)

Residual Risks

RiskLikelihoodImpact