AI MEETS THIRD-PARTY RISK.
Vendor risk. Real pressure. Measured.
Which AI models can do the work—and which fall apart under pressure? We put them through vendor-risk investigations and adversarial tests. You get the evidence.
Built by SecurityScorecard. Grounded in the work of third-party risk analysts.
Capable doesn’t mean safe.
A model can refuse a direct harmful request and still fail when the attack changes shape. A capability score alone won’t tell you that.
Same model. Different pressure. A different answer.
Three independent suites probe prompt injection, wrapped attacks, and direct harmful requests. Read each result on its own terms.
Test an analyst’s judgment.
Exposure triage
– Multiple CVEs in the vendor’s frozen findings
– CISA KEV identifies active exploitation
– EPSS ranks exploitation probability
The agents job
Identify the KEV-listed CVE with the highest EPSS. Grading: Exact CVE match
Prioritize the vulnerability that matters—not the longest list of findings.
Remediation blind spots
– A vendor’s stated remediation plan
– Its issue with the largest absolute score impact
– A comparison of promised work and observed risk
The agents job
Identify the issue type missing from the affected vendor’s plan. Grading: Exact issue-type match
Catch the gap between a reassuring plan and the risk it actually addresses.
Supply-chain concentration
– Technology fingerprints across the portfolio
– Shared components across multiple vendors
– A frozen view of fourth-party dependencies
The agents job
Identify the technology shared by the most vendors. Grading: Exact technology match
Surface the shared dependency that turns a single failure into portfolio-wide exposure.
Spin resistance
– A vendor’s self-attested posture claim
– Independent measured rating-factor evidence
– Conflicting statements in the same investigation
The agents job
Answer from the measured evidence, despite the vendor’s spin. Grading: Typed flag match
Test whether persuasion can override the facts in an agent’s risk decision.
Every number has a backstory.
The result matters. So do the task set, the run settings, and everything we didn’t measure.
A scripted reference solver verifies task solvability. Capability answers are graded against typed flags, not an LLM judge’s opinion.
Benchmark methodology
The current leaderboard is archive-attested. A separate committed substrate lets you reproduce baselines, not these historical scores.
Verification & artifacts
Unmeasured cost, unrun modes, and incomplete safety coverage stay visible. Absence is never a passing score.
Model directoryFor Media Inquiries
