SaaS · third-party-risk, security-review, procurement, compliance
Vendor security questionnaires: the questions that change the answer
A vendor review is not a longer questionnaire. These are the identity, data, assurance and contract questions where the reply decides the outcome, what the evidence costs, and how to finish the review in weeks.
SaaS
Key points
- Most of a 300-question vendor questionnaire changes nothing; the questions that matter are the ones whose answer would stop the signature or force a clause.
- Ask who at the vendor can impersonate your users, who approves that session, and whether it appears in an audit log you can read.
- Customer-managed keys are not free: AWS KMS charges $1 per key a month plus $0.03 per 10,000 requests, so 1,000 tenant keys cost $1,000 a month before any traffic.
The long questionnaire is not the review
The standard questionnaire is built to produce a document, and it does that well. It cannot tell a vendor that revokes access in a second from one that emails an engineer. The same 300 questions go to a marketing tool and a payroll processor, and both come back marked Compliant.
The questions worth asking share one property: a bad answer changes what happens next. It blocks the purchase, changes the tier, or forces a clause into the contract. Everything else is context the assurance report already holds.
Tier the questions by what the vendor will hold. Anything touching personal data or production systems gets the full set; a tool that handles only public content gets five questions and a signature. The calendar, not the questionnaire, is usually the bottleneck.
| The question that decides it | The evidence to request | What a thin answer costs | |
|---|---|---|---|
| Identity | Who can act as your users, and does the session reach your log | SCIM documentation and a sample impersonation audit event | An offboarded employee keeps a live session; support access is unprovable in an incident |
| Data | Where the data sits, who holds the keys, whether your tenant is shared | Region list, key-management documentation, isolation architecture note | Regulatory exposure in one jurisdiction; a breach you cannot scope to your tenant |
| Assurance | Which report, for which product, for which months | The report itself, the pen-test summary, the last DR test result | You rely on a report that never covered the service you bought |
| Contract | How fast the vendor tells you, pays you, and deletes your data | SLA credit table, subprocessor list, deletion and export terms | A breach you learn about from the news; credits too small to matter; an exit with no export |
| Process | Whether the answer is asserted or evidenced | Named artefacts with dates and owners | A clean spreadsheet, an unassessed vendor, and the same work again next renewal |
Identity: who can get in, and as whom
Start with SSO over SAML and SCIM provisioning, and whether SCIM is on by default or sold as an enterprise tier. The test is not the protocol list. It is what happens the hour someone leaves: access should die when their directory account is disabled, not when someone remembers to email the vendor.
Then apply the same discipline to the vendor's own staff. Whether support and contractors must use multi-factor authentication, and whether the policy has exceptions, states the risk of someone logging in as them. An account protected by a password alone is a path into your data that no setting on your side closes.
Support impersonation is the least-documented answer in most reviews. Ask three things: can vendor staff log in as your users, who approves the session, and does it reach an audit log. A yes to the first with no answer to the other two belongs in the contract rather than a spreadsheet cell.
Advertisement
Data: residency, keys and the bill for BYOK
Residency is settled by a region list, not by a word like sovereign. Ask which regions the service runs in, which one your tenant can be pinned to, and where backups and replicas are written. A vendor with no published region list has an unverified claim.
On encryption, ask for the algorithm, the standard and the custody. AES-256 at rest and TLS 1.2 or later in transit are the floor. If the vendor holds the key, their operator can decrypt your data; customer-managed keys move part of that control to you, and they are a paid product, not a checkbox.
AWS publishes the shape of that bill. A KMS key costs $1 a month, prorated hourly, and symmetric requests cost $0.03 per 10,000 after a 20,000-request free tier; rotation adds $1 a month for the first and second rotation, capped there. One key per tenant therefore costs $1,000 a month at 1,000 tenants before a single encrypt call, and 10 million requests add $30. That is the number to put beside customer-managed keys when a vendor quotes it as a feature.
Assurance: the report, not the badge
Ask which SOC 2 report, for which months. The badge on the trust page proves little: a Type I report speaks to control design at a point in time, a Type II report to how those controls operated across a stated period. Read the scope section to check the system described is the product you buy, then the period, then the exceptions.
ISO 27001 carries the same trap: ask for the certificate and the statement of applicability, because certification covers the scope written on it. For penetration testing, ask for cadence, tester and sharing, and request the summary letter and remediation status when the full report is withheld.
For vulnerability handling, ask whether there is a published disclosure policy and a security contact, and what the remediation timelines are for critical findings. Then disaster recovery: when the test last ran, and what recovery time and recovery point were recorded. The plan is a goal; the exercise is the measurement.
Advertisement
Contract: credits are not compensation
Four terms decide what a bad day costs. Breach notification: the window in which the vendor must tell you, because the regulatory clock is yours. Uptime SLA: how availability is defined and how credits are calculated. AWS publishes a useful benchmark: its EC2 region-level SLA is 99.99 percent, with credits of 10 percent below that, 30 percent below 99 percent and 100 percent below 95 percent of the monthly bill, while the instance-level SLA is 99.5 percent.
Price the credit before you accept it as the remedy. Ten percent of one month is not compensation for a week of downtime in a system that runs payroll, and the AWS table is generous by SaaS standards. Insurance is the second term: ask for the limits on cyber liability and technology errors and omissions cover, and check the certificate names the entity that signed your contract.
The exit terms are the ones nobody reads until they matter. Where is the subprocessor list, how are changes announced, and can you object. Deletion: the timeline after termination, and whether the vendor certifies it in writing. Export: whether your data leaves in an open format such as CSV or JSON.
The decision rule, and the three objections
The first objection is that the vendor already sent a completed questionnaire. A completed form is an assertion; the artefacts behind it are the review. Ask for the report, the certificate, the pen-test summary, the subprocessor page, the SLA table and the last DR result, and date each one.
The second is that the trust page shows a badge. Read the scope and the period before the badge. The third is that legal will not accept a new clause: do not argue it and do not accept it silently. Record the gap as an open item with an owner and a date, and if no artefact arrives, move the risk into the contract as a notification, logging or audit obligation.
Then choose. Run the full review when the vendor holds personal data or production credentials, and a short form plus a report link when it holds public content. Choose the vendor whose evidence covers the service and the months you are buying, and price the gap you cannot close.
Advertisement
Sources 5
