Security Questionnaire Copilot Stack for SaaS Teams

Vendor security questionnaires are the kind of work AI should do, but only if the system is built like a review lane rather than a chat assistant. The job is not to be clever. It is to take an inbound form, spreadsheet, or portal export, sanitize it, map each question to approved evidence, draft a cited answer, and stop before anything leaves the building without review.

That makes this a good micro-niche for a production AI stack. The task is repetitive enough to automate, risky enough to require controls, and specific enough that you can define a real budget, a real privacy boundary, and a real acceptance test. If you want the broader control-plane thinking behind the component choices, Structured Outputs for Reliable AI APIs and File Uploads: Threat Model and Controls are the two most useful companions.

Recommended stack

Recommended stack - questionnaire response lane

Interface: Responses API for the draft and review loop. Use the Agents SDK only if you later need multi-specialist handoffs, resumable approvals, or a managed agent loop.

Model roles: a cheap extractor/redactor pass for question classification and PII cleanup; a stronger draft model for cited answers; a final review pass that can refuse or escalate.

Skills: openai-docs for current API behavior and supabase-postgres-best-practices for schema, RLS, and audit-log design.

Plugins: Slack for approval queues, Google Drive or SharePoint for approved policy docs, Notion only if the answer library already lives there.

MCP/tool categories: read-only docs search, read-only Postgres, file quarantine and OCR, Slack approval queue, browser QA for the reviewer UI.

Memory pattern: store policy hashes, answer IDs, reviewer decisions, trace IDs, and approved snippets. Do not store raw questionnaires or attachments in durable memory.

Approval rule: the model may draft; only a human may approve, edit, or send.

The reason this stack works is simple: the most expensive failure is a confident wrong answer that reaches a customer, a prospect, or a procurement portal. The most dangerous failure is a model seeing more private data than it needs. The stack should therefore be conservative by default and narrow at every boundary.

Why this niche is worth solving

A security questionnaire sits in the awkward middle between sales, security, legal, and engineering. It is not a pure support problem, because the answer needs evidence. It is not a pure compliance problem, because the question set changes. It is not a pure retrieval problem, because the same answer can be right for one customer and wrong for another jurisdiction or contract path.

The persona here is a senior AI or full-stack engineer at a B2B SaaS company that receives enough questionnaires to hurt, but not enough to justify a separate compliance automation platform. Typical constraints look like this:

• Risk level: high. A bad answer can create contractual exposure, erode trust, or send a prospect down a bad procurement path.
• Budget: moderate. The business can afford a few model passes and one human review, but not a long free-form reasoning loop over every question.
• Privacy boundary: strict. Raw questionnaires may contain internal architecture details, employee names, data-processing terms, incident references, and security exceptions. Those do not belong in durable memory.
• Success criterion: reduce turn-around time without lowering answer quality or weakening approval discipline.

This is also why the workflow should be treated as a document pipeline, not a conversational UI. The question arrives as a file or export. The system should convert it into a normalized record, retrieve approved evidence, draft a response, and then wait.

Layer 1: choose the stack around the failure

The right model choice is not "which model knows security best." It is "which model does the cheapest correct job at each stage."

The OpenAI docs are clear on the interface choice. Use the Responses API when you want to own the loop. Use the Agents SDK when you want the SDK to manage repeated tool calls, orchestration, and resumable approvals. For this workflow, direct control is the better default because the approval line is part of the product, not an implementation detail.

The same docs also make the output contract clear: Structured Outputs should be used when the model needs to return schema-adherent data. That matters here because every question should become a stable object with fields such as question ID, answer draft, confidence, evidence references, and escalation flags. A free-form paragraph is not enough.

Layer 2: connect only approved evidence

The evidence layer is where most questionnaire stacks go wrong. Teams either dump everything into one giant prompt or they build a retrieval system that has no idea which documents are approved.

The safer pattern is to split evidence into three classes:

1. Approved policy docs - security pages, privacy statements, architecture overviews, incident-response summaries, and DPA-approved language.
2. Jurisdiction or customer-specific overrides - contract exceptions, regional clauses, and explicit redlines that only apply to a specific buyer.
3. Working notes - temporary analysis, draft answers, and reviewer comments that should not become canonical evidence until a human promotes them.

Store the first two classes in a read-optimized, versioned store. If your docs already live in Google Drive, SharePoint, or Notion, use the relevant plugin or connector to read them, but do not let the model write back directly. If you want a dedicated review database, mirror only the approved subset into Supabase and enforce RLS on every table.

Supabase's current RLS guidance is the right baseline: enable row-level security on exposed tables, grant only the minimal Postgres roles, and treat the service role as an administrative escape hatch, not a browser credential. For questionnaire data, the rules should be even tighter. The reviewer UI can read a case only if it belongs to the right org, and the worker can write only to the case it created.

Courtesy: OpenAI. Source: Structured Outputs docs. Captured July 29, 2026 UTC.

That screenshot is useful because the whole stack depends on a reliable contract. When the model is asked to emit JSON, the answer should be parseable without retry gymnastics. OpenAI documents Structured Outputs as a way to make the response adhere to a JSON Schema, and it explicitly recommends it over JSON mode when possible.

File intake is a security problem first

Questionnaires often arrive as PDF uploads, DOCX files, spreadsheets, or portal exports. Treat them like any other untrusted file. The OWASP File Upload Cheat Sheet says not to trust Content-Type, to validate file signatures, to rename files, to enforce size limits, and to scan or sandbox the content when possible. That advice matters here because a questionnaire is not just text; it is a delivery mechanism for arbitrary bytes.

The practical intake path is:

• accept the file into quarantine storage;
• record a hash, size, MIME hint, and uploader identity;
• run OCR or parsing in a disposable worker;
• extract the text into a sanitized canonical format;
• reject or flag anything that fails validation;
• only then feed the redacted text into the model lane.

The result is a cleaner prompt and a safer audit trail. It also means the model never has to parse a malicious attachment directly.

Layer 3: shape the context and steer the agent

Once the evidence boundary is clean, the rest is context engineering.

The stable prefix should contain:

• the role instruction;
• the answer policy;
• citation rules;
• escalation rules;
• the output schema;
• a few approved examples.

The variable suffix should contain only the current questionnaire section, the redacted question text, and the evidence snippets returned by retrieval.

That structure is what makes prompt caching useful. OpenAI says caching is automatic for eligible prompts of 1,024 tokens or more, and GPT-5.6 family models use a 1.25x cache-write rate for writes. The operational trick is to keep the static policy pack and examples at the front, then append the question-specific material at the end. If you reuse the same prompt_cache_key for the same policy template, repeated runs become cheaper and faster without changing the answer contract.

For this workflow, a good context boundary looks like this:

1system prefix
2  - mission
3  - risk policy
4  - escalation rules
5  - answer schema
6  - one or two short examples
7
8retrieval context
9  - approved policy snippets
10  - evidence hashes
11  - jurisdiction note
12
13current question
14  - redacted question text
15  - question id
16  - desired answer format

A schema beats a paragraph

The output should be a JSON object, not an essay. A useful schema for one question might include:

• question_id
• short_answer
• detailed_answer
• confidence
• evidence_refs
• risk_flags
• needs_human_review
• review_reason

Here is the shape in TypeScript:

1import OpenAI from "openai";
2import { z } from "zod";
3import { zodTextFormat } from "openai/helpers/zod";
4
5const QuestionnaireAnswer = z.object({
6  question_id: z.string(),
7  short_answer: z.string(),
8  detailed_answer: z.string(),
9  confidence: z.number().min(0).max(1),
10  evidence_refs: z.array(z.string()).max(8),
11  risk_flags: z.array(z.string()).max(6),
12  needs_human_review: z.boolean(),
13  review_reason: z.string(),
14});
15
16export async function draftAnswer(questionText: string, evidencePack: string, policyVersion: string) {
17  const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
18
19  return client.responses.parse({
20    model: "gpt-5.6",
21    input: [
22      {
23        role: "system",
24        content:
25          "You draft answers to vendor security questionnaires. Use only approved evidence. If evidence is missing, mark needs_human_review true.",
26      },
27      {
28        role: "user",
29        content: `Evidence pack:\n${evidencePack}\n\nQuestion:\n${questionText}`,
30      },
31    ],
32    text: {
33      format: zodTextFormat(QuestionnaireAnswer, "questionnaire_answer"),
34    },
35    prompt_cache_key: `questionnaire-policy:${policyVersion}`,
36    reasoning: { effort: "low" },
37    max_output_tokens: 700,
38  });
39}

The important part is not the SDK syntax. It is the behavior contract:

• the model sees only approved evidence;
• the answer must match the schema;
• the cache key ties repeated policy packs together;
• the output is short enough for review, not long enough to become a new source of confusion.

Tools, plugins, and MCP boundaries

The stack should use tools only where they reduce human error:

• Slack plugin or Slack MCP: send a compact review request with the draft, evidence refs, and decision buttons.
• Google Drive or SharePoint plugin: read the approved policy corpus if that is already where your organization stores it.
• Notion plugin: only if the approved answer library is already curated there.
• Read-only Postgres MCP: read questionnaire cases, approved snippets, and audit logs.
• Browser MCP or Playwright: verify the reviewer UI, keyboard flow, and approval steps.
• Repository/CI MCP: store prompt versions, schema versions, and regression tests.

Do not give the model a tool that can send the final answer directly to a customer mailbox, a procurement portal, or a CRM record. The approval step should live outside the model loop so that "send" is always a human choice.

The original architecture

The diagram below shows the lane I would actually ship. The key design choice is that raw documents never become durable context. They are quarantined, normalized, and redacted before they reach the answer model.

Original diagram by Essa Mamdani. It is a conceptual workflow based on OpenAI Structured Outputs, OpenAI Prompt Caching, Supabase RLS, and OWASP file-upload guidance.

Build it step by step

The cleanest implementation is small and boring. A good initial file structure looks like this:

1src/
2  app/
3    questionnaire/
4      page.tsx
5    api/
6      questionnaire/
7        ingest/route.ts
8        draft/route.ts
9        approve/route.ts
10  lib/
11    questionnaire/
12      schema.ts
13      redact.ts
14      retrieval.ts
15      prompts.ts
16      policy-cache.ts
17      safety.ts
18supabase/
19  migrations/
20    20260729_questionnaire_cases.sql
21    20260729_questionnaire_evidence.sql
22public/
23  images/article-visuals/security-questionnaire-copilot-stack/
24    openai-structured-outputs-docs.png
25    architecture.svg

The operating workflow should be:

1. Ingest the questionnaire. Accept the upload or email attachment into quarantine storage and create a case record.
2. Parse and redact. OCR or extract the document, normalize the question order, and remove anything not needed for drafting.
3. Classify each question. Mark simple yes/no answers, evidence-heavy answers, legal-sensitive answers, and anything that needs a manual check.
4. Retrieve approved evidence. Pull only the snippets that match the question class, product tier, jurisdiction, and customer exception state.
5. Draft the answer. Return structured JSON with a short answer, a detailed answer, evidence refs, and a review flag.
6. Run a second pass on the risky items. If the answer concerns encryption, backups, incident response, sub-processors, retention, or exceptions, route it to the stronger reviewer pass.
7. Show the review packet. The reviewer sees the question, the draft, the evidence, and the risk flags in one screen.
8. Require a human decision. Approve, edit, or escalate. No automated send.
9. Log the decision. Write the policy version, reviewer identity, answer hash, and trace IDs to the audit table.
10. Publish the final answer. Only the signed record can trigger an outbound response or portal update.

Supabase schema and RLS

Supabase is a good fit for the case ledger because the data is small, audit-heavy, and access-controlled. But the tables should be deliberately narrow. Keep the uploaded file metadata, the redacted question set, the evidence links, and the review decision separate.

1create table questionnaire_cases (
2  id uuid primary key default gen_random_uuid(),
3  org_id uuid not null,
4  source text not null,
5  file_hash text not null,
6  status text not null default 'ingested',
7  policy_version text not null,
8  created_at timestamptz not null default now(),
9  updated_at timestamptz not null default now()
10);
11
12create table questionnaire_answers (
13  id uuid primary key default gen_random_uuid(),
14  case_id uuid not null references questionnaire_cases(id) on delete cascade,
15  question_id text not null,
16  answer jsonb not null,
17  reviewer_id uuid,
18  approved_at timestamptz,
19  created_at timestamptz not null default now()
20);
21
22alter table questionnaire_cases enable row level security;
23alter table questionnaire_answers enable row level security;
24
25create policy "org members can read questionnaire cases"
26on questionnaire_cases
27for select
28to authenticated
29using (org_id = (select auth.jwt() ->> 'org_id')::uuid);
30
31create policy "system worker can insert questionnaire cases"
32on questionnaire_cases
33for insert
34to service_role
35with check (true);

That schema is intentionally modest. If you need more, add it later. The danger with a questionnaire product is turning a useful lane into a second CRM.

Why prompt caching matters here

Questionnaires are repetitive. The policy pack, approved boilerplate, legal disclaimers, and evidence rules change much less often than the incoming questions. That makes the workflow a natural fit for prompt caching.

Use a stable cache key per policy bundle, for example:

• one key for the general SaaS security pack;
• one key for enterprise overrides;
• one key for regional or jurisdiction-specific exceptions;
• one key for internal-only reviews.

Keep the common prefix above the 1,024-token threshold so it actually qualifies for caching. If the policy pack is short, add only the approved examples and instructions needed to make the prefix stable. Do not try to force caching with junk text. That only inflates the prompt without making the workflow safer.

What good review looks like

The reviewer should not be looking at a raw prompt dump. The review screen should show:

• the original question;
• the redacted question text;
• the draft answer;
• evidence snippets;
• any jurisdiction-specific note;
• the risk flags;
• the confidence score;
• the recommended action.

The reviewer does not need to see internal chain-of-thought. They need a concise packet they can approve or reject quickly. If the answer is unsupported, the system should say so plainly and route to human drafting instead of manufacturing certainty.

This is where a supporting observability layer helps. OpenTelemetry GenAI Observability: A Production Guide is the right companion if you want to trace where time is spent: upload quarantine, OCR, retrieval, model call, review wait, and outbound send. Without those spans, you will not know whether the slowdown is the model, the reviewer, or the file parser.

Evaluation and failure modes

If the system cannot be measured, it cannot be trusted. Build a regression set with real-but-redacted questionnaire examples and expected outcomes. The questions should cover:

• encryption at rest;
• key management;
• access control;
• backups and recovery;
• incident response;
• vulnerability management;
• data retention;
• subprocessors;
• SSO and MFA;
• regional hosting;
• custom exceptions.

The core metrics are:

• citation coverage: does every non-trivial claim point back to approved evidence?
• unsupported-claim rate: how often does the draft say something the evidence does not support?
• needs-review precision: how often does the system flag a question that actually needs human review?
• answer latency: how long from ingest to draft and from draft to approved send?
• revision rate: how often does the human reviewer need to edit more than a few words?
• rejection rate: how often is the answer too risky to send at all?

The main failure modes are predictable:

If you want a broader evaluation discipline for retrieval systems, RAG Eval Gates for TypeScript Support Agents is the closest pattern match, even though the domain here is different.

Safety, privacy, and human approval

This stack should be conservative by design:

• Never auto-send. Drafting is allowed; sending is not.
• Never store raw questionnaires in long-term memory. Keep only decisions, hashes, and approved snippets.
• Never use production secrets in prompts. If a ticket includes a token or private link, redact it before any model call.
• Never give the model direct write access to the outbound channel. The approval action should be separate from the draft action.
• Never trust the portal export or upload metadata. Validate the bytes, not just the labels.
• Never skip the reviewer on a new policy version. Any policy change can invalidate old answers.

The file and container side of the stack should also be hardened. Secure AI Containers with SBOMs and Provenance is the useful companion if your questionnaire worker runs in Docker or Kubernetes. The same principle applies here: prove what was built, what was scanned, and what was deployed.

Alternatives worth considering

There are three sane variants of this stack.

1. Responses API plus custom orchestration

Best when you want explicit control over routing, approval, caching, and retries. This is the default I would choose for most teams. It is also the easiest to reason about during incident review because the application owns the loop.

2. Agents SDK with handoffs

Best when legal, security, and sales all need their own specialist behavior and separate tool sets. The SDK can own more of the orchestration, but it also makes the runtime more opinionated. Use it when the process has outgrown a single bounded loop.

3. SQL-first retrieval plus a small vector layer

Best when your approved answer bank is short, curated, and mostly exact-match. In many questionnaire workflows, full-text search plus tags and jurisdiction filters are better than a heavy vector database. Use vectors only where semantic fallback actually improves recall.

If you compare those options honestly, the default winner is usually "simple retrieval, strict schema, human approval." It is boring, but boring is what you want when the product deals with evidence and risk.

FAQ

Is this a chatbot or a workflow?

It should be a workflow. The user interface can feel conversational, but the underlying system should behave like an intake, retrieval, drafting, and approval pipeline.

Should I use the model to answer every question end to end?

No. Use the model for classification, synthesis, and drafting. Use human review for anything that changes legal scope, customer commitments, or security exceptions.

Do I need a vector database?

Not necessarily. If the answer library is curated and the documents are short, Supabase tables plus full-text search and metadata filters may be enough. Add vectors only if recall quality actually improves on your test set.

Where does prompt caching help most?

It helps most on the repeated policy prefix: the stable rules, examples, and answer format. It does not help if you keep rewriting the policy pack or mixing too much variable content into the cached section.

Can the model send the final response automatically?

Not in the version I would ship. A human should still own the last approval, because the business risk is in the outbound commitment, not the draft.

Sources

• OpenAI Structured Outputs
• OpenAI Prompt Caching
• OpenAI Agents SDK
• Supabase Row Level Security
• Supabase Storage access control
• OWASP File Upload Cheat Sheet