Epilepsy Trial Evidence Extraction Stack

Recommended stack

Recommended stack Build a read-only evidence intake loop with a fast query planner, a schema-locked extractor, a second-model reviewer, and a human sign-off step.

• Skill: pubmed-literature-search
• Sources: PubMed E-utilities, ClinicalTrials.gov API v2, OpenAlex
• Storage: Postgres or Supabase with one row per study and one row per review run
• Contract: Structured Outputs with strict: true
• Boundary: public literature only, no PHI, no autonomous clinical decisions
• Budget: cap query rewrites, cache the prompt prefix, and send only disagreement cases to the reviewer

The right output for this kind of work is not a long summary. It is a traceable evidence table that a clinical research lead can trust, audit, and reuse. A rare-disease team may be trying to answer a question like, "Which recent studies mention seizure frequency outcomes for a specific epilepsy subtype, and which trials are still recruiting?" That is a narrow job with high stakes, limited time, and a hard privacy boundary.

This guide is the medical/research slot in the series rotation. Most of the recent posts on the site are about model releases, agent infrastructure, observability, or database safety. Useful, but not enough if you also need a real research pipeline for a clinical team. The gap here is a public-literature workflow that is specific enough to be useful and constrained enough to be safe.

The intended reader is a senior AI/full-stack engineer at a biotech, med-tech, clinical data, or research tooling company. The budget is moderate. The risk tolerance is low. The system should be able to search public sources, extract structured fields, and stop before it drifts into diagnosis, medical advice, or PHI handling.

Stack selection

The core design is simple: use deterministic sources for retrieval, a strict schema for extraction, and a human for the final inclusion decision.

The most important decision is not the model. It is the contract boundary. In a workflow like this, the model should never be the only thing standing between a source document and a published table. It should only propose fields that your application can verify.

Why these sources

PubMed via the NCBI E-utilities is the backbone because it gives you stable biomedical search and retrieval. The NCBI Bookshelf help page describes E-utilities as the structured interface into Entrez, with fixed URL syntax for search and retrieval. That matters because a literature workflow needs repeatability more than it needs a clever prompt.

ClinicalTrials.gov is the registry layer. It catches trial records earlier than publications and lets you compare registry fields with later papers. The v2 API is the modern REST path, and the official docs ship an OpenAPI spec, which makes client generation and validation much easier than hand-rolled HTML scraping.

OpenAlex is the enrichment layer. It helps you resolve DOIs, look at related works, and expand from one citation to the graph around it. That is especially helpful when a PubMed abstract is thin but the citation network is rich.

The three planes

1) Search plane

The search plane should answer one question only: "What should the extractor see?"

For a narrow epilepsy question, I would start with a MeSH-first PubMed query and only then widen it with title and abstract terms. The goal is to reduce noise before the model ever sees it. The pubmed-literature-search skill is useful here because it explicitly pushes boolean search, MeSH expansion, author and journal filters, article-type filters, and date ranges.

Use public queries as code, not as ad hoc chat prompts. A stored query can be reviewed, diffed, and rerun.

1curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=epilepsy%5BMeSH%20Terms%5D%20AND%20(randomized%20controlled%20trial%5BPublication%20Type%5D%20OR%20trial%5BTitle%2FAbstract%5D)&retmode=json&retmax=25&usehistory=y"

That query is intentionally boring. It is also auditable. If a clinician asks why a study was included or excluded, you can point to the exact search rule that produced the candidate pool.

2) Extraction plane

Once the candidate set exists, the extractor should fill a schema and nothing else. OpenAI's Structured Outputs docs are the right pattern here because they make schema adherence the primary contract rather than an afterthought. The docs also recommend strict: true for schema-constrained output, and they explicitly distinguish between structured responses and tool calls.

The extractor should return one StudyRecord at a time.

1import { z } from "zod";
2
3const StudyRecord = z.object({
4  source: z.enum(["pubmed", "clinicaltrials", "openalex"]),
5  source_id: z.string(),
6  title: z.string(),
7  year: z.number().int().nullable(),
8  pmid: z.string().nullable(),
9  doi: z.string().nullable(),
10  nct_id: z.string().nullable(),
11  study_design: z.enum(["rct", "cohort", "case_report", "review", "registry", "other"]),
12  population: z.string().nullable(),
13  intervention: z.string().nullable(),
14  comparator: z.string().nullable(),
15  outcomes: z.array(z.string()),
16  inclusion_decision: z.enum(["include", "exclude", "maybe"]),
17  exclusion_reason: z.enum([
18    "wrong_population",
19    "wrong_intervention",
20    "no_epilepsy_outcome",
21    "duplicate",
22    "protocol_only",
23    "other",
24  ]).nullable(),
25  evidence_notes: z.string(),
26});

The shape above is not arbitrary. It forces the model to separate facts from judgment. If the study is not relevant, the model still needs to say why. If the PMID is missing, the model still has to say so. If a registry entry and a paper disagree, that disagreement becomes data instead of hidden ambiguity.

If you are using OpenAI, the current docs show how to parse a schema directly into typed objects with responses.parse. The same structural idea works with any vendor that supports JSON Schema or a comparable strict-output contract.

3) Review plane

Do not let the extractor be the final arbiter. For borderline abstracts, the next pass should be either a second model with a different prompt shape or a human reviewer who can read the evidence in context.

This is where the stack becomes operational instead of aspirational. If the first model says "maybe" or the confidence is low, send the row to review. If the paper is a protocol, a conference abstract, or a registry duplicate, treat it as a special case. The system should be able to preserve uncertainty without pretending to resolve it.

The reviewer should see the source text, the extracted fields, the search query that produced the candidate, and the exact exclusion reason options. Nothing else.

Sources you should wire first

PubMed and the NCBI docs

Courtesy: NCBI. Source: https://www.ncbi.nlm.nih.gov/books/NBK25501/. Screenshot captured on 2026-07-28.

That screenshot is the right visual anchor because it shows the official E-utilities help page, not a third-party tutorial. The page itself frames E-utilities as the structured interface into Entrez, which is exactly the mental model this stack needs.

Architecture diagram

Original diagram by Essa Mamdani. It is a conceptual workflow for public literature and registry evidence only.

This flow is intentionally narrow. Public sources enter on the left, a normalizer deduplicates and canonizes IDs in the middle, and the system does not publish anything until a reviewer clears it on the right. That is the right shape for a regulated research assistant.

Ecosystem integration

This is where the stack becomes a system instead of a one-off prompt.

The pubmed-literature-search skill should own query building and source retrieval. It is a good fit because it already encodes medical search habits that most general agent stacks miss: MeSH terms, boolean logic, filters, citation metadata, and full-text access links where available. You do not need to reinvent that logic every time a clinician changes the question.

For external tools, think in MCP categories rather than brands:

I would not make Notion or Google Drive part of the core path. They can sit outside the boundary if your protocol or screening template already lives there, but they should not become the system of record. If you need a plugin at all, treat it as a convenience layer, not a source of truth.

For memory, do not keep the whole conversation. Keep a small run record:

That memory pattern pairs well with the state discipline from Structured Outputs for Reliable AI APIs and the regression approach from RAG Eval Gates for TypeScript Support Agents.

Context engineering

The model should see the smallest context that can still answer the question.

For the planner, that means the research question, the inclusion criteria, a list of approved source types, and the latest saved query. For the extractor, that means one abstract or one registry record plus the schema. For the reviewer, that means the source snippet, the parsed fields, and the exact reason codes.

Do not ask the model to "analyze the literature." Ask it to fill fields.

That distinction matters because the broad instruction invites the model to invent its own method. The narrow instruction forces the model to operate inside your protocol. This is the same logic behind Prompt Caching for Real-World LLM Apps: keep the stable prefix stable, and only vary the part of the prompt that actually needs to change.

Practical budget controls

The budget should be small enough that a bad run is annoying, not catastrophic.

Those controls are not about being cheap for its own sake. They are about making the workflow predictable enough that an analyst can estimate runtime and review load before the batch starts.

Implementation path

The smallest useful implementation is a compact TypeScript service with four folders.

1app/
2  lib/search/pubmed.ts
3  lib/search/clinicaltrials.ts
4  lib/search/openalex.ts
5  lib/schemas/study.ts
6  lib/workflows/evidence-intake.ts
7  lib/review/disagreement-router.ts
8  jobs/weekly-refresh.ts

The sequence should be:

1. Capture the question and inclusion criteria.
2. Build and store the PubMed query.
3. Fetch candidate PMIDs, PMCID links, and abstracts.
4. Fetch registry records from ClinicalTrials.gov for trial-side context.
5. Enrich the row with OpenAlex or DOI metadata.
6. Run the extractor against one row at a time.
7. Deduplicate by canonical ID before the reviewer sees anything.
8. Route disagreement cases to a human.
9. Persist approved rows into Postgres.
10. Export the final evidence table with source links and timestamps.

The registry side is important because many trial questions cannot be answered from papers alone. The modern ClinicalTrials.gov v2 API is the right place to reconcile registered endpoints, recruitment status, and protocol details with the published literature.

If you are already using a database review pattern, the same discipline from AI Postgres Migration Review Stack for Subscriptions applies here: make the schema explicit, keep the writes narrow, and review the diff before anything becomes visible to users.

A simple parser boundary

If the model output is not parseable, do not guess. Reject it and route it back through the schema boundary.

1const result = await runExtractor(candidateRow);
2
3if (result.status === "refused") {
4  return { status: "needs_review", reason: "model_refusal" };
5}
6
7const parsed = StudyRecord.safeParse(result.data);
8
9if (!parsed.success) {
10  return { status: "needs_review", reason: "schema_validation_failed" };
11}
12
13return { status: "ok", row: parsed.data };

That snippet looks simple because it should be simple. The more branches you add inside the model boundary, the more you turn a research workflow into an accidental application framework.

Failure modes

The failure modes here are predictable, which is good news. Predictable failures are easier to test.

The evaluation pattern from OpenTelemetry GenAI observability helps here because you can trace retrieval, extraction, review, and export separately. If a batch slows down, you want to know whether the bottleneck is search, extraction, or human review.

Safety, approval, and privacy

This workflow should never be allowed to drift into patient-level advice.

• Public literature and public registry data only.
• No patient charts, no EHR exports, no raw case notes.
• Human approval before the final table is shared externally.
• If you ingest internal PDFs, de-identify them first and treat them as restricted input.
• Do not let the model decide treatment, eligibility, or dosing.
• Use a second reviewer for borderline studies and sensitive summaries.
• Keep audit logs, but do not over-log sensitive source text.

The governance layer is not decoration. NIST AI RMF is useful because it gives you a clean way to talk about trustworthiness without pretending the risk is zero. Govern the process, map the use case, measure the output quality, and manage the residual risk. For a medical workflow, that is the minimum standard, not the advanced one.

If your team also uses shared drives or notes apps, keep those outside the review boundary unless a human explicitly approves the document source. The system should remain readable even if the optional outer tools disappear.

Verification checklist

Before you let the stack run unattended, verify it against a small hand-labeled set.

1. The PubMed query returns the expected PMIDs for a known test question.
2. The ClinicalTrials.gov lookup returns the expected trial identifiers for the same question.
3. Twenty random rows match the source text on title, year, and canonical IDs.
4. Every exclusion reason comes from the approved enum, not a free-form string.
5. Every approved row has a traceable source URL and fetch timestamp.
6. A human can explain each include/exclude decision in less than 30 seconds.
7. A retry after schema failure still produces the same field order and meaning.
8. The weekly refresh adds new evidence without duplicating old rows.
9. No PHI appears in logs, screenshots, or exported tables.
10. The reviewer can reject a bad row without editing the underlying source record.

If you want a regression gate, borrow the idea from RAG Eval Gates for TypeScript Support Agents: keep a fixed fixture set, rerun it after prompt or model changes, and fail the build if the evidence quality moves in the wrong direction.

FAQ

Can this stack screen patient charts?

No. Keep it on public literature and registry data only. If you need chart review, you are in a different risk class and need a separate governance model, separate access control, and a very different review workflow.

Why not just use a generic RAG app?

Because a generic RAG app tends to optimize for answer generation. This workflow needs source traceability, inclusion/exclusion reasons, canonical IDs, and human approval. That is evidence operations, not chat search.

Do I need vector search?

Not at the start. Use exact retrieval and canonical IDs first. Add embeddings later only if your corpus becomes large enough that exact search is no longer enough for recall.

Where does a second model help most?

The second model is best as a disagreement detector, not as a replacement for the first extractor. Use it when the record is ambiguous, the abstract is thin, or the registry and paper do not line up cleanly.

How do I keep the model from making up evidence?

Make the schema narrow, give it only the source text and the current record, and require enums for all important decisions. Then verify the final row against the source before the reviewer sees it.

What if the search query changes every week?

That is fine, as long as the query itself is versioned. Store the query hash, the inclusion criteria version, and the date range. A change in the question should be visible in the audit trail.

What is the smallest useful deployment?

A single TypeScript service, one Postgres schema, one read-only retrieval path, and one human reviewer. You do not need a giant multi-agent mesh to make this useful.

Related guides

• Structured Outputs for Reliable AI APIs for the schema boundary that keeps extraction typed.
• Prompt Caching for Real-World LLM Apps for repeated search and extraction batches.
• OpenTelemetry GenAI Observability: A Production Guide for tracing retrieval, extraction, and review separately.
• MCP Security Threat Modeling Guide for keeping tool access read-only and auditable.
• AI Postgres Migration Review Stack for Subscriptions for the database discipline behind a clean evidence store.

Sources

• NCBI APIs - Develop for the official E-utilities entry point.
• Entrez Programming Utilities Help for the official E-utilities help page and fixed URL interface.
• ClinicalTrials.gov API v2 overview for the public registry API and migration guidance.
• OpenAI Structured Outputs guide for strict JSON schema output and typed parsing.
• OpenAlex API overview for scholarly works and citation graph enrichment.
• NIST AI Risk Management Framework for governance, mapping, measurement, and management.