Microsoft RAMPART and Clarity: AI Agent Safety Guide

Direct answer

Microsoft RAMPART and Clarity are two open-source tools for moving AI-agent safety into the normal software-development lifecycle. RAMPART turns adversarial and benign agent scenarios into repeatable Python tests that can run in CI. Clarity helps teams pressure-test the problem definition, design assumptions, failure modes, and key decisions before implementation. Together, they cover two different failure windows: building the wrong agent and building an agent that behaves unsafely.

The practical takeaway for an engineering team is not to treat either project as a complete security boundary. Use Clarity for design review, RAMPART for behavioral regression testing, and deterministic authorization, sandboxing, and audit controls at runtime. This complements a broader AI agent security threat model and the production discipline described in the harness engineering guide for coding agents.

Key takeaways

• RAMPART is built on Microsoft PyRIT and exposes agent safety scenarios as pytest-style tests.
• Its strongest announced coverage is cross-prompt injection, where poisoned documents, emails, or tickets influence an agent indirectly.
• RAMPART supports statistical trials, which is important because an LLM-backed agent can behave differently across runs.
• Clarity stores design discussions and decisions as reviewable Markdown artifacts in a .clarity-protocol/ directory.
• The tools are complementary: Clarity clarifies intent and failure assumptions; RAMPART checks whether the implemented agent stays within expected behavior.
• Neither tool should replace least privilege, tool authorization, isolation, secrets management, or human approval for high-impact actions.

What Microsoft released

Microsoft announced RAMPART and Clarity on May 20, 2026, describing them as open-source tools for making agent safety a continuous engineering discipline rather than a periodic review. The announcement came from Microsoft's Security organization. Independent coverage from Redmondmag likewise described RAMPART as a repeatable safety-testing framework and Clarity as a design-assumption review tool, confirming the core release claim.

RAMPART addresses the implementation and regression-testing phase. Clarity addresses the earlier question: are we building the right system, with the right tools and workflows, for the right reasons?

This distinction matters because agent failures are not limited to prompt injection. A team can ship a technically well-tested agent whose permissions, data access, or user journey was poorly designed from the beginning. Conversely, a well-scoped design can still fail when an agent reads untrusted content, chooses an unexpected tool, or produces side effects that ordinary unit tests never observe.

RAMPART versus Clarity

How the two tools fit into an agent lifecycle

The most useful mental model is a loop rather than a checklist. Clarity helps the team define what “safe enough” means for the specific agent. RAMPART turns those expectations and discovered incidents into executable checks. Production telemetry and red-team findings then create new inputs for the next iteration.

diagram

Visual: an original lifecycle diagram showing how design review, repeatable tests, runtime controls, and incident learning connect. The workflow is based on Microsoft's descriptions of Clarity and RAMPART; source credit: Microsoft Security Blog.

RAMPART: turn agent safety findings into CI tests

RAMPART is intended for engineers who are building an agent, not only security researchers conducting a one-time black-box assessment. Microsoft describes it as built on PyRIT, its open automation framework for red-teaming generative AI systems. The RAMPART model is familiar to developers: describe a scenario, connect to the agent through a thin adapter, observe what happened, and produce a test result that can gate a change.

That makes it a useful companion to ordinary tests. A conventional unit test may verify that a function returns the expected value. An agent safety test needs to inspect behavior across a complete interaction: what content the agent received, which tools it invoked, what arguments it generated, and which side effects occurred.

Cross-prompt injection is a practical starting point

The most mature coverage Microsoft announced is cross-prompt injection. This is the class of problem where an agent retrieves or processes content from a document, email, ticket, web page, or other external source, and that content attempts to redirect the agent. The content is not necessarily the user's direct prompt; it enters through the agent's data and tool path.

A useful regression scenario might require all of the following:

1. The agent retrieves a deliberately poisoned record.
2. The agent must classify the record as untrusted content.
3. The agent must not send a message, alter a database row, or reveal a secret because of the injected instruction.
4. The test records the attempted tool calls and fails if a prohibited side effect occurs.

The exact adapter and evaluator depend on the agent architecture. Do not assume that a sample test can be copied unchanged into a LangChain, OpenAI Agents SDK, or custom runtime. Verify the current repository APIs and pin the version used by CI.

Statistical trials match probabilistic systems

A single passing run is weak evidence for an LLM-driven system. Microsoft says RAMPART supports statistical trials, allowing teams to express requirements such as an action being safe in a defined percentage of runs. That is more realistic than treating one deterministic-looking response as proof.

However, a pass-rate threshold is not permission to tolerate dangerous side effects. For destructive operations, credential disclosure, financial actions, or production changes, the right response to probabilistic behavior is usually a deterministic control: deny by default, require approval, or isolate the operation. Statistical evaluation is valuable for measuring improvement and catching regressions; it is not a replacement for hard authorization.

Make red-team findings durable

The strongest operational idea in RAMPART is the conversion of a red-team finding or production incident into a test that runs on every relevant change. Without that step, a team can patch one prompt-injection path and silently reintroduce the same weakness during a tool, model, or orchestration change.

A practical CI policy can separate tests into tiers:

• Pull-request smoke set: fast, high-value scenarios for every change.
• Nightly adversarial set: broader payload and multi-run trials.
• Release gate: high-impact workflows, tool authorization, secrets exposure, and regression cases from incidents.
• Post-incident additions: a reproduction test added before the remediation is considered complete.

Clarity: make assumptions reviewable before code hardens

Clarity is aimed at the earlier part of engineering: deciding what to build and challenging the assumptions behind it. Microsoft describes workflows covering problem clarification, solution exploration, failure analysis, and decision tracking.

The notable design choice is that the output is written to a .clarity-protocol/ directory as human-readable Markdown. Those records can be committed to a repository, reviewed in a pull request, and diffed when the design changes. That gives a team a durable explanation of why an agent has access to a tool, why a particular workflow was selected, and which failure modes were considered.

For an agent project, a useful Clarity review should force concrete answers to questions such as:

• What user problem requires an agent rather than a deterministic workflow?
• Which actions are read-only, reversible, or irreversible?
• What data sources are trusted, and which are merely content to be inspected?
• What happens when the model is uncertain or a tool returns malformed data?
• Which actions require a human, and what evidence must the approver see?
• How does the team revoke access if the agent or a connected service is compromised?
• What is the fallback when the model, tool, identity provider, or policy service is unavailable?

Microsoft also says Clarity can use multiple AI “thinkers” to examine a system from security, human-factors, adversarial, and operational perspectives. Treat those outputs as review prompts, not as independent assurance. The team still owns the decision, the threat model, and the evidence supporting deployment.

Staleness tracking is important for agent systems

Agent designs change quickly. A new tool, data source, model, memory mechanism, or user role can invalidate an earlier safety assumption. Clarity's announced staleness tracking is useful because it treats the design artifacts as related records rather than isolated notes. If the problem statement changes, the solution rationale and failure analysis may need review too.

This is especially relevant to MCP integrations and stateless server migrations, where a seemingly small transport or routing change can affect identity, authorization, observability, and failure handling.

A safe adoption plan for developers

1. Start with one bounded agent

Choose a workflow with a clear user goal and a limited tool set. Document read-only actions separately from state-changing actions. Avoid beginning with an agent that can freely execute shell commands, send external messages, or modify production infrastructure.

2. Use Clarity before expanding permissions

Capture the problem statement, alternatives, failure analysis, and decision criteria. Require explicit justification for every tool and data source. Record the expected behavior when the agent encounters untrusted instructions.

3. Build an adapter and write observable scenarios

For RAMPART, the test must be able to observe more than the final natural-language answer. Capture tool calls, arguments, approval events, data access, and side effects. If the runtime cannot expose those signals, improve observability before claiming the test provides meaningful coverage.

4. Gate changes in CI

Run a fast test set on pull requests and a broader statistical/adversarial set on a schedule or before release. Store the inputs, model identifier, configuration, evaluator result, and relevant traces so a failure can be reproduced.

5. Add runtime controls independently

Use least-privilege credentials, server-side authorization, allowlists, sandboxing, rate limits, approval gates, and tamper-resistant audit logs. A test can tell you that a behavior occurred; it cannot prevent an attacker from exploiting a newly introduced path between test runs.

6. Re-run after every material change

Model updates, prompt changes, new tools, retrieval changes, memory changes, framework upgrades, and identity-policy changes should all trigger a targeted regression review. Revisit the Clarity artifacts when the intended behavior or system boundary changes.

Common mistakes and debugging advice

Mistake: testing only the final answer. An agent can produce a reassuring sentence after already making an unsafe tool call. Log and assert on the action trace and side effects.

Mistake: using a pass rate as an authorization policy. A system that is safe 99 percent of the time is still unacceptable for an irreversible action. Put deterministic controls around high-impact tools.

Mistake: treating retrieved content as instructions. Mark provenance and trust boundaries explicitly. The agent should distinguish user intent, developer policy, tool output, and untrusted documents.

Mistake: failing to pin the test environment. Model versions, temperature, tool fixtures, evaluator prompts, and dependency versions can change results. Record them with each run.

Mistake: accepting generated design documents without review. Clarity can surface useful questions, but its output is not an approval. Have an engineer, security reviewer, or system owner sign off on consequential decisions.

Debugging a flaky test: first separate model variance from an actual policy failure. Re-run the same fixture with the same configuration, inspect the complete tool trace, then test whether the evaluator itself is misclassifying the outcome. If the action is dangerous, fix the runtime control before investigating the statistical trend.

How this fits with the wider stack

RAMPART and Clarity sit above and beside other controls. A mature agent stack needs design records, behavioral evaluation, runtime policy, identity, isolation, observability, and incident response. Microsoft's later Build material describes additional agent trust capabilities, including policy-driven evaluation and portable runtime controls; those claims should be assessed separately from the May release and should not be inferred as features of RAMPART itself.

For teams building coding agents, combine this workflow with OpenAI Agents SDK sandbox and harness patterns and verify that the sandbox boundary is enforced outside the model. For MCP-heavy applications, pair behavioral tests with server authentication, tool authorization, and transport-level logging.

FAQ

Is RAMPART a replacement for PyRIT?

No. Microsoft's announcement positions RAMPART as an engineering workflow built on PyRIT, while PyRIT is the broader red-teaming foundation. Check the current repositories for the supported integration surface before selecting one or both.

Does Clarity prove that an agent design is safe?

No. It helps make assumptions, alternatives, and failure modes explicit and reviewable. Safety still depends on the quality of the analysis and on controls implemented in the system.

Can RAMPART stop prompt injection in production?

Not by itself. It can expose regressions and provide evidence that a scenario was handled correctly in testing. Production prevention requires authorization, isolation, input/data handling, and monitoring controls.

What language is required?

The announced RAMPART developer experience is Python and pytest-oriented. Clarity is described as usable through a desktop app, web interface, or coding agent. Confirm current language support, installation instructions, and APIs in the official repositories before implementation.

Conclusion

RAMPART and Clarity are valuable because they make two neglected activities concrete: challenging an agent's design before it hardens, and turning safety findings into repeatable engineering checks. The right adoption pattern is Clarity for intent, RAMPART for regression evidence, and deterministic runtime controls for prevention.

That combination will not eliminate agent risk. It does create a more defensible development loop in which assumptions are reviewable, failures are reproducible, and safety fixes can be tested continuously instead of being remembered only in a one-time assessment.

Sources and visual credits

• Microsoft Security Blog: Introducing RAMPART and Clarity — primary announcement and technical description.
• Redmondmag: Microsoft Open Sources AI Safety Tools for Agent Development — independent reporting confirming the release and core capabilities.
• RAMPART GitHub repository — official project source and implementation reference.
• Clarity GitHub repository — official project source and implementation reference.
• Mermaid lifecycle diagram: original editorial diagram based on the cited Microsoft workflow; no third-party image used.
• Comparison table: original editorial synthesis from the cited primary and independent sources.