--- title: "Ways to check AI before it acts" description: "How NeMo Guardrails, Guardrails AI, LlamaFirewall, OpenAI Guardrails, Microsoft's Agent Governance Toolkit and AWS Automated Reasoning checks work, set beside QWED: what decides, and what happens when a check fails." url: https://qwedai.com/compare --- # Ways to check AI before it acts. Several open-source and managed tools check what an AI system reads, says or does. Here is how six of them work, set beside QWED, from each project's own documentation and code. Two questions separate them 1. **What makes the decision?** A model, a policy, or a proof. 2. **What happens when the check fails?** Does the content stop, or carry on? Checked on 15 October 2026 ## Three ways a check can _decide_ Most tools mix more than one. What matters is which one has the final word. - ### A model judges A classifier or a language model decides whether the input or output looks unsafe, off topic or injected. It suits properties that have no formal answer. - [NeMo Guardrails](#nemo) - [Guardrails AI](#guardrails-ai) - [LlamaFirewall](#llamafirewall) - [OpenAI Guardrails](#openai) - ### A policy decides Rules in a policy language allow or deny an action. The decision is repeatable, and only as good as the rules someone wrote. - [Agent Governance Toolkit](#agt) - ### A proof decides The claim becomes a formal statement that a solver or parser works out again. Only a claim proven to hold is admitted. - [QWED](#qwed) - [Bedrock Automated Reasoning](#bedrock) ## When the check itself _fails_ A model times out, a validator errors, a claim cannot be translated. Each tool has a default for that moment, and it is the easiest thing to miss in a demo. ### Stopped by default An error or an open question denies the claim or action. - [QWED](#qwed) Errors and unprovable claims are denied. - [Agent Governance Toolkit](#agt) Any error during evaluation is a deny. ### Depends on configuration The outcome depends on which rail, validator or scanner is running. - [NeMo Guardrails](#nemo) No library-wide policy; each rail and engine differs. - [Guardrails AI](#guardrails-ai) A failed check raises; an error depends on the validator. - [LlamaFirewall](#llamafirewall) Not documented; differs from scanner to scanner. ### Goes through by default Content carries on unless the app chooses otherwise. - [OpenAI Guardrails](#openai) The library passes content on when a check errors. - [Bedrock Automated Reasoning](#bedrock) Detect-only: it returns findings and never blocks. A default is not enforcement. In every case here, QWED included, the application or a gateway has to act on the result. A tool that returns a deny cannot stop an app that ignores it. [What fail-closed means for AI checks](https://qwedai.com/guides/fail-closed) ## Side by side | Tool | What decides | When a check errors or cannot decide | Who acts on the decision | | --- | --- | --- | --- | | [QWED](#qwed) | Deterministic engines: SymPy, Z3, SQLGlot, AST rules and JSON Schema. On the natural-language paths a model translates the text into a formal statement; that step is not verified, and no model decides the verdict. | Stopped by defaultAn error comes back BLOCKED · DENY. A claim that no engine can prove comes back UNVERIFIABLE · DENY. Only a proven claim is admitted. | Your integration, or a QWED product such as QWED Security, acts on the decision. QWED does not stop anything by itself. | | [NVIDIA NeMo Guardrails](#nemo) | Whatever each rail is configured with: Colang flows, Python actions, an LLM checking itself, NVIDIA safety models, jailbreak heuristics or third-party services. Tool-call validation against a JSON Schema in the IORails engine uses no model. | Depends on configurationThe docs say no library-wide failure policy applies to every rail. The IORails engine blocks when a rail fails; the default LLMRails engine returns a fixed internal-error reply; some third-party rails offer a fail-open setting. | The library, on the paths the app routes through it. With check(), the app receives a passed, modified or blocked status and acts on it. The app runs tools and handles identity. | | [Guardrails AI](#guardrails-ai) | Validators chosen per guard. Some are rules, such as regular expressions; some run machine-learning models, such as PII or toxic-language detectors; some call an LLM. | Depends on configurationA failed validation raises an exception by default (on\_fail is EXCEPTION). A validator that errors re-raises to the app. No error or timeout policy is documented, so the result of an error depends on the validator. | The Guard object in the app’s process raises, filters, fixes or asks again, as each validator’s on\_fail says. The app must catch the exception. | | [Meta LlamaFirewall](#llamafirewall) | PromptGuard 2, a classifier model, for prompt injection. AlignmentCheck, an LLM judge, for the agent’s trace. CodeShield, built on Semgrep and regular expressions, for code. | Depends on configurationNot documented; the code differs by scanner. When AlignmentCheck’s model call fails, it holds the trace for a person. When the trace or the user message is missing, it allows with an error status. | The app. scan() returns a decision, a reason and a score; the library does not halt the agent itself. | | [OpenAI Guardrails and Agents SDK guardrails](#openai) | The library: per check, an LLM, the OpenAI Moderation API, rules such as keyword and URL filters, or Presidio for PII. The SDK: the developer’s own guardrail function, which may call a model or run plain code. | Goes through by defaultThe library’s default mode passes content on when a check errors; raising instead is an opt-in setting. The SDK re-raises an error in a guardrail and ends the run, but input guardrails run in parallel by default, so tools may already have run. | The library’s client wrapper raises a tripwire exception that the app catches. The SDK’s Runner raises and halts the run. | | [Microsoft Agent Governance Toolkit](#agt) | A policy engine: Open Policy Agent Rego, Cedar, or one the host supplies. Optional annotators, including an LLM, add labels that the policy reads. | Stopped by defaultThe specification says any error during evaluation yields a deny, and an annotator timeout fails closed. When no rule matches, the policy’s own default decides; the quick-start example allows. | The host, through AGT’s SDKs, adapters or govern() wrapper. The specification calls a host that ignores a deny non-conformant. The engine runs in the agent’s process. | | [Amazon Bedrock Guardrails: Automated Reasoning checks](#bedrock) | Several foundation models translate the text into formal logic; their agreement sets a confidence score. Formal logic validation against the policy then returns a finding such as VALID, INVALID or TRANSLATION\_AMBIGUOUS. The policy itself is extracted by models from your own document. | Goes through by defaultThe checks return findings and do not block content. When nothing can be translated, AWS recommends serving the response without validation. A request without the guardrail tag skips the check with no error. | The application decides what to do with the findings. Other Bedrock Guardrails policies can block; these checks do not. | ## Tool by tool Each entry comes from the project's own pages or its code at a named release. The links go to the exact source. ### [QWED](https://qwedai.com/engines) - Solver - Parser - Rules Stopped by defaultAn error comes back BLOCKED · DENY. A claim that no engine can prove comes back UNVERIFIABLE · DENY. Only a proven claim is admitted. What it checks Claims in AI output that its engines can put into formal terms: arithmetic, logic, SQL, code and JSON schemas. Policy guards flag unsafe commands, leaked secrets and untrusted tool calls. What decides Deterministic engines: SymPy, Z3, SQLGlot, AST rules and JSON Schema. On the natural-language paths a model translates the text into a formal statement; that step is not verified, and no model decides the verdict. Who acts on the decision Your integration, or a QWED product such as QWED Security, acts on the decision. QWED does not stop anything by itself. Licence and hosting Apache-2.0. Self-hosted: Python library, CLI, API server or Docker image. Status Single maintainer. Versions and status on the ecosystem page. Sources [What QWED does not verify](https://qwedai.com/limits) [Engines](https://qwedai.com/engines) ### [NVIDIA NeMo Guardrails](https://github.com/NVIDIA-NeMo/Guardrails) - Model - Rules - Schema Depends on configurationThe docs say no library-wide failure policy applies to every rail. The IORails engine blocks when a rail fails; the default LLMRails engine returns a fixed internal-error reply; some third-party rails offer a fail-open setting. What it checks User prompts, retrieved chunks, multi-turn dialogue, custom actions, tool calls and results, and model output, each through its own kind of rail. What decides Whatever each rail is configured with: Colang flows, Python actions, an LLM checking itself, NVIDIA safety models, jailbreak heuristics or third-party services. Tool-call validation against a JSON Schema in the IORails engine uses no model. Who acts on the decision The library, on the paths the app routes through it. With check(), the app receives a passed, modified or blocked status and acts on it. The app runs tools and handles identity. Licence and hosting Apache-2.0. Self-hosted library, API server or Docker image; a separate Kubernetes microservice is part of NVIDIA AI Enterprise. Status v0.24.1, 16 September 2026. Owned by NVIDIA. **Where it goes further than QWED**Dialog rails: Colang flows that keep a multi-turn conversation on defined paths. Steering a conversation is outside QWED’s engines. Sources [Runtime security FAQ](https://docs.nvidia.com/nemo/guardrails/resources/runtime-security-faq) [Tool calling rails](https://docs.nvidia.com/nemo/guardrails/configure-guardrails/guardrail-catalog/tool-calling) [rail\_guard.py at v0.24.1](https://github.com/NVIDIA-NeMo/Guardrails/blob/v0.24.1/nemoguardrails/guardrails/rail_guard.py#L16-L25) ### [Guardrails AI](https://github.com/guardrails-ai/guardrails) - Rules - Model Depends on configurationA failed validation raises an exception by default (on\_fail is EXCEPTION). A validator that errors re-raises to the app. No error or timeout policy is documented, so the result of an error depends on the validator. What it checks Model inputs and outputs, including structured output checked against a Pydantic or RAIL schema. What decides Validators chosen per guard. Some are rules, such as regular expressions; some run machine-learning models, such as PII or toxic-language detectors; some call an LLM. Who acts on the decision The Guard object in the app’s process raises, filters, fixes or asks again, as each validator’s on\_fail says. The app must catch the exception. Licence and hosting Apache-2.0. Self-hosted library and server. The free hosted inference servers shut down on 25 August 2026. Status v0.11.0, 14 August 2026. Harvey announced the acquisition of Guardrails AI on 9 September 2026; the announcement does not mention the open-source framework. **Where it goes further than QWED**Model-based validators for properties with no formal truth value, such as toxic language or competitor mentions. In QWED those claims fall outside the engines. Sources [on\_fail actions](https://guardrailsai.com/guardrails/docs/how-to-guides/use_on_fail_actions) [validator\_base.py at v0.11.0](https://github.com/guardrails-ai/guardrails/blob/v0.11.0/guardrails/validator_base.py#L139-L140) [Hub update notice](https://github.com/guardrails-ai/guardrails/blob/main/HUB_UPDATE.md) [Harvey announcement](https://www.harvey.ai/blog/guardrails-ai-joins-harvey) ### [Meta LlamaFirewall](https://github.com/meta-llama/PurpleLlama/tree/main/LlamaFirewall) - Classifier - LLM judge - Rules Depends on configurationNot documented; the code differs by scanner. When AlignmentCheck’s model call fails, it holds the trace for a person. When the trace or the user message is missing, it allows with an error status. What it checks User and system messages, the agent’s trace of reasoning and tool calls, and generated code. What decides PromptGuard 2, a classifier model, for prompt injection. AlignmentCheck, an LLM judge, for the agent’s trace. CodeShield, built on Semgrep and regular expressions, for code. Who acts on the decision The app. scan() returns a decision, a reason and a score; the library does not halt the agent itself. Licence and hosting MIT for the LlamaFirewall code. The PromptGuard 2 weights carry the Llama 4 licence. AlignmentCheck calls a hosted model API by default. Status 1.0.3 on PyPI, 29 May 2025. Later commits are maintenance only. **Where it goes further than QWED**AlignmentCheck: whether an agent’s actions still serve the user’s stated goal. QWED does not check intent. Sources [LlamaFirewall README](https://github.com/meta-llama/PurpleLlama/blob/d934ea285d124d81c10f7e15a06a1e1f74eff2c3/LlamaFirewall/README.md) [alignmentcheck\_scanner.py](https://github.com/meta-llama/PurpleLlama/blob/d934ea285d124d81c10f7e15a06a1e1f74eff2c3/LlamaFirewall/src/llamafirewall/scanners/experimental/alignmentcheck_scanner.py#L76-L130) ### [OpenAI Guardrails and Agents SDK guardrails](https://github.com/openai/openai-guardrails-python) - Model - Rules Goes through by defaultThe library’s default mode passes content on when a check errors; raising instead is an opt-in setting. The SDK re-raises an error in a guardrail and ends the run, but input guardrails run in parallel by default, so tools may already have run. What it checks The library: inputs and outputs of OpenAI client calls, and tool calls for prompt injection. The Agents SDK: the first agent’s input, the final output, and each function-tool call before and after it runs. What decides The library: per check, an LLM, the OpenAI Moderation API, rules such as keyword and URL filters, or Presidio for PII. The SDK: the developer’s own guardrail function, which may call a model or run plain code. Who acts on the decision The library’s client wrapper raises a tripwire exception that the app catches. The SDK’s Runner raises and halts the run. Licence and hosting MIT for both. Self-hosted libraries; model-based checks call the OpenAI API or any compatible endpoint. Status Library v0.3.3, 10 September 2026, marked Preview. Agents SDK v0.24.0, 15 October 2026. **Where it goes further than QWED**Moderation against OpenAI’s categories, and model classification of jailbreak, NSFW and off-topic content. These are judgements, not formal claims. Sources [Guardrails quickstart, fail-safe mode](https://github.com/openai/openai-guardrails-python/blob/v0.3.3/docs/quickstart.md#L132-L133) [Agents SDK guardrails](https://github.com/openai/openai-agents-python/blob/v0.24.0/docs/guardrails.md#L16-L90) ### [Microsoft Agent Governance Toolkit](https://github.com/microsoft/agent-governance-toolkit) - Policy engine Stopped by defaultThe specification says any error during evaluation yields a deny, and an annotator timeout fails closed. When no rule matches, the policy’s own default decides; the quick-start example allows. What it checks Agent actions at eight points, from startup to shutdown: input, model calls, tool calls and their results, and final output. What decides A policy engine: Open Policy Agent Rego, Cedar, or one the host supplies. Optional annotators, including an LLM, add labels that the policy reads. Who acts on the decision The host, through AGT’s SDKs, adapters or govern() wrapper. The specification calls a host that ignores a deny non-conformant. The engine runs in the agent’s process. Licence and hosting MIT. Self-hosted packages for Python, TypeScript, .NET, Rust and Go, plus Docker and sidecar deployment. Status v5.0.0, 27 July 2026, labelled Public Preview. Microsoft says it aims to move the project to a foundation. **Where it goes further than QWED**Cryptographic identity for each agent, with trust scoring and delegation chains. QWED A2A signs its decisions, but QWED has no identity for agents. Sources [Agent Control Specification](https://github.com/microsoft/agent-governance-toolkit/blob/v5.0.0/policy-engine/spec/SPECIFICATION.md#L430) [README at v5.0.0](https://github.com/microsoft/agent-governance-toolkit/blob/v5.0.0/README.md) ### [Amazon Bedrock Guardrails: Automated Reasoning checks](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-automated-reasoning-checks.html) - Model translation - Formal logic Goes through by defaultThe checks return findings and do not block content. When nothing can be translated, AWS recommends serving the response without validation. A request without the guardrail tag skips the check with no error. What it checks A model’s response against an Automated Reasoning policy, using the user’s query as premises. There is no input-only check. What decides Several foundation models translate the text into formal logic; their agreement sets a confidence score. Formal logic validation against the policy then returns a finding such as VALID, INVALID or TRANSLATION\_AMBIGUOUS. The policy itself is extracted by models from your own document. Who acts on the decision The application decides what to do with the findings. Other Bedrock Guardrails policies can block; these checks do not. Licence and hosting Proprietary managed AWS service, charged per validation request, in three US and three EU Regions. Status Generally available since August 2025; policy refinement added June 2026. **Where it goes further than QWED**Builds a formal rule set from your own natural-language policy document, with a report linking each rule to its source text. QWED’s engines cover fixed domains. Sources [Automated Reasoning checks overview](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-automated-reasoning-checks.html) [Integrating the checks](https://docs.aws.amazon.com/bedrock/latest/userguide/integrate-automated-reasoning-checks.html) ## What QWED does not cover A fair comparison says where the others go further. A system may need another tool for these. [Everything QWED does not verify](https://qwedai.com/limits) - Content safety and topic control - Steering a conversation - Prompt-injection classifiers - Checking what the user meant - Identity for agents - Sandboxing and kill switches - Rules from your own policy documents - Evaluating models over datasets ## How this page is kept - Every statement about another tool comes from its own documentation, its code at a pinned release, or its owner's announcements, checked on 15 October 2026. - There are no accuracy, speed or popularity figures here, for any tool, QWED included. Those depend on the workload; measure them on yours. - Projects change. If something here is out of date or wrong, [tell us](https://qwedai.com/contact) and it will be corrected.