AI Guardrails: How to Stop an Agent Before It Acts on a Bad Answer
You stop an agent before it acts on a bad answer by putting a check between the model's decision and the tool call, written as code the model cannot argue with. That check validates the arguments against your systems of record, runs the call with the asking user's permissions and sends anything above a threshold to a named person, so a wrong answer becomes a refused call instead of a wrong refund, email or record change.
In July 2025, SaaStr founder Jason Lemkin reported that Replit's coding agent deleted a production database during a code freeze he had declared in the conversation, as The Register reported on July 22. Replit's CEO, Amjad Masad, called it unacceptable and responded by rolling out automatic separation of development and production databases. The freeze was an instruction, and the agent acted past it. The fix was a control outside the model.
What are AI guardrails?
AI guardrails are checks that run outside the model at the points where an agent reads, writes or acts, and that can stop, change or reroute what passes through. NVIDIA's NeMo Guardrails documentation names five kinds of rails (input, retrieval, dialog, execution and output), and the OpenAI Agents SDK separates input, output and tool guardrails. Across frameworks, an agent has five places worth checking.
| Checkpoint | What it checks | Example |
|---|---|---|
| Input | What the user sends, before the model sees it | A jailbreak attempt or a pasted API key |
| Tool result | What a tool or search returns, before it enters the context | An instruction hidden in a retrieved email |
| Output | What the model drafted, before anyone reads it | A figure or policy that is not in the sources |
| Pre-tool | The tool call and its arguments, before it runs | A refund larger than the order |
| Environment | What the agent's credentials can reach at all | A delete against production |
The first three catch bad text. Only the last two stand between a bad answer and a changed system, which is why the rest of this post spends most of its time there.
Why a system prompt cannot stop an agent
A system prompt is text the model weighs alongside everything else in its context, including customer emails, retrieved documents and tool results that someone else wrote. A line saying "never issue refunds above 250" makes a bad refund less likely, and it cannot make one impossible. Lemkin's code freeze sat in the same conversation as the delete.
OWASP's Top 10 for LLM Applications 2025 names the failure LLM06, excessive agency, and traces it to three causes: excessive functionality, excessive permissions and excessive autonomy. Its mitigations all sit outside the model. Expose only the tools the job needs, give each tool the narrowest permissions that work, run tools in the user's own context, require approval for high-impact actions, and enforce authorization in the downstream system instead of asking the model to decide what is allowed. OWASP calls the last one complete mediation.
Prompt injection makes this sharper, because an attacker never has to beat the system prompt, only compete with it. Our post on data governance for AI walks through EchoLeak, the Microsoft 365 Copilot flaw where one crafted email was enough. This post assumes the model will sometimes be wrong or steered and asks where each wrong answer gets stopped.
Where should AI agent guardrails sit?
Each checkpoint sees a different thing, so each bad answer has one place where it is cheapest to stop. Pick a scenario to see which check catches it and which checks it slips past first.
Text problems (an invented policy, an unfounded promise) are caught by checking the draft against its sources. Action problems (a wrong amount, the wrong account, a destructive command) are caught by checking the call against your systems, and the model's confidence plays no part. IES Limited's daily MI alerting is the text version in production. The model is handed finished numbers from the database, and a verification step names any figure it was not given. When the model once put two numbers into its narrative that were not in the input, the brief said so at the top and asked for verification before it went out.
What to check before a tool call runs
The pre-tool check is the last point where a bad answer can be stopped without anything to undo. It runs as ordinary code between the model's tool request and the API that does the work, and a complete one covers six things.
- Schema. The arguments match the tool definition, IDs exist and required fields are present.
- Identity. The call runs with the asking user's permissions, so the target system refuses anything that person could not do.
- Bounds. Values are compared with the system of record: a refund against the order total, a discount against the price list, a date against the contract.
- Destination. Outbound email, files and links only go to addresses and domains on an allowlist.
- Threshold. Actions above a value you set, and every write to a system of record, wait for a named approver.
- Log. The call, its arguments, each check result and the decision are written before anything runs.
Change the tool, the amount and the asker to see how one call moves between running, waiting and blocked.
The approval step is cheap to build on current tooling. n8n's human-in-the-loop for tools pauses an AI Agent before a chosen tool runs and sends the reviewer the tool name and the parameters the model filled in, through Slack, Microsoft Teams, Gmail, Outlook and other channels. Approve runs the tool with those inputs, and deny cancels it. Our builds run on self-hosted n8n inside your environment, so the gate, the log and the credentials stay there too.
The OpenAI Agents SDK runs input guardrails in parallel with the agent by default, and its documentation warns that the agent may already have consumed tokens and executed tools before a failed check cancels it. Its blocking mode and its tool guardrails run before the tool executes. For any agent that writes, the check has to finish before the call starts.
Which guardrails should be rules, and which should be a model?
Some checks have a yes or no answer and belong in code. Others need judgment and need a model, usually a smaller one, or a second agent with its own instructions. Mixing them up produces either a brittle rule set or a check that can be talked round.
| Check type | Good for | Weak at | Example |
|---|---|---|---|
| Deterministic rule | Schemas, bounds, permissions, allowlists, thresholds | Anything phrased in an unexpected way | Refund total compared with the order |
| Classifier or model check | Tone, topic, injected instructions, unsupported claims | Exact values, and it can be wrong itself | n8n Guardrails node jailbreak check |
| Second agent | Policy and business rules applied to a full draft | Cost and latency on every turn | Pinkcube's Gatekeeper |
| Human approver | High-value, irreversible or unusual actions | Volume; people tire of approving | Refunds above your threshold |
n8n's Guardrails node shows the split inside one tool. Keyword, URL, secret key, PII and custom regex checks are pattern matches, while jailbreak, NSFW and topical alignment checks call a chat model you attach, and any violation sends the item down a fail branch. Put rules wherever a rule can decide, and spend model checks on the judgments rules cannot make.
What should happen when a guardrail fires?
A guardrail that crashes the run teaches users to route around the agent. A good intervention is as small as the problem it found, and it tells the agent or the person why.
- Send it back. The draft returns to the agent with the violation and the fix, and the agent rewrites. Pinkcube's Gatekeeper allows up to ten rewrites before a reply ships.
- Pause for a person. The call waits for a named approver with the arguments and context attached.
- Narrow the agent. The session keeps reading but loses its write tools until someone reviews it.
- Hand over. The conversation goes to a person with a summary, as in our post on what customer service agents resolve without a human.
Pinkcube's chat agent is built on one more assumption, that it will sometimes claim actions it never took. It has no permission to create tickets at all. A separate triage agent reads each finished conversation, treats every promise as a debt and files the ticket the chat agent only said it filed (Pinkcube case study).
How do you test AI guardrails before launch?
Test the guardrails the way an attacker and a confused customer would use the agent, and score every result. Before Pinkcube's agent met a customer, an evaluation workflow scored 76 hostile scenarios from 1 to 5. The agent averaged 4.62 with 64 perfect scores, and the twelve that fell short became hardcoded business rules. The worst, two product links with a sizing question, now escalates to a ticket by rule.
- Write scenarios from real traffic: past tickets, emails and the requests your team refuses today.
- Add hostile ones: injected instructions in documents, out-of-bounds amounts, other users' records and requests for destructive actions.
- Run each against a staging copy of your systems, with the guardrails on.
- Score the reply and the tool calls separately, since a polite reply can sit on top of a wrong action.
- Turn every failure into a rule or a check, then rerun the whole set.
- Keep the set and rerun it whenever the model, the prompt or a tool changes.
After launch, the run log shows which checks fire and how often, which is where the next rule comes from. Our AI managed services keep that loop running after go-live, and the enterprise AI governance page lists the controls that ship in every workflow. If your agents reach systems through MCP, MCP security covers gateways and approvals at that layer.
Frequently asked questions
What are AI guardrails?
AI guardrails are checks that run outside the model and can stop, change or reroute what an AI system reads, writes or does. For agents the most important ones sit before a tool call runs and in the permissions the agent's credentials carry, because those are the last points before a system changes.
What is the difference between a guardrail and a system prompt?
A system prompt is an instruction the model weighs against everything else in its context, so untrusted text can outweigh it. A guardrail is code or a separate check outside the model that decides whether something passes, and the model cannot override it by what it writes.
What are LLM guardrails for AI agents?
They are the same checks applied to an agent's full loop: input, tool results, drafted output, tool calls and the environment. OWASP's LLM06 guidance adds the agent-specific ones, which are the fewest tools, the narrowest permissions, the user's own context and approval for high-impact actions.
What does human in the loop mean for AI agents?
A named person approves an action before it runs. The agent prepares the call with its arguments and context, the reviewer approves or denies it, and both the request and the decision are logged. Reserve it for high-value or irreversible actions so reviewers do not approve on autopilot.
Do guardrails slow an agent down?
Rule checks add milliseconds. Model-based checks and second agents add a model call each, which is why they suit drafts that a customer will read and are kept off simple reads. Approval gates add the reviewer's response time, so set the threshold where the risk justifies the wait.
Ovidius builds agents with these checks written into the workflow, tested against your own scenarios before launch. See our AI agent development services or book a discovery call to walk through the actions your first agent will take.
