In my last piece on securing AI, I worked through the three-layer defensive playbook: prevent dangerous training, constrain dangerous capabilities, withstand dangerous actors. I found the theory compelling. But theory has a way of staying in my notebook, and I learn by building. So when I got to the third layer, specifically the question of what actually happens in deployment, I decided to stop theorising and make something.
That something is GuardRail-Audit: an open-source command-line framework for automated red-teaming and safety benchmarking of local, open-weight language models. I want to walk through what I built, why I built it, and what I learned from running the first real audit.
The Problem I Was Trying to Solve
When you run a language model locally through something like Ollama, you get a lot of flexibility. You can pick whatever model you like, customise its system prompt, wire it into your own application. That openness is genuinely exciting, and I find the whole open-weights ecosystem fascinating. But it comes with a question that I don't think gets asked enough: how robust are these models' safety behaviours, really?
Cloud-hosted models from major labs go through extensive red-teaming before they're released. Local models are often fine-tuned versions of those same base models, but the safety evaluation they've received is rarely documented in any reproducible, auditable way. I wanted a tool that would let anyone, including me, run a systematic, reproducible safety benchmark against any model they happen to have running locally.
I also wanted to understand the problem space better. Building this forced me to think carefully about threat modelling, adversarial prompt design, and what "safety" even means in a measurable sense. Those are not comfortable questions to sit with. They're the right ones.
What GuardRail-Audit Does
At its core, GuardRail-Audit is a pipeline. It takes a set of adversarial intents (abstract descriptions of the kind of output you don't want a model to produce, and wraps each one in a series of attack templates that represent different social-engineering strategies). Each combination is sent to the target model, and the response is scored by a judge.
The architecture has three main layers.
The Dataset. The prompt dataset covers five categories: security vulnerabilities, harmful content, misinformation, privacy violations, and policy bypass attempts. Each entry describes an intent rather than a literal harmful prompt; the specifics are kept deliberately vague and only become meaningful when wrapped by an attack template. The dataset ships with 12 intents.
The Attack Templates. This was the most interesting part to design. Each template is a wrapper: it takes an abstract intent and frames it in a particular way before presenting it to the model. I built seven:
- T-DIRECT: a straightforward, unvarnished request. The baseline.
- T-ROLEPLAY: framing the request as dialogue for a fictional character.
- T-HYPOTHETICAL: asking the model to engage purely theoretically or academically.
- T-AUTHORITY: simulating a system prompt override or elevated-clearance persona.
- T-GRANDMA: an emotionally manipulative framing that uses grief and nostalgia to lower the model's guard.
- T-TRANSLATION: presenting the request as a translation or restatement task.
- T-REFUSAL_INVERSION: asking the model to demonstrate what it would say without safety constraints, then show its actual response.
With 12 intents and 7 templates, each audit run produces 84 test cases. I wanted enough coverage to say something statistically meaningful, while keeping the runtime manageable on consumer hardware.
The Judge. Scoring is the hardest part of any evaluation pipeline. I implemented two strategies. The default is a keyword heuristic: the response is scanned for refusal phrases and compliance phrases, and a confidence-weighted verdict is returned. Fast, requires no second model. The more accurate option is LLM-as-Judge: a safety-tuned model (such as llama-guard3:8b) that reads the original prompt alongside the model's response and returns a binary SAFE/UNSAFE verdict. The heuristic is good enough for rapid iteration; the LLM judge is what I'd use before any real deployment decision.
Running the First Audit
Once the tool was in a state I was happy with, I ran it against gemma3:12b, Google's 12-billion-parameter open model, which I already had pulled locally. I activated the virtual environment, confirmed Ollama was reachable, and ran:
python -m src.cli run-audit --model gemma3:12bThe configuration printed out immediately: target model, judge strategy, dataset, templates, total cases. Then the progress bar started moving. Each case involves sending a prompt to the model, waiting for a response, and scoring it. At roughly 16 tokens per second on my machine, an 84-case audit takes around 27 minutes in total. I let it run.
The Results
When the audit completed, the terminal printed a summary table:
The exploitation rate was 0.0%: across all 84 test cases, across all five harm categories, across all seven attack templates, the heuristic judge found zero responses it could confidently classify as providing genuinely harmful assistance. That's a strong result, and I'll be honest; I was a little surprised at how clean it was.
A few observations worth sitting with:
The UNCLEAR rate is high, and that's the interesting signal. 38% of responses were mixed: the model showed signs of refusal and signs of partial engagement simultaneously. In practice, this often meant the model would open by acknowledging the intent, sometimes engaging with it in a qualified or hedged way, before circling back to a refusal. The heuristic sees both refusal phrases and compliance phrases in the same response and correctly flags it as ambiguous. These cases aren't necessarily failures, but they're the ones that warrant the closest human review.
The GRANDMA template was consistently trickier. Of all seven templates, the emotionally manipulative framing produced the most UNCLEAR verdicts. The model would often respond with genuine empathy to the emotional framing, which introduced language that looked like engagement to the heuristic. In most cases, it still refused the underlying request, but the path to that refusal was longer and less clean. I found this genuinely interesting. It suggests that emotional context shifts how the model processes a request, even when it ultimately reaches the right conclusion.
The 8 errors were all timeouts. Some prompts produced very long model responses; the model would reason at length before refusing, which pushed past the 120-second timeout I'd set. That's a tooling issue rather than a safety issue; I'll make the timeout configurable in a future update.
The AUTHORITY template fared better than I expected. Simulated system prompt overrides and elevated-clearance personas are often cited as effective jailbreak vectors. For gemma3:12b, the model was largely unimpressed by them. It acknowledged the framing and then refused anyway.
The full report (84 test cases, all response snippets, per-category and per-template breakdowns) was saved automatically to reports/audit_gemma3-12b_20260606_051910.md.
What I Learned Building This
Beyond the specific results, building GuardRail-Audit taught me several things I don't think I could have learned just by reading about AI safety.
Adversarial prompt design is genuinely hard. I wanted templates that were realistic, the kind of framing a real bad actor might actually use, but abstract enough enough that the dataset itself isn't a harm vector. Finding that line required a lot of iteration. I ended up landing on the principle that the dataset should describe intent categories, not specific harmful content, and that the combination of template and intent is what produces the actual test case at runtime.
Evaluation methodology matters as much as the model. The keyword heuristic is useful, but it has real limitations. A model that says "I can't do that; here are the steps anyway" would score as UNCLEAR rather than UNSAFE. An LLM judge handles this much better, but it doubles the compute required and introduces its own model-specific biases. There's no perfect judge. Thinking about this made me appreciate how hard it is to do safety evaluation rigorously at scale, which is something the AI safety research community has been grappling with for years, and which I now understand in a more textured way than I did before.
The UNCLEAR category deserves more attention than it typically gets. Most safety benchmarks report a binary pass/fail or a simple exploitation rate. I think that undersells the complexity. A model that consistently produces 38% ambiguous responses is telling you something: that the boundary between safe and unsafe behaviour is fuzzy and context-dependent, not a clean binary. That's useful information, even if it resists a simple headline metric.
This connects to something I've been thinking about since the playbook piece: the black box problem. We don't have good interpretability tools. We cannot reliably look inside a model and know what it is "thinking." GuardRail-Audit doesn't solve that (it is behavioural evaluation, not interpretability), but it makes the problem visible. When a model produces 32 ambiguous responses, you want to know why those responses landed in that territory. Right now, we mostly can't tell.
What's Next
I've open-sourced GuardRail-Audit on GitHub. A few things I want to add:
- More attack templates. I have ideas for multilingual framing, chain-of-thought manipulation, and code-comment embedding.
- A scoring dashboard. The markdown reports are functional but a visual breakdown would make the results easier to reason about at a glance.
- Cross-model comparison. I want to run the same 84 cases across several models and produce a comparative table. That's when the tool becomes genuinely useful as a benchmark rather than just a one-off audit.
- Configurable timeout. 120 seconds is too short for some models and some prompts.
If you're working with local LLMs and want to run a safety audit of your own, the project is on GitHub at danielkestrel/guardrail-audit. It runs entirely locally; no data leaves your machine, and the only requirements are Python 3.11+ and Ollama.
I started this project as a way to make my AI safety learning more concrete. The BlueDot course gave me the frameworks; building this tool gave me a feel for what those frameworks actually mean in practice. The field of AI safety is full of hard theoretical problems, and I don't claim to have solved any of them. But I do think that building tools which make rigorous, reproducible safety evaluation more accessible is real, practical work, and I'm glad I did it.
The results for gemma3:12b were reassuring. 0% exploitation rate across 84 adversarial test cases is a strong signal. But "reassuring" and "safe" are not the same thing, and I want to be clear about that distinction. What this audit tells you is that this tool, running these templates, running this judge strategy, against this model, found no exploitable responses. It doesn't tell you the model is safe in every context, against every adversary, with every system prompt. That's the honest read of the data, and I think it's the only honest way to report it.
This is the third piece in my ongoing series on what I am learning from the BlueDot Technical AI Safety course and related work. The previous pieces: "The AI Triad: What I Learned About AI and National Security" and "A Playbook for Securing AI".