Published: June 07, 2026, 13:25 UTC
NVIDIA published a comprehensive tutorial this week for garak, its open-source LLM vulnerability scanner, walking security teams through building a complete defensive red-teaming workflow from scratch. The tutorial, published by MarkTechPost, covers everything from basic setup to creating custom probes and detectors.
What garak does
Garak is an open-source framework developed by NVIDIA for testing the security of large language models. It works by running probes against a target model and using detectors to determine whether an attack succeeded. The framework includes dozens of built-in plugins and thousands of test prompts covering jailbreak techniques, prompt injection, encoding attacks, and other adversarial approaches.
The tool operates on four plugin types: probes, which define the attack to run; detectors, which analyze model outputs to determine if the attack succeeded; generators, which define the target model; and buffs, which transform inputs before they reach the model.
What the tutorial covers
The tutorial, authored by Sana Hassan, builds an end-to-end workflow in Python. It starts with installation and environment setup, then moves through five stages:
Plugin inventory — listing available probes, detectors, generators, and buffs to understand the tool’s capabilities.
Dry run — running a fast test against a simulated model to confirm garak is working without requiring an external model or API key.
Real model scan — testing a Hugging Face model (GPT-2) against the DAN 11.0 jailbreak probe, a well-known “Do Anything Now” prompt injection technique.
Multi-probe evaluation — running several probes simultaneously to generate a richer report for analysis, using the garak CLI programmatically.
Report analysis and custom plugin creation — parsing garak output for safety scores and attack success rates, then writing custom probes and detectors for proprietary or domain-specific attack scenarios.
The tutorial also covers exporting results in AVID format, an open standard for AI vulnerability disclosure.
Why it matters now
LLM security has become a pressing concern as organizations deploy AI agents with tool access, database connections, and the ability to act on behalf of users. The same prompt injection techniques that garak tests for have been demonstrated against production systems, including the self-replicating AI worm shown by University of Toronto researchers this week.
Garak’s open-source nature means any organization can run these tests without relying on a vendor’s security assessment. The tutorial is designed to make the framework accessible to security engineers and ML teams who may not have deep red-teaming experience.
The broader picture
NVIDIA’s investment in garak reflects a growing recognition that LLM security testing cannot remain an afterthought. As models move from chatbots to autonomous agents, the attack surface expands dramatically. A compromised agent with tool access can read sensitive data, execute commands, or manipulate other systems.
The tutorial’s emphasis on custom probes and detectors is particularly notable. It signals that NVIDIA expects organizations to need domain-specific tests, not just generic jailbreak scans. A healthcare AI needs different security testing than a financial services AI, and garak’s plugin architecture lets teams build those tests themselves.
Garak is available on GitHub under an open-source license.
Sources: MarkTechPost (June 6, 2026); NVIDIA garak (GitHub); garak.ai