AI Security Tools
The open-source toolkit for attacking and defending AI systems, ranked by GitHub stars. Grouped by what they do — 13 tools.
Scanners
Probe models & apps for vulnerabilitiesRed-Team
Attack & test LLM systemsDefense
Guardrails, filters, input/output scanning- guardrailsguardrails-aiAdd programmable guardrails to LLM outputs7.2k
- GuardrailsNVIDIA-NeMoToolkit for adding programmable rails to LLM conversations6.9k
- llm-guardprotectaiSecurity toolkit for LLM interactions — input/output scanning3.2k
- rebuffprotectaiPrompt-injection detector for LLM applications1.5k
Benchmarks
Robustness datasets & adversarial libraries- cleverhanscleverhans-labAdversarial-example library for benchmarking ML robustness6.5k
- adversarial-robustness-toolboxTrusted-AIAdversarial ML — evasion, poisoning, extraction defenses6.2k
- llm-attacksllm-attacksUniversal & transferable adversarial attacks on aligned LLMs4.8k
- jailbreakbenchJailbreakBenchOpen robustness benchmark for LLM jailbreaking641
Star counts from the GitHub API, snapshotted at build time. Inclusion is editorial — a curated set, not an exhaustive index.