![](/assets/headers/header-logo.png)

![](https://img.shields.io/badge/Tools%20%26%20Resources%20Available-20-757575?style=for-the-badge)

### Evasion, Poisoning, Extraction

* [Adversarial Robustness Toolbox (ART)](https://github.com/Trusted-AI/adversarial-robustness-toolbox) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference. ![GitHub last commit](https://img.shields.io/github/last-commit/Trusted-AI/adversarial-robustness-toolbox?style=flat)
* [CounterFit](https://github.com/Azure/counterfit) - A CLI that provides a generic automation layer for assessing the security of ML models. ![GitHub last commit](https://img.shields.io/github/last-commit/Azure/counterfit?style=flat)
* [Foolbox](https://github.com/bethgelab/foolbox) - A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX. ![GitHub last commit](https://img.shields.io/github/last-commit/bethgelab/foolbox?style=flat)

### Jailbreaking

* [EasyJailbreak](https://github.com/EasyJailbreak/EasyJailbreak) - An easy-to-use Python framework to generate adversarial jailbreak prompts. ![GitHub last commit](https://img.shields.io/github/last-commit/JailbreakBench/jailbreakbench?style=flat)
* [JailbreakBench](https://github.com/JailbreakBench/jailbreakbench) - An Open Robustness Benchmark for Jailbreaking Language Models. ![GitHub last commit](https://img.shields.io/github/last-commit/JailbreakBench/jailbreakbench?style=flat)

### LLM & Generative AI Red Teaming

* [Garak](https://github.com/NVIDIA/garak) - The LLM vulnerability scanner. ![GitHub last commit](https://img.shields.io/github/last-commit/NVIDIA/garak?style=flat)
* [PromptBreach](https://github.com/microsoft/promptbench) - The Python Risk Identification Tool for generative AI. ![GitHub last commit](https://img.shields.io/github/last-commit/microsoft/promptbench?style=flat)
* [PyRIT](https://github.com/Azure/PyRIT) - The Python Risk Identification Tool for generative AI. ![GitHub last commit](https://img.shields.io/github/last-commit/Azure/PyRIT?style=flat)
* [Promptmap2](https://github.com/utkusen/promptmap) - A security scanner for custom LLM applications ![GitHub last commit](https://img.shields.io/github/last-commit/utkusen/promptmap?style=flat)
* [Purple Llama](https://github.com/meta-llama/PurpleLlama) - Set of tools to assess and improve LLM security. ![GitHub last commit](https://img.shields.io/github/last-commit/meta-llama/PurpleLlama?style=flat)

### Model Extraction & Inference Attacks

* [Privacy Meter](https://github.com/privacytrustlab/ml_privacy_meter) - Audit data privacy in statistical and machine learning algorithms. ![GitHub last commit](https://img.shields.io/github/last-commit/privacytrustlab/ml_privacy_meter?style=flat)
* [SecretFlow](https://github.com/secretflow/secretflow) - A unified framework for privacy-preserving data analysis and machine learning. ![GitHub last commit](https://img.shields.io/github/last-commit/secretflow/secretflow?style=flat)
* [ShadowAttack](https://github.com/hncszyq/ShadowAttack) - Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon. ![GitHub last commit](https://img.shields.io/github/last-commit/hncszyq/ShadowAttack?style=flat)

### Data Poisoning & Supply Chain Attacks

* [Backdoor Box](https://github.com/THUYimingLi/BackdoorBox) - A universal pytorch platform to conduct security researches of image classification in deep learning. ![GitHub last commit](https://img.shields.io/github/last-commit/THUYimingLi/BackdoorBox?style=flat)
* [TrojanZoo](https://github.com/ain-soph/trojanzoo) - The open-sourced Python toolbox for backdoor attacks and defenses. ![GitHub last commit](https://img.shields.io/github/last-commit/ain-soph/trojanzoo?style=flat)

### LLM Safety & Guardrails

* [Alibi Detect ](https://github.com/SeldonIO/alibi-detect) - Algorithms for outlier, adversarial and drift detection. ![GitHub last commit](https://img.shields.io/github/last-commit/SeldonIO/alibi-detect?style=flat)
* [Guardrails AI](https://github.com/guardrails-ai/guardrails) - Adding guardrails to large language models. ![GitHub last commit](https://img.shields.io/github/last-commit/guardrails-ai/guardrails?style=flat)
* [LLM Guard](https://github.com/protectai/llm-guard) - A comprehensive tool designed to fortify the security of Large Language Models. ![GitHub last commit](https://img.shields.io/github/last-commit/protectai/llm-guard?style=flat)
* [NeMo Guardrails](https://github.com/NVIDIA-NeMo/Guardrails) - Toolkit for easily adding programmable guardrails to LLM-based conversational systems. ![GitHub last commit](https://img.shields.io/github/last-commit/NVIDIA-NeMo/Guardrails?style=flat)
* [Vigil](https://github.com/deadbits/vigil-llm) - Detect prompt injections, jailbreaks, and other potentially risky Large Language Model inputs. ![GitHub last commit](https://img.shields.io/github/last-commit/deadbits/vigil-llm?style=flat)