giskard

Practical Jupyter notebooks from Andrew Ng and Giskard team's "Red Teaming LLM Applications" course on DeepLearning.AI.

✍️ Collaborate on writing technical content for the Giskard Community

🎓 As part of a course project with a classmate, we explored prompt injection attacks by directly interacting with large language models (LLMs) to understand their vulnerabilities. We also presented how the open-source tool Giskard can help secure these models through automated testing and vulnerability detection.

Mutant is a red team and behavioral data generation library for LLMs, RAG pipelines, and AI Agents. It provides two powerful capabilities

Adversarial LLM red-teaming with Giskard: automated vulnerability scans of DeepSeek-R1, GPT-4o-mini & Llama 3.2 (prompt injection, hallucination, harmful output) plus tabular ML scanning, with per-model HTML reports

A RAG system to answer questions about flights

Develop a system to generate diverse types of subjective questions from PDFs and automate the grading of answers.

Multi-agent LLM-driven SOC pipeline (n8n + Ollama), adversarially red-teamed against the CSA Agentic AI Red Teaming Guide.

Different approaches to evaluate RAG !!!

Security testing toolkit for AI agents

The complete pipeline for a fine-tuning of a classifier model.

Independent SR 11-7-style model validation of an ML trading strategy - deepchecks + giskard, formal report, reproducible. Verdict: not approved for deployment.

PRML pre-registration for Giskard scenario results: commit an eval claim to a SHA-256 before the run, then verify the result against it.