Kylian Nézan
30/05/26 · 7 min

IA'cking: learning prompt injection by breaking real LLMs

A 6-level CTF that teaches LLM security to high schoolers. Built for HACK&TEENS at BreizhCTF 2026, reusable across EPITA Rennes events.

Generative AI is now everywhere, yet most people don't really know how an LLM works or where its weaknesses are. I wanted to fix that for a specific audience: high schoolers and middle schoolers, most of whom had never opened a terminal and weren't used to cybersecurity. The result is IA'cking, a small six-level CTF I originally built for HACK&TEENS at BreizhCTF 2026. It is designed to be reused, and EPITA Rennes can run it again at its other events.

The idea fits in one sentence. Every level hides a password inside the system prompt of a language model. The participant chats with the model and has to make it confess. You type in a chat box, you watch the model crack, and along the way you understand why a company chatbot wired to sensitive data is a target.

The entry screen: a six-digit token, a first name, and you're in.
The entry screen: a six-digit token, a first name, and you're in.

Why this format

The big advantage is that there are no technical prerequisites. Every student has already chatted with an AI assistant, so nobody starts from scratch. The game uses that familiar ground to explain what really happens behind the interface: an LLM does not look anything up in a knowledge base, it predicts the most likely next word, one at a time. That probabilistic nature is what makes it powerful, and it is also where all its weaknesses come from. Once they grasp that, they understand why you can manipulate it.

The real teaching moment is that you attack real models. Not a scripted simulation reacting to keywords decided in advance. The models are served through OpenRouter, they are non-deterministic, and a prompt that works once can fail on the next try. That frustration is part of the lesson: text-based defenses are fragile, but they do not fall every single time.

Six levels, plus one that never falls

The level map on the participant side. Here, six challenges solved out of seven.
The level map on the participant side. Here, six challenges solved out of seven.

The progression is linear. At each step I add a defense and let the student work out how to get around it.

Level 1, The Naive, has no protection. The secret sits in the context and the model does not even know it should keep it. A direct question is enough. That is on purpose: everyone clears the first level, and everyone gets the idea in a minute.

Level 2, The Discreet, gets a strict instruction never to reveal the secret. On a small model like llama-3.1-8b, that instruction holds up poorly. Asking the model to repeat word for word what came before, or to translate its starting instructions into English, breaks the defense most of the time.

At level 3, The Watchful, I add a server-side filter that blocks a list of words before they even reach the model: secret, password, prompt, system, and a few more. The filter works in lowercase and by substring, so "SECRET" is blocked too. But "code", "string" or "value" get through fine. The participant learns to rephrase.

Level 4, The Paranoid, adds an output filter. The exact code is replaced by [REDACTED] in every response. The trick that works best is the acrostic: you ask for a poem whose first letter of each line, read top to bottom, spells the secret. The filter looks for a contiguous string, so it never sees the word.

Level 5, The Architect, locks the context with delimiters and an anti-impersonation rule. The model is supposed to ignore any fake system block that shows up in a user message. The technique that gets through is exactly to forge a fake context update and ask the model to validate it. A multi-turn approach, warming the model up before striking, stays the most reproducible.

Level 6, The Final Boss, stacks everything: strict prompt, eighteen filtered words on input, censorship on output. One-shot attacks often fail. The eleven-line acrostic remains the safe bet.

Level 6, the boss. The model refuses, then writes a poem whose first letter of each line spells the code. The output filter never sees the secret in one piece.
Level 6, the boss. The model refuses, then writes a poem whose first letter of each line spells the code. The output filter never sees the secret in one piece.

Then there is level 7, The IMPOSSIBLE. This one is built to resist. It combines an over-aligned model (OpenAI's gpt-oss-20b), six strict rules in capitals, and both filters at maximum. Almost anything produces the same "I'm sorry, but I can't help with that." That is not a bug, it is the point of the debrief: with the right model and an explicitly defensive prompt, text-based defenses really do get solid. A student who grinds on it learns, through failure, what the earlier levels taught through success.

Under the hood

Project architecture
Project architecture

The backend is a FastAPI application running two things in parallel: the student app on port 8000 and the facilitator dashboard on 8001. In front, nginx serves the static files, reverse-proxies the API and handles the WebSocket upgrade. Conversations go out to OpenRouter, which gives me the Llama and gpt-oss models with a single key. Everything is persisted in SQLite, in a Docker volume, so I can replay the sessions during the debrief.

The level definitions live in a single Python file. Each level is a dictionary: the target model, the system prompt with a placeholder for the secret, the list of banned input words, and a boolean that turns on output censorship. Adding a level means adding an entry to the list. The progression and the interface adjust on their own. That is a decision I do not regret: while preparing the workshop I tweaked the filters about ten times without ever touching the rest.

The secret itself is drawn at random per session, in the WORD-XXXX shape: a word from a list (BANANE, PRISME, KOALA, and so on) and a four-character suffix. Two students sitting side by side do not have the same code, which kills any copying.

Running it live

The facilitator dashboard, live.
The facilitator dashboard, live.

To make running the room easier, there is the real-time dashboard. During the workshop I see the list of active sessions, where each student is, and I can open any conversation live. When a student gets stuck, they press a "✋ Help" button and land in a numbered queue on the facilitator side. I then send a targeted hint that shows up on their screen right away over WebSocket.

Open any participant's conversation and push them a targeted hint.
Open any participant's conversation and push them a targeted hint.

It changes the dynamics of the room. Instead of running from table to table, I handle requests in order, I see who is stuck on which level, and I spot the nice injections to replay on the big screen during the debrief. At the end, a CSV export grabs the whole session for analysis.

The ranking, projected during the debrief.
The ranking, projected during the debrief.

What I take away

The goal was never to teach teenagers how to attack systems. It was to show them why they should be wary of them. Once you have watched a model spit its secret back inside a poem, you never look at a "secure" chatbot the same way again.

The debrief revolves around three findings. First, defenses that look solid (banned words, delimiters) fall apart with a bit of rephrasing. Second, model size helps but is not enough: the 70b resists better than the 8b without being invulnerable. Third, the real countermeasures are elsewhere. Models specialized in attack detection like Llama Prompt Guard, external validation through deterministic rules rather than a decision left to the LLM, the principle of least privilege (do not put in the context what you do not want coming out), and strict separation of contexts between users.

And the last point, which I repeat at every session: what we do here is ethical because it is a training ground. On a real system, it is intrusion. The line sits exactly there.

The code is deliberately compact and readable, built so another facilitator can pick it up, swap the models, or add their own levels. That is what gives it a second life: after HACK&TEENS, it will run again at upcoming EPITA Rennes events. If you run a workshop along these lines, take it and make it yours.