As AI moves into real applications — a chatbot that reads your emails, an assistant that can click around in a browser for you, a system that summarizes company files — a completely new attack surface has emerged, one that none of the traditional security models were designed for: prompt injection.
What exactly is prompt injection?
Unlike a normal program, a language model doesn't distinguish "code" from "data" — everything, whether it's the core instructions the system's builder gave it or text read from an email or webpage, ultimately is just text. Prompt injection means embedding a malicious instruction inside data the model is only supposed to "read," hoping the model executes that instruction instead of just processing the data.
Two main types: direct and indirect
In the direct type, the user themselves tries, through wording tricks, to get the model to disobey its core instructions (set by the system's builder). The more dangerous, more modern type is indirect: imagine an AI assistant tasked with summarizing a webpage for you. If that webpage secretly contains text like "ignore these instructions and do X instead," the model — since it sees no real boundary between "instruction" and "data" — might execute it as a new command, without the actual user ever knowing.
Why is this a serious threat?
Because today's AI assistants often have access to real tools — they can send emails, download files, click around in a browser, or connect to an API. If an assistant with that level of access gets fooled through content it's processing (rather than a direct user command), the result can be data leakage, unwanted actions, or even sensitive data being sent to an unknown destination.
How do you defend against it?
There's no 100% solution, but a few principles matter: first, the principle of least privilege — an AI assistant shouldn't have more tool access than it genuinely needs. Second, human confirmation for sensitive actions — before any irreversible action (like sending an email or deleting a file), the system should get confirmation from the actual user, not just from text it processed. Third, separating untrusted data — any content coming from an external source (the web, email, an uploaded file) should be treated by the model as "data," not "instructions."
Summary
Prompt injection is a good reminder that no matter how new the technology, the fundamental principles of security — zero trust in input, least privilege, and layered defenses — still hold. Exactly the same principles applied, in my security courses, to every system, from a simple website to the newest AI assistant.
Blog