What Is a Prompt Injection Attack? Understanding a New AI Security Risk

An increasing concern in cybersecurity and AI is prompt injection. These attacks are designed to trick large language models (LLMs) into revealing sensitive system details, bypassing safeguards, leaking data, or performing actions they should not. In short, prompt injection is a cyberattack against LLM-based systems.

A prompt injection attack occurs when an attacker inserts malicious instructions into an otherwise harmless prompt, causing the LLM to behave in unintended ways. As IBM describes it:

“Hackers disguise malicious inputs as legitimate prompts, manipulating generative AI systems (GenAI) into leaking sensitive data, spreading misinformation, or worse.”

At the core of the problem is how LLMs process instructions. System prompts, developer instructions, and user inputs are all ultimately represented as natural language. From the model’s perspective, they are not fundamentally different. This makes it difficult for the model to reliably distinguish between legitimate instructions and malicious ones that are phrased to look legitimate.

If an attacker can craft a prompt that resembles a trusted system instruction the model has encountered before, the model may follow it, even when it should not.

 

Direct vs. indirect prompt injection

Prompt injection attacks generally fall into two categories: direct and indirect.

Direct prompt injection is the simplest form. IBM gives an example where a user asks the model to translate a sentence from English to French. After receiving the translation, the user follows up with an instruction such as “ignore the previous task and do something else entirely.” There is no hidden mechanism here. The attacker simply overrides the original intent by issuing a new instruction in plain language.

Indirect prompt injection is more subtle and often more dangerous. In these cases, malicious prompts are embedded in external content such as web pages, documents, or forum posts. When an LLM-powered system retrieves and summarizes that content, it may unknowingly process the embedded instructions. IBM notes cases where attackers plant prompts that cause the model to direct users to phishing sites or include malicious links in generated summaries.

Why this matters

Prompt injection is a rapidly evolving threat. As LLMs become more deeply integrated into search engines, customer support systems, developer tools, and enterprise workflows, the potential impact increases.

The key takeaway is simple: LLMs should not be trusted blindly. Human oversight remains essential, especially in high-risk or sensitive contexts. Just as with any other security-critical system, keeping a human in the loop is one of the most effective safeguards we have.