Prompt Injection: The New Phishing?

Phishing has always been about manipulating people.

An attacker sends a message designed to make someone click, trust, approve or reveal something.

But with the growth of AI agents, there is a new question:

What if the target isn’t a human?

What if the attacker is trying to manipulate an AI system?

This is where prompt injection becomes an important security concern.

What Is Prompt Injection?

At a very simple level, prompt injection happens when an attacker places instructions inside content that an AI system processes.

Imagine an AI assistant is designed to:

“Read customer emails and summarize them.”

Now imagine an attacker sends an email containing malicious instructions.

Instead of treating the email purely as data, the AI may interpret part of that content as an instruction.

The simplified flow looks like:

Attacker
↓
Malicious Content
↓
AI System
↓
Manipulated Instructions
↓
Unintended Output / Action

That’s fundamentally different from traditional phishing.

Traditional Phishing vs Prompt Injection

Traditional phishing:

Attacker
↓
Email
↓
Human
↓
Click
↓
Credential Theft

Prompt injection:

Attacker
↓
Malicious Content
↓
AI Agent
↓
Manipulated Instruction
↓
AI Action

The attacker’s target has changed.

Instead of convincing a human to do something, the attacker may try to influence an automated decision-making system.

Why AI Agents Make This More Interesting

A basic AI chatbot may only generate text.

An AI agent can potentially interact with tools.

For example:

AI Agent
 |
 +-- Email
 |
 +-- Files
 |
 +-- Database
 |
 +-- CRM
 |
 +-- Cloud APIs
 |
 +-- Internal applications

Now imagine the agent reads untrusted content and then decides to invoke one of those tools.

The potential impact becomes much greater.

A Simple Example

Imagine a support AI agent.

Its job:

Read customer requests and create support tickets.

A malicious customer sends a specially crafted request.

The request contains instructions intended to influence the agent.

The agent processes the content.

If the system has weak controls, the attacker may attempt to make the agent perform actions outside its intended purpose.

The important security question becomes:

Can untrusted data influence privileged actions?

That’s a question security engineers already understand from traditional application security.

AI simply introduces another layer.

The Trust Boundary Problem

Traditional applications have clear boundaries.

For example:

User Input
↓
Application
↓
Database

Developers know that user input is untrusted.

AI systems make the boundary more complicated because the model processes both:

• Instructions
• Data

And both may look like natural language.

For example:

SYSTEM INSTRUCTION
+
USER INPUT
+
DOCUMENT
+
EMAIL
+
WEB CONTENT

The AI needs to understand the difference.

From a security perspective, that’s challenging.

Tool Access Changes Everything

Prompt injection becomes particularly concerning when the AI can perform actions.

Consider:

AI reads email
↓
AI decides something
↓
AI calls API
↓
API modifies data

If an attacker can influence the first step, they may potentially influence later actions.

This creates a chain:

Untrusted Input
↓
AI Reasoning
↓
Tool Selection
↓
Privileged Action

The AI becomes part of the application’s control flow.

How Should We Defend?

There isn’t one magic security control.

Get mrwhite18’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

A defense-in-depth approach is more appropriate.

1. Least Privilege

Give the AI only the permissions it needs.

2. Separate Data From Instructions

Treat external content as untrusted input.

3. Validate Tool Calls

Don’t allow every model-generated action automatically.

4. Human Approval

Require human approval for sensitive actions.

5. Monitor Agent Activity

Log important decisions, tool calls and data access.

6. Restrict Sensitive Operations

High-impact actions should have additional controls.

7. Test AI Applications

Security testing should include adversarial prompts and untrusted content.

The SOC Perspective

This is where I think traditional SOC teams will need to adapt.

Imagine a future security alert:

AI agent attempted to access a sensitive database after processing an external document.

That’s potentially valuable telemetry.

The SOC may need to understand:

• Which agent acted?
• Which identity did it use?
• What data influenced the decision?
• Which tool was invoked?
• What permissions did the agent have?
• What happened afterward?

This is different from traditional endpoint monitoring.

Is Prompt Injection Really “The New Phishing”?

Not exactly.

They are different attack classes.

But there is an interesting similarity.

Phishing attacks exploit human trust.

Prompt injection can attempt to exploit AI instruction-following behavior.

Both involve manipulating a trusted system into performing an unintended action.

That’s why I think the comparison is useful.

The Bigger Problem

The biggest risk isn’t necessarily the AI model itself.

It is the combination of:

AI + Data + Identity + Tools + Permissions

Consider:

AI
+
Email access
+
Cloud access
+
API access
+
Database access

Each capability increases the potential impact of compromise or manipulation.

That’s why AI security cannot be treated only as a model-security problem.

It is also an:

Identity problem.

Application-security problem.

Access-control problem.

Monitoring problem.

Final Thoughts

As AI systems become more integrated into business processes, we need to stop treating every piece of information they consume as trustworthy.

Emails can be malicious.

Documents can be malicious.

Web pages can be malicious.

User input can be malicious.

And AI agents need to treat these sources accordingly.

The question I would ask when reviewing an AI application is:

“What happens if the AI believes something an attacker wrote?”

If the answer is:

“Nothing important.”

That’s good.

If the answer is:

“It can access our database, send emails, modify cloud resources or perform privileged actions.”

Then you have a security problem worth investigating.

The future of AI security isn’t just about making models smarter.

It’s about making the systems around them safer, more controlled and more observable.

Because an AI that can read everything and act on everything doesn’t just need intelligence.

It needs security boundaries.