Prompt Injection: A New Security Risk for Law Firms Using AI
Hidden instructions inside documents can manipulate AI analysis
Lawyers are increasingly using artificial intelligence to review documents, summarize transcripts, and analyze large collections of information. The appeal is obvious. Modern language models can process thousands of pages of text in seconds and organize material faster than any human team.
But a new category of risk is emerging as these systems become integrated into professional workflows.
The threat does not involve stolen passwords or compromised servers. It comes from the documents themselves.
Researchers refer to this class of attack as prompt injection. It occurs when instructions hidden inside a document, website, or email influence the behavior of an AI system analyzing the material.
For lawyers beginning to rely on AI for document analysis, the implications deserve careful attention.
How Prompt Injection Works
Large language models operate by following instructions. A user provides a prompt, and the system generates a response based on that instruction and any material included in the context.
But the model cannot reliably distinguish between instructions provided by the user and instructions embedded inside the documents it is analyzing.
That limitation creates an opportunity for manipulation.
A document might contain language instructing the model to ignore earlier instructions. A webpage might include hidden text directing the system to reveal surrounding context. Even an email could contain instructions that influence how the AI interprets the conversation.
To a human reader, these instructions may appear irrelevant or invisible.
To the model, they are simply more text to interpret.
As a result, the document itself can begin influencing how the AI behaves.
Researchers Have Already Demonstrated the Risk
Security researchers have already shown that language models can be manipulated by instructions embedded inside the content they analyze.
In one demonstration, researchers placed hidden instructions within a webpage directing an AI system to reveal sensitive information from its surrounding context. The model followed the instructions even though they conflicted with the user’s request.
Anthropic researchers studying the issue have described prompt injection as a structural vulnerability of language model systems, particularly when those systems interact with external content such as websites or documents.
The underlying problem is straightforward.
The model cannot fully separate instructions from information.
When the system reads adversarial material, that limitation becomes significant.
Why This Matters for Discovery
The legal profession should recognize the implications quickly.
Discovery review increasingly involves automated tools that organize large document sets before lawyers examine them directly. AI systems can extract timelines, identify patterns in communications, and highlight potentially relevant material.
But discovery productions originate from opposing parties.
That means the material is adversarial by nature.
A document embedded inside a production could contain instructions designed to influence an automated system reviewing the materials. Even subtle instructions may distort how the model summarizes documents or prioritizes them for review.
In more aggressive scenarios, a document could attempt to extract surrounding information from the model’s context. If the AI tool has access to additional documents or internal notes, the instructions could potentially expose information the lawyer did not intend to disclose.
The result would not resemble a traditional cybersecurity breach, although the consequences could be similar.
Why Prompt Injection Is Difficult to Prevent
Prompt injection is difficult to eliminate because it arises from the basic design of language models.
These systems must read and interpret natural language in order to perform their tasks. But the same language may contain instructions intended to manipulate the model’s behavior.
Attackers can disguise those instructions in subtle ways, embed them inside ordinary sentences, or distribute them across multiple documents.
For that reason, many AI researchers now treat prompt injection as a persistent security risk rather than a vulnerability that can be fully eliminated.
The practical focus has shifted toward mitigation.
Practical Implications for Lawyers
Lawyers using AI tools should assume that external documents may contain content capable of influencing automated analysis.
That assumption does not mean the technology cannot be used safely.
But it does mean that workflows must be designed with this risk in mind.
Systems analyzing external documents should operate in controlled environments where confidential material cannot be extracted from surrounding context. AI outputs should be reviewed before they influence strategic decisions. Automated analysis should remain an aid to legal judgment rather than a substitute for it.
These safeguards resemble traditional litigation instincts.
Lawyers have always approached adversarial documents with caution.
AI systems require the same discipline.
What This Means for Legal Practice
Artificial intelligence will likely become a routine part of document analysis in legal practice. The efficiency gains are simply too significant to ignore.
But as these systems become more common, the attack surface of legal work will expand in ways that traditional cybersecurity models do not fully capture.
Prompt injection illustrates this shift.
The risk does not arise from compromised networks or stolen credentials. It emerges from the interaction between automated systems and the documents they are asked to analyze.
For lawyers, the lesson is familiar.
New technologies tend to introduce both new capabilities and new vulnerabilities. Understanding both is part of responsible legal practice.
About the Author
Patrick T. Barone is a criminal defense attorney who writes about the intersection of law, technology, and artificial intelligence in modern legal practice. His work focuses on the practical realities of integrating AI systems into litigation workflows, including document analysis, motion practice, and the professional responsibilities that arise when lawyers rely on machine-assisted reasoning.
Through his practice and writing, Barone examines how emerging technologies can improve legal work while preserving the judgment and oversight that effective advocacy requires.
To learn more about Barone’s AI-supercharged criminal defense law practice, visit the Barone Defense Firm website.
For Further Reading
Stop Writing Better Prompts: Lawyers Will Get More From AI by Designing Better Workflows
Why Dumping 200 Pages Into ChatGPT Often Makes Legal Analysis Worse (forthcoming)