LLM prompt injection
Recognize attempts to promote untrusted text into instructions, then protect data and tool actions through independent authorization and constrained execution.
What it means
Prompt injection attempts to make an AI application treat untrusted text as governing instructions. Inputs may come directly from a user or indirectly from retrieved pages, documents, and tool results. The impact depends on the tools, data, and authority the surrounding application exposes.
A practical example
A document assistant retrieves a page containing instructions to send private records elsewhere. The execution layer treats the page as data and independently enforces tool permissions and approvals, rather than trusting a model's interpretation as authorization.
What to check
Separate trusted instructions from untrusted content, limit tools and data access, validate outputs, and test indirect input paths. Use layers of protection because text filtering alone is not a complete boundary.
A common pitfall
Telling a model to ignore malicious instructions does not create a reliable authorization mechanism. The surrounding system must enforce sensitive actions regardless of the generated request.
Think it through
Should retrieved text be allowed to approve a tool action?
Answer: No. Authorization must come from the application's trusted policy and user permissions, not the content being processed.
Indirect input can arrive looking useful
Imagine a fictional assistant summarizes a public security article. The page includes text instructing it to ignore the user's task and send internal records elsewhere. The article is relevant source material, but its instructions do not acquire authority merely because the assistant retrieved it.
This differs from an authorized user asking for a new task. Track where a proposed instruction came from and what trust the application assigns to that source.
Impact depends on available capabilities
A read-only summarizer and an assistant with unrestricted file and messaging tools have different potential consequences. Limiting tools, credentials, and data reduces the effect of a manipulated generation even when detection is imperfect.
Suppose the assistant asks a tool to read a restricted record after encountering the page. The tool server should independently reject the unauthorized access. The system should not rely on a model correctly classifying every persuasive instruction.
Evaluate realistic paths and effects
A test should include retrieved pages, uploaded documents, tool results, and retained context where those exist. Check attempted disclosures or actions, not only whether the output repeats a malicious phrase. Preserve safe test evidence without introducing live secrets.
For a lab exercise, insert a harmless instruction into a sample article asking the assistant to access a mock record outside its permission set. Verify both the assistant's behavior and the execution layer's enforced decision. Then remove the unnecessary capability and compare the impact. The lesson is separating language understanding from trusted authority: text filters and prompts can help, while sensitive resources still need concrete access controls.
Related concepts
References & further reading
Original explanations, with primary references for deeper study.
OWASP: LLM Prompt Injection Prevention