Retrieval-augmented generation feeds retrieved text straight into the model. If any source in your index is attacker-writable — a wiki, a ticket system, a scraped web page — that text can carry instructions that reach the model as trusted context.
Where poisoning enters
- User-editable sources indexed without provenance (support tickets, wikis, forums).
- Scraped web content that includes hidden or agent-only instructions.
- Documents uploaded by untrusted users.
Controls
- Track provenance for every chunk; label trusted vs untrusted sources and weight accordingly.
- Separate the instruction channel from the data channel in your prompt structure.
- Strip markup, hidden characters, and zero-width content before indexing.
- For high-trust actions, require the model to cite a trusted source, not just any retrieved chunk.
RAG turns your corpus into part of your prompt. Anything that can write to the corpus can influence the answer — govern it like input, not like storage.