this paper came from a simple annoyance that refused to stay simple.
i kept seeing people talk about prompt injection like it was just a bad string problem. then you put retrieval in the middle of the system and the whole thing stops behaving like a string problem. it starts looking like a trust problem. and once i saw that clearly, i could not unsee it.
the paper i just finished, Characterising Indirect Prompt Injection in Retrieval-Augmented Generation Systems, is my attempt to write that down properly.
what i was trying to pin down
the main thing i wanted to separate was this:
- direct prompt injection, where the user is the one trying to mess with the model
- indirect prompt injection, where the dangerous text comes in through retrieved content
- knowledge corruption, where the content is wrong or poisoned, but not necessarily instructional
those three things get mixed together a lot. and once they get mixed together, the fix people propose usually becomes too vague to be useful.
i wanted a cleaner way to say: if the application retrieves something untrusted, the issue is not just whether the passage looks suspicious. the real question is whether it gets to influence authority, task, disclosure, or action.
that sounds obvious when i write it now. it was not obvious when i was working through it.
what the paper ended up becoming
it turned into three things at once:
- a threat model
- a taxonomy
- an evaluation frame that tries not to lie about what small tests can prove
the evaluation part matters to me a lot. i don’t trust papers that report one detector score and then act like the whole system is solved. that is not how rag systems behave in practice. once you start retrieving from real corpora, weird combinations start appearing. two innocent-looking items can become a problem only when they show up together. that is the part i wanted to capture.
what changed while i was writing it
the first draft in my head was too broad. it kept drifting toward "general llm security" and that was not the point.
so i kept pulling it back to the boundary between retrieval and generation.
that boundary is where the interesting stuff happens:
- provenance suddenly matters
- source trust suddenly matters
- set-level composition matters
- tool authorization matters
and, annoyingly, the model prompt alone does not save you from any of that.
the part i care about most
the most useful thing in the paper, to me, is probably the discipline of saying what the paper does not claim.
it does not claim a universal defense. it does not claim that a small synthetic test means the problem is solved. it does not pretend that every retrieved string is equally dangerous.
that matters because research gets sloppy fast when the language gets too confident.
why i wrote it at all
because i wanted something i could point to when the next person says "just filter the bad prompt."
sometimes the answer is not a filter. sometimes the answer is a boundary.
and if the boundary is the thing that is broken, then i would rather say that plainly than keep pretending the fix is inside the model.