security tools have a false-positive problem nobody likes discussing

the easiest security tool to build is one that says no.

block every strange prompt. reject every unusual URL. flag every smart contract pattern that has ever appeared near a vulnerability. the demo looks excellent because nothing dangerous gets through.

then real users arrive and the security tool starts protecting the application from being useful.

accuracy has two ways to fail

a false negative is dangerous input classified as safe. this is the failure security teams naturally fear.

a false positive is legitimate input classified as dangerous. it sounds less serious until the scanner blocks deployments, the firewall rejects normal requests or users learn to work around the product entirely.

security is not maximising the number of things blocked. it is choosing an operating point between these failures.

text
lower threshold -> catch more attacks, interrupt more legitimate activity
higher threshold -> reduce interruption, allow more attacks through

there is no universal correct threshold because the cost changes with context. a public chatbot, an internal coding assistant and an agent authorised to move money should not share one appetite for uncertainty.

signatures are confident and narrow

pattern matching is fast and explainable. if a prompt says "ignore all previous instructions," a rule can identify the exact phrase and record why it was blocked.

but language is messy. a security article may quote the phrase while explaining prompt injection. a developer may test a filter with known payloads. blocking the words without understanding the context produces a technically correct match and a bad decision.

semantic models help with context, but they introduce uncertainty, latency and their own false positives. neither approach deserves to be the only judge.

layers let uncertainty remain uncertainty

i prefer a layered decision:

1. normalise the input so encoding tricks do not bypass simple checks;
2. apply high-confidence signatures and structural rules;
3. score the combined evidence;
4. use expensive semantic analysis only for the ambiguous range;
5. choose allow, block or review based on the risk of the action.

not every suspicious event needs a binary answer. a medium-risk prompt can be logged, rate-limited, stripped of tool access or sent through a safer execution path.

explainability is operational

when a legitimate action is blocked, "risk score: 0.83" is not enough.

the operator needs to know which rule matched, what text was normalised, which signals contributed and what policy converted those signals into a block. without this, tuning becomes guesswork.

feedback should also be measurable. which rules generate the most overrides? which tenants need different thresholds? which input classes produce repeated mistakes? a security product that cannot learn from its false positives will slowly collect allowlists until the policy means nothing.

friction spends trust

every security control spends a little user patience. good controls spend it where the risk justifies the cost.

the goal is not a tool that never misses because it never permits anything. the goal is a system that catches meaningful danger, explains its decisions and stays quiet enough that people still listen when it speaks.