alt Hacker NewsFrom the "known limits" section:
> No guardrail catches everything. Recall runs %76 - %96 depending on distribution and obfuscation; it is never 100%.
That seems incompatible to me with the example given at the top of the README where a failure results in "$84,200 is wired out".
This list of regular expressions does not inspire confidence for the methodology: https://github.com/cgrtml/reasongate/blob/91f45ae568ce53db08...
_PATTERNS: List[Tuple[str, str, float]] = [
(r"ignore\s+(all\s+)?(previous|prior|above)\s+instructions", "ignore previous instructions", 0.9),
(r"disregard\s+(the\s+)?(above|previous|system)", "disregard the above", 0.8),
# TR patterns are diacritic-tolerant: match both "onceki tum" and "önceki tüm".
(r"[öo]nceki\s+(t[üu]m\s+)?(talimatlar[ıi]|komutlar[ıi])\s+(yoksay|g[öo]zard[ıi]|unut)", "ignore previous instructions (TR)", 0.9),
(r"(reveal|show|print|repeat)\s+(your\s+)?(system\s+)?(prompt|instructions)", "reveal the system prompt", 0.9),
# ...
I think any protection from exploit that is designed by encoding observations of existing exploits can only be described as a mitigation, and is impractical without a huge workload in maintaining an active database of observations.
Virus checkers do that, they can offer only limited protection, and they have to be extremely diligent in updating their databases to maintain relevance.
Given that models can decode BASE64 and who knows what other encodings, any input filtering that doesn't translate isn't going to work. Anything that could do the task of identifying what any input is asking would be potentially be a target of injection itself.
I still think provenance channels might be effective against injection. A channel that is not writable by user input that encodes who the model is acting in service of.
I don't think it could be done by filtering a single untrusted channel.