I'm just wondering, given that everyone and their uncle want to build apps on top of LLM, what if a "rebellion" group targets those apps using prompt injection?
They don't want to steal data or kill people (if they do, it's collateral). They just want to make people/gov't distrust LLMs/AI, thus putting a brake on this AI arms race.
Right now most of these tools are focused on servicing you. In that case it's not really that interesting to show someone "look, I managed to intentionally use this tool to get an incorrect answer". That's a relatively easy thing to do with any tool and not really all that interesting, beyond showing people any genuine misunderstandings about what the tool does.
Any apps that are focused on interacting with 3rd parties directly will be in a tough area though. It's a bit like intentional RCE except less rigid playbooks.
They don't want to steal data or kill people (if they do, it's collateral). They just want to make people/gov't distrust LLMs/AI, thus putting a brake on this AI arms race.
Not implying anything.