...and my last worry is people subverting the prompt to ask "stupid" questions - I send the prompts to a moderation API and simply block invalid requests.
Folks, we have solutions for these problems and it's always going to be a cat and mouse game.
"There is no such thing as perfection" (tm, copyright and all, if you use this quote you have to pay me a gazzilion money)
If the only thing you're building is a chat app, and the only thing you're worried about is it swearing at the user, then sure, GPT is great for that. If you're building a Twitch bot, if you're building this into a game or making a quick display or something, then yeah, go wild.
But people are wiring GPT up to real-world applications beyond just content generation. Summarizing articles, invoking APIs, managing events, filtering candidates for job searches, etc... Greshake wrote a good article summarizing some of the applications being built on top of LLMs right now: https://kai-greshake.de/posts/in-escalating-order-of-stupidi...
Prompt injection really heckin matters for those applications, and we do not have solutions to the problem.
Perfection is the enemy of the good, but sometimes terrible is also the enemy of the good. It's not really chasing after perfection to say "maybe I don't want my web browser to have the potential to start trying to phish me every time it looks at a web page." That's just trying to get basic security around a feature.
...and my last worry is people subverting the prompt to ask "stupid" questions - I send the prompts to a moderation API and simply block invalid requests.
Folks, we have solutions for these problems and it's always going to be a cat and mouse game.
"There is no such thing as perfection" (tm, copyright and all, if you use this quote you have to pay me a gazzilion money)