It doesn't know what it's about. It just knows that their product was relevant to it. I don't think this is a big deal. It's like saying that if a user downloads a gacha game, then the game studio learns that the user is likely interested in gacha games. Learning that a user was talking about gacha games with ChatGPT does not really give any additional information.
Approach it from the other angle - what scenario would it be bad in. It's not hard to see very real possibilities in the short term where it does matter: A 16yro looks up on chatGPT how she can check discreetly whether she is pregnant or not and what potential avenues she has. The advertiser could literally by anyone targeting pregnancies, including government or action groups who now have some information about that user's conversations in this scenario.
Any data exfiltration or reporting on the users would quickly be developed by the industry to merge this information and improve inferences with confidence values on target populations/individuals.
Hard disagree. Advertisers (or people with worse motives) will be very creative in how they use the targeting parameters offered by chatGPT ads and suddenly they can make educated guesses about groups or even individuals. I remember a couple years ago, someone posted a story about how they were able to circumvent Facebook rules and display ads for just one person: their roommates and used that to freak them out.
