The feeling you need to explain yourself comes from where? I don't need an answer to that. But look within yourself, figure out why you feel you need to explain yourself. Did you feel unheard as a kid and now you need to justify your thoughts and needs? You're an adult now. You make the rules and boundaries surrounding your own life (within the confines of the law) with no other justification than I am.

You don't drink coffee... millions of people don't. I don't eat peanuts, I'm not allergic to them, I just don't like peanuts. When there's peanuts, I don't eat them. No justification required.

Not that I'm trying to project what I think you should do, but I will tell you, when you stop feeling like you need to justify everything (I went through this too), suddenly life feels a little easier... and life's pain in the ass enough without getting in your own way.

You don't have to explain yourself to anyone, just say "I don't like coffee, I'll get a tea/water/juice/etc". Easy—a lot of people don't. Whatever ideas about people thinking you're weird or disliking you because of that are false.

I don't get sugary coffees either. I just order a black coffee. If anyone asks (rarely happens) I just tell them I don't like sugary or sweet/caramel drinks (I don't).