Sorry, but you gave them permission to do it.

Why are people blaming GitHub?

Making it easy and frictionless for developers to build GitHub integrations is what a good developer platform should do.

You've learned a lesson to not just blindly click through an application requesting permissions to your account.