Any site you grant this permission to could abuse it. The abuse is on the abuser, not the person granting permissions. Allowing the abusive behavior to happen at all - authorized or not - is on Github.