The argument isnt that we are unique but that we are far, far away.
It is considered fake because the perceived difficulty of space travel combined with the complete lack of direct evidence despite the ongoing search for it.
Why are we so confident in our understanding of reality being so perfect nobody in the whole universe could have found a way to transcend the limitations it imposes? I would totally expect an advanced alien race to consider our physical science ridiculous.
It is illogical to build confidence on top of lack of evidence of your wrongness. In a virtually-infinite (relatively to your size and age and the limits of your senses and cognition) universe there always is some evidence you just don't know about. Our science is quite decent from our perspective, doing a great job describing the reality we perceive, but it seems absurdly naïve to consider it the ultimate knowledge. As history shows it changes regularly and will change a lot more someday in future.
It is considered fake because the perceived difficulty of space travel combined with the complete lack of direct evidence despite the ongoing search for it.