That's funny, you got me! It comes down to what we mean by know and perception.
If by know we mean prove, then we can know things in other universes that may or may not exist by creating axiomatic systems that don't match our own. If we mean know as believe to the core (such as believing the mechanics of a proof) than we can know many things, some which may be true but unprovable or may simply be false.
I can say I'm aware of neutrinos and believe they exist as formulated, but couldn't say they absolutely exist in our universe. Earlier models only turned out to be approximations.
To extend your argument, nothing we perceive is _exactly_ how it exists in the universe. Everything that comes into our senses is an approximation, whether directly or via technological instrumentation. I don’t think any of that contradicts the possibility that consciousness is some state of matter.