Only mostly so. It's latency sensitive in that your eye and head movements need very fast responses to avoid nausea in lots of people and produce a good experience. It's not latency sensitive in that a video stream needs to react to you at 90-120hz kinds of speeds. As long as the position and rotation of the video reacts and moves with the rest of the environment, the application itself won't need to respond any quicker than it already does outside vr.

Now I think you're thinking of running the vr application that's doing all the rendering remotely and strraming thst to the headset, it would definitely run into issues because of the latency.