You can give a medically informed body that understands the issues much more deeply some authority over public health policy, while still being able to think about those issues if you're interested. They wouldn't make all decisions for you, just those that have such a broad and important impact to society as a whole that irresponsible individuals wouldn't make choices that could have a net effect of significantly harming others like skipping vaccinations does.

Perhaps we could call that authority "civilization."