Personally it's not the UX that I'm afraid of but when random drivers don't work or updates break things. Once upon a time I enjoyed learning and doing that kind of thing but nowadays I just don't have the time.

Admittedly I haven't installed Linux for a few years and don't even have my own computer now so maybe the situation has improved.