I think that may be the shitty economy more than the BS in Physics. When all the on-campus recruiters were visiting my senior year in 2004, I'd go to these info sessions for investment banks or consulting firms and they'd ask my major, and then when I said "Physics", they were like, "Oh, you should have no problem. I was just worried you were an English major or something."

I ended up switching to CS my last semester, so nobody knows I was a physics major for most of my college career and that basically all of my programming knowledge is self-taught, but I've never felt at a disadvantage to all the folks who bulked up on CS courses in undergrad.