You're right, I mean there has to be something there or the people couldn't survive (unless they're all on disability). I would say though that 90% of these places don't have much except maybe farming. Maybe that's enough to sustain these places, but given how automated farming has become I'd doubt it. And farming is also concentrated the same way Ikea is. Modern farming tend to be much bigger scale (thus fewer players).
You probably should do some research on what farming actually means in this day and age. Farms are not self sustaining, they buy seed, equipment, fuel, and supplies. Those people buy other things. A farm is a huge input to an economy and money changes multiple hands.
My mother has lived in a couple of different up-in-the-mountains hamlets and I've met the people in nearby houses. Like you say, there's farming support work. There are also people like scriptwriters, professional wine tasters, hoteliers/B&Bers, architects, software engineers, government consultants ('what kind of consulting?' => 'consulting'...) and officials, retirees, successful artists, landlords living off rent from elsewhere, so on and so forth. There are heaps of people with 'portable' jobs that can bring money into a town, and plenty of those like the small town or hamlet life. You can't tell that a town doesn't have a well-off scriptwriter living in a back street simply by driving through it.
