I wonder if this has to do with the spatial aspect. Americans use that awkward imperial system, which makes it really hard to easily chop up distances.
Almost all other countries use the metric system, which makes it very easy to understand dividing lengths.
Remember that for ordering and numbers many people use a 2-dimensional line to represent things in their head.
The splitting out over more units is a little awkward, but it is usually possible to quickly divide imperial units by 2,3 and 4 (of course it is easy to divide decimals by 2 and 4, my point is that imperial doesn't lose this).
For people frequently working with medium distances, all they have to do is learn that a mile is 80 chain (for longer distances, decimal miles are about equivalent to decimal km).
Remember that for ordering and numbers many people use a 2-dimensional line to represent things in their head.