I don't like these sort of comparisons, because the density (and a lot of other things) depends a lot on where you place the boundaries of the city, and that's very arbitrary.
For instance your figure for Paris is only the actual city of Paris, which doesn't include its own suburbs: it's really just the center. The subway extends a fair bit beyond the actual city boundaries. As an example "La Défense", the business district of Paris, isn't even inside Paris.
So of course it's very dense. The equivalent for NYC would be Manhattan without the 4 boroughs. Maybe even a subset of Manhattan.
And yet the density of "Paris" gets trotted out in pretty every one of these discussions to show that you don't need skyscrapers.
The density of the Paris Urban area--which is about 1/6 the size of the Paris Metro area but about 20x the area of the "City"--is about 10,000/sq. mi. (10m people in about 1,000 sq. mi.
The point is that a density comparable to Paris or Manhattan does not turn a place into a hellhole. A lot of people love living in central Paris, or Manhattan, and pay a lot of money for the privilege.
The main problem in the above is the "city" of San Jose. it's low rise and tract housing all the way. It's an agglomerated suburb which calls itself a city. It could easily handle 4 million, if they built the transit infrastructure for it and allowed mid-rises --but no.
I guess I understated my case re St Jose. But you can find the low rise component in many areas tucked in the middle of tract housing, detached, etc.
They exist and they tend to be eyesores hidden behind trees. The tell-tale sign is the parking found underneath the living level exemplified with non-structural walls. So yes, it's even worse than I stated. Land is grossly underutilized for a city the size of sanctus iosephus. We can blame proper suburbs like MTV, Sunnyvale, &c, but S. Joe takes the cake. It's a city, at least on paper, and should act like one.
A big problem San Jose has is that they stuck their airport in the middle of the city. Height limits are imposed along the air traffic patterns.
(Not that the other parts of San Jose are any better.)
As an historical note, the key planner/city manager from San Jose came from the Los Angeles area - Hamann ( https://en.wikipedia.org/wiki/A._P._Hamann ). For twenty years (1950-1969), he did his best to make San Jose a sprawling city that sucked up as much of Santa Clara county as he could make happen.
San Jose at 4 million would have no water pressure. In the bay area, only San Francisco has significant water rights. The rest of the area has to stay low-density.
These numbers would be even more exaggerated if San Francisco's city boundaries were drawn similarly to NYC's. A reasonable comparison would be if SF city included every region on the BART (so, all of Oakland and Berkeley and Fremont) and also San Mateo. Or if NYC was just Manhattan (which has a density of 72,033 ppl/sq mi).
If San Francisco’s city boundaries included most of the Bay Area, regional transit and development planning would be much better integrated, and density would surely rise.
Your point on density is a good one. And one could argue (and I have tried to at times) that cities could benefit by sending transit to their "density points" preferentially. That would maximize the value for commuters. When I see plans for a station or train stop with fewer than 10,000 people within a mile of it, I wonder if that will be useful or not.
Great idea but way too late in this case. And, at least in this area, urban planning is a moving target since there is no single authority that can move all of the pieces.
For example, a city can zone land to be high density residential but that doesn't necessarily mean that people who live there will sell their low density houses so that they can be redeveloped.
A high density area can "want" a public transit station, but the transit authority may not have the funds, or the motivation to give them a station. And there is no means for a city government for "force" them to.
Instead, we're left with the "master plan" system which is a living document that the city government maintains which expresses a "vision" of what the city would like (their plans) and the city council signs up to help make it real when there is something within their powers to do to help it.[1]
A previous example of how hard that can be is the "Coyote Valley" area in South San Jose that was going to be "anchored" in part by large campuses of big employers (Cisco was the headline client). But the dot com crash hit and Cisco never moved in.
A current example is the new Apple "spaceship" campus in Cupertino which is to far away from the train station and
in a set of roads that don't easily feed or drain to the 280 interstate.
[1] And part of that is the problem in Sunnyvale, the plan says one thing, and when councilmembers have voted to help implement it, it has irritated citizens who didn't realize that was what the plan said.
As someone who commutes [usually] on 280S from Mountain View to San Jose -- adding about 5 miles to my optimal commute -- because 101S-->87S is even worse, I'm dreading the opening of the new Apple campus.
Note though that most of the new Apple Campus was the rather large HP Wolfe Road campus. I don't know how the population of the campus changes between the two companies (and it's obviously near zero at the moment) but it's not like there was nothing there before.
For sure it's not as dense as NY. But even 3,000 ppl/sq mi is more than enough to provide a bus/rail which drives every 7-15 minutes during rush hour.
It would help a lot if the Caltrain would be improved with more rides and less street crossings. Then there should be more buses bringing people to the Caltrain.
Paris: 55,673 ppl/sq mi
New York City: 27,016.3 ppl/sq mi
San Francisco: 17,179.2 ppl/sq mi
Zurich: 12,000 ppl/sq mi
Vienna: 11,205 ppl/sq mi
San Bruno: 7,505.0 ppl/sq mi
Mountain View: 6,034.8 ppl/sq mi
San Jose: 5,256.2 ppl/sq mi
Cupertino 5,179.1 ppl/sq mi
Menlo Park: 3,271.3 ppl/sq mi