You do not have to respond to everything. Some writing is an opertunity for reflection. If you respond without some reflection, you become more of an NPC.

On a forum I could give things a couple of days and then write a response. Or I could start my own thread on the topic. But that's less of an option in places like HN. If I write a comment here two days from now, there's a good chance that zero people will see it. If I want to discuss the topic but not the essay, what are my options? I could start my own blog, write my thoughts on the matter, submit it to HN, hope that I'm one of the 1% of the submissions that make it past the screeners who hang out on "New," then hope I actually generate some discussion and don't immediately fall off the page.

I actually agree with you that more reflection in general is a good idea (though I don't necessarily agree that these kinds of essays engender that kind of reflection, but that's a separate topic). However, the online communities that exist now are designed to dissuade people from doing anything (reflection, research, editing, etc.) that take more time.

There is no reason to expect you will find the sort of dialogue you want unless you take some steps to initiate it yourself. The people who regularly appear on the front page of HN did not start off doing so.

This kind of writing has its own rewards. It is just as valuable to the writer as the reader. Paul Graham makes the point that developing writing skills also develops your ideas. If you cannot articulate your ideas well enough for others to understand it, it is likely the case your idea is still fully undeveloped.

I mean, certainly there are plenty of things people want to comment on without writing a blog post on it. This discussion, for example. We're discussing this with relatively quickly written comments, not as blog posts that we spend a great deal of time on, put away for a day, come back to edit, etc.

It's also the case that time is limited, and there are some topics we don't want to spend much time on. It's common to see people argue that if you don't spend as much time on the topic as them, then your opinions on it aren't as worthy, but I can't really agree with that. It's very often used as a way to defend poor beliefs against obvious criticisms. You see it a lot with conspiracy theories. "You can't dismiss this unless you've read all of the writings on it!" But only true believers are going to subject themselves to dozens of books on a crank theory.

I think the rise of meander is more likely a sign of inattention and the inability to focus. An essay is much more powerful and memorable if it can state a clear thesis and defend it memorably and undeniably.

In today's writing, perhaps because we demand so great a volume of it, purposeful prose financially rewards the author's extra effort less than ever before, and is less appreciated by readers because they're less willing to pause their pace of consumption to reflect on subtleties and unobvious insights. The online written word has evolved into a 24x7-driven ehpemeral commodity, where cleverness alone is the desiderata that makes or breaks the work and its auteur.