True (I made such a proposal myself a few hours ago, albeit in vaguer terms). The thing is deployment infrastructure is good enough now that we can just treat it as modular signal flows and experiment a lot without having to engineer a whole pile of custom infrastructure for each impulsive experiment.

Isn't that what the Singularity was described as a few decades ago? Progress so fast it's unpredictable even in the short term.

Same as it ever was, scientific revolutions arrive all at once, punctuating otherwise uneventful periods. As I understand, the present one is the product of the paper "Attention is all you need": https://arxiv.org/pdf/1706.03762.pdf.

... that one has 52K citations, and the 2D to 3D paper "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis" with 1488 citations.

https://arxiv.org/abs/2003.08934

>as with a majority of ML research

Plus "we did the same thing, but with 10x the compute resources".

But yeah.

> This seems like basically plugging a couple of techniques together that already existed

Do this enough times and eventually the thing you have looks indistinguishable from something completely novel.