Who the hell cares? We're about to invent a certain technology, known as AI. Imagine being able to generate a video of a dog on a skateboard, or an elephant flying to the moon. A "hothouse earth" is a small price to pay - we CAN'T slow down.
There is a certain logic to it though. If the scaling approaches DO get us to AGI, that's basically going to change everything, forever. And if you assume this is the case, then "our side" has to get there before our geopolitical adversaries do. Because in the long run the expected "hit" from a hostile nation developing AGI and using it to bully "our side" probably really dwarfs the "hit" we take from not developing the infrastructure you mentioned.
Any serious LLM user will tell you that there's no way to get from LLM to AGI.
These models are vast and, in many ways, clearly superhuman. But they can't venture outside their training data, not even if you hold their hand and guide them.
Try getting Suno to write a song in a new genre. Even if you tell it EXACTLY what you want, and provide it with clear examples, it won't be able to do it.
This is also why there have been zero-to-very-few new scientific discoveries made by LLM.
Most humans aren't making new scientific discoveries either, are they? Does that mean they don't have AGI?
Intelligence is mostly about pattern recognition. All those model weights represent patterns, compressed and encoded. If you can find a similar pattern in a new place, perhaps you can make a new discovery.
One problem is the patterns are static. Sooner or later, someone is going to figure out a way to give LLMs "real" memory. I'm not talking about keeping a long term context, extending it with markdown files, RAG, etc. like we do today for an individual user, but updating the underlying model weights incrementally, basically resulting in a learning, collective memory.
Virtually all humans of average intelligence are capable of making scientific discoveries -- admittedly minor ones -- if they devote themselves to a field, work at its frontiers, and apply themselves. They are also capable of originality in other domains, in other ways.
I am not at all sure that the same thing is even theoretically possible for LLMs.
Not to be facetious, but you need to spend more time playing with Suno. It really drives home how limited these models are. With text, there's a vast conceptual space that's hard to probe; it's much easier when the same structure is ported to music. The number of things it can't do absolutely outweighs the number of things it can do. Within days, even mere hours, you'll become aware of its peculiar rigidity.
Are you seriously comparing chips running AI models and human brains now???
Last time I checked the chips are not rewiring themselves like the brain does, nor does even the software rewrite itself, or the model recalibrate itself - anything that could be called "learning", normal daily work for a human brain.
Also, the models are not models of the world, but of our text communication only.
Human brains start by building a model of the physical world, from age zero. Much later, on top of that foundation, more abstract ideas emerge, including language. Text, even later. And all of it on a deep layer of a physical world model.
The LLM has none of that! It has zero depth behind the words it learned. It's like a human learning some strange symbols and the rules governing their appearance. The human will be able to reproduce valid chains of symbols following the learned rules, but they will never have any understanding of those symbols. In the human case, somebody would have to connect those symbols to their world model by telling them the "meaning" in a way they can already use. For the LLM that is not possible, since it doesn't habe such a model to begin with.
How anyone can even entertain the idea of "AGI" based on uncomprehending symbol manipulation, where every symbol has zero depth of a physical world model, only connections to other symbols, is beyond me TBH.
Speaking as someone who thinks the Chinese Room argument is an obvious case of begging the question, GP isn't about that. They're not saying that LLMs don't have world models - they're saying that those world models are not based in physical world and thus cannot properly understand what they talk about.
I don't think that's true anymore, though. All the SOTA models are multimodal now, meaning that they are trained on images and videos as well, not just text; and they do that is precisely because it improves the text output as well, for this exact reason. Already, I don't have to waste time explaining to Claude or Codex what I want on a webpage - I can just sketch a mock-up, or when there's a bug, I take a screenshot and circle the bits that are wrong. But this extends into the ability to reason about real world, as well.
I would argue that is still just symbols. A physical model requires a lot more. For example, the way babies and toddlers learn is heavy on interaction with objects and the world. We know those who have less of that kind of experience in early childhood will do less well later. We know that many of today's children, kept quiet and sedated with interactive screens, are at a disadvantage. What if you made this even more extreme, a brain without ability to interact with anything, trained entirely passively? Even our much more complex brains have trouble creating a good model in these cases.
You also need more than one simple brain structure simulation repeated a lot. Our brains have many different parts and structures, not just a single type.
However, just like our airplanes do not resemble bird flight as the early dreamers of human flight dreamed of, with flapping wings, I also do not see a need for our technology to fully reproduce the original.
We are better off following our own tech path and seeing where it will lead. It will be something else, and that's fine, because anyone can create a new human brain without education and tools, with just some sex, and let it self-assemble.
Biology is great and all but also pretty limited, extremely path-dependent. Just look at all the materials we already managed to create that nature would never make. Going off the already trodden bio-path should be good, we can create a lot of very different things. Those won't be brains like ours that "Feel" like ours, if that word will ever even apply. and that's fine and good. Our creations should explore entirely new paths. All these comparisons to the human experience make me sad, let's evaluate our products on their own merit.
One important point:
If you truly want a copy, partial or full, in tech, of the human experience, you need to look at the physics. Not at some meta stuff like "text"!!
The physical structure and the electrical signals in the brain. THAT is us.
And electrical signals and what they represent in chips are so completely and utterly different from what can be found in the brain, THAT is the much more important argument against silly human "AGI" comparisons. We don't have a CPU and RAM. We have massively parallel waves of electrical signals in a very complex structure.
Humans are hung up on words. We even have fantasy stories hat are all about it. You say some word, magic happens. You know somebody's "true name", you control them.
But the brain works on a much lower deeply physical level. We don't even need language. A human without language and "inner voice" still is a human with the same complex brain, just much worse at communication.
The LLMs are all about the surface layer of that particular human ability though. And again, that is fine, but it has nothing to do with how our brains work. We looked at nature and were inspired, and went and created something else. As always.
In some ways no, because to learn something you have to LEARN that then thats in the training data. But humans can do it continuously and sometimes randomly, and also being without prompted.
If you're a scientist -- and in many cases if you're an engineer, or a philosopher, or even perhaps a theologian -- your job is quite literally to add to humanity's training data.
I'd add that fiction is much more complicated. LLMs can clearly write original fiction, even if they are, as yet, not very good at it. There's an idea (often attributed to John Gardner or Leo Tolstoy) that all stories boil down to one of two scenarios:
So I'd tentatively expect tomorrow's LLMs to write good fiction along those well-trodden paths. I'm less sanguine about their applications in scientific invention and in producing original music.
I mean yeah, but that's why there are far more research avenues these days than just pure LLMs, for instance world models. The thinking is that if LLMs can achieve near-human performance in the language domain then we must be very close to achieving human performance in the "general" domain - that's the main thesis of the current AI financial bubble (see articles like AI 2027). And if that is the case, you still want as much compute as possible, both to accelerate research and to achieve greater performance on other architectures that benefit from scaling.
How does scaling compute does not go hand-in-hand with energy generation? To me, scaling one and not the other puts a different set of constraints on overall growth. And the energy industry works at a different pace than these hyperscalars scaling compute.
The other thing here is we know the human brain learns on far less samples than LLMs in their current form. If there is any kind of learning breakthrough then the amount of compute used for learning could explode overnight
Scaling alone wont get us to AGI. We are in the latter half of this AI summer where the real research has slowed down and even stopped and the MBAs and moguls are doing stupid things
For us to take the next step towards AGI, we need an AI winter to hit and the next AI summer to start, the first half of which will produce the advancement we actually need
Well, I tried to specifically frame it in a neutral way, to outline the thinking that pretty much all the major nations / companies currently have on this topic.
I agree with you with the caveat that all the "ease of building" benefits, for me, could potentially be dwarfed by job losses and pay decreases. If SWE really becomes obsolete, or even if the number of roles decrease a lot and/or the pay decreases a lot (or even fails to increase with inflation), I am suddenly in the unenviable position of not being financially secure and being stuck in my 30s with an increasingly useless degree. A life disaster, in other words. In that scenario the unhappiness of worrying about money and retraining far outweighs the happiness I get from being able to build stuff really fast.
Fundamentally this is the only point I really have on the 'anti-AI' side, but it's a really important one.
I've always wondered... if Lidar + Cameras is always making the right decision, you should theoretically be able to take the output of the Lidar + Cameras model and use it as training data for a Camera only model.
That's exactly what Tesla is doing with their validation vehicles, the ones with Lidar towers on top. They establish the "ground truth" from Lidar and use that to train and/or test the vision model. Presumably more "test", since they've most often been seen in Robotaxi service expansion areas shortly before fleet deployment.
I don't have a specific source, no. I think it was mentioned in one of their presentation a few years back, that they use various techniques for "ground truth" for vision training, among those was time series (depth change over time should be continuous etc) and iirc also "external" sources for depth data, like LiDAR. And their validation cars equipped with LiDAR towers are definitely being seen everywhere they are rolling out their Robotaxi services.
"Exactly" is impossible: there are multiple Lidar samples that would map to the same camera sample. But what training would do is build a model that could infer the most likely Lidar representation from a camera representation. There would still be cases where the most likely Lidar for a camera input isn't a useful/good representation of reality, e.g. a scene with very high dynamic range.
No, I don't think that will be successful. Consider a day where the temperature and humidity is just right to make tail pipe exhaust form dense fog clouds. That will be opaque or nearly so to a camera, transparent to a radar, and I would assume something in between to a lidar. Multi-modal sensor fusion is always going to be more reliable at classifying some kinds of challenging scene segments. It doesn't take long to imagine many other scenarios where fusing the returns of multiple sensors is going to greatly increase classification accuracy.
The goal is not to drive in all conditions; it is to drive in all drivable conditions. Human eyeballs also cannot see through dense fog clouds. Operating in these environments is extra credit with marginal utility in real life.
But humans react to this extremely differently than a self driving car.
Humans take responsability, and the self-driving disengages and say : WELP.
Oh sorry were you "enjoying your travel time to do something useful" as we very explicitely marketed ? Well now your wife is dead and it's your fault (legally). Kisses, Elon.
No, but if you run a shadow or offline camera-only model in parallel with a camera + LIDAR model, you can (1) measure how much worse the camera-only model is so you can decide when (if ever) it's safe enough to stop installing LIDAR, and (2) look at the specific inputs for which the models diverge and focus on improving the camera-only model in those situations.
The guy's a master of spin, no doubt. I don't know how you start out with the concept of ads in AI chatbots and end with "This time belongs to the builders, not the people who want to control them." What a bunch of generic nonsense... and yet people lap it up like puppy dogs.
I'm for AI the technology. Who doesn't want cool sci-fi future tech? But I'm against AI being co-opted by our dear leaders to end white collar work and lock us all in a permanent underclass.
I have no doubt that someday in the future these world models will revolutionize gaming. However, they are clearly very far off from that point in capability, not to mention the cost. And a lot of these articles I'm seeing are confidently stating very incorrect facts like "this new model will completely change the workflow of game developers and accelerate game development." No it won't.
I don't trade individual stocks but it does seem like an easy case of "buy the dip" here.
The intersection of investors who understand what it takes to build a game engine and also the capabilities and plausible future capabilities of Google’s model is practically zero.
I don't really get this take... not when Tesla is by a wide mile the world's most valuable automaker. How does Tesla ending production of the S and X equate to the old auto establishment "stomping over them"?
Worth related statistics doesn't mean anything in the realm of hard engineering. I completely look from the point of "what the companies are doing tech-wise".
When Tesla came about, they were distinctively different. A different chassis, a different weight distribution, completely different dynamics. Since they started with a blank slate, their cars were greenfield projects, and they correctly took note of the pitfalls, and avoided them.
On the other hand, avoiding past pitfalls or remedying them doesn't make you immune from the future ones, and doesn't mean the other companies can't learn, too. This is where they made the mistake.
They overpromised (esp. with the Autopilot thingy) and underdelivered massively on that front, and while they "made" the software-defined-vehicle, they underestimated the problems and behaved like the problems they face are as simple as configuring a web service right. This is what slowly broke them. They also underestimated hardware problems of the car (like using consumer grade parts in the critical parts of the hardware. Remember wearing down flash chips and bricking cars?)
Because while car is software defined now, it's also an "industrial system". It has to be robust. It has to be reliable, idiot-proof even. Playing fast and loose with these things allowed automakers to catch them, maybe slowly but surely.
Because, "the old automakers" has gone through a lot of blood, sweat and tears (both figuratively and literally), and know what to do and what not to do. They can anticipate pitfalls better then a "newbie" carmaker. They shuddered, sputtered, hesitated, but they are in the move now. They will evolve this more slowly, but in a more reliable and safer way. They won't play that fast, but the products will be more refined. They won't skimp on radars because someone doesn't believe in them, for example.
Not everything is numbers, valuations and great expansions which look good on quarterlies, news, politics, and populists. Sometimes the slow and steads wins, and it goes for longer.
Physics and engineering doesn't care for valuations. They only care about natural laws.
Thank you for the explanation. I guess the thing I don't understand is what exactly the problems are that you are seeing. We've all heard the stories of wooden parts in initial production runs of Tesla models, sure. But it does seem like they iron out these kinks over time. Maybe I'm biased because I'm in the bay area, but it seems like every 3rd car you see on the highway is a Tesla, and lots of my coworkers speak very highly of theirs that they own. It just doesn't seem to me like there is actually a quality issue here?
If anything, ending production of SX and giving more focus to 3Y would probably increase the quality of those models, I'd imagine.
If you're pointing to Autopilot / camera-only as the main transgression here, yeah I'll agree that they have definitely overpromised, but it doesn't really seem to me like the lack of a L5 system is actually a deal-breaker for anyone, because from what I hear they are just damn good cars anyway.
Honestly, how long do you guys think we have left as SWEs with high pay? Like the SWE job will still exist, but with a much lower technical barrier of entry, it strikes me that the pay is going to decrease a lot. Obviously BigCo codebases are extremely complex, more than Claude Code can handle right now, but I'd say there's definitely a timer running here. The big question for my life personally is whether I can reach certain financial milestones before my earnings potential permanently decreases.
It's counterintuitive but something becoming easier doesn't necessarily mean it becomes cheap. Programming has arguably been the easiest engineering discipline to break into by sheer force of will for the past 20+ years, and the pay scales you see are adapted to that reality already.
Empowering people to do 10 times as much as they could before means they hit 100 times the roadblocks. Again, in a lot of ways we've already lived in that reality for the past many years. On a task-by-task basis programming today is already a lot easier than it was 20 years ago, and we just grew our desires and the amount of controls and process we apply. Problems arise faster than solutions. Growing our velocity means we're going to hit a lot more problems.
I'm not saying you're wrong, so much as saying, it's not the whole story and the only possibility. A lot of people today are kept out of programming just because they don't want to do that much on a computer all day, for instance. That isn't going to change. There's still going to be skills involved in being better than other people at getting the computers to do what you want.
Also on a long term basis we may find that while we can produce entry-level coders that are basically just proxies to the AI by the bucketful that it may become very difficult to advance in skills beyond that, and those who are already over the hurdle of having been forced to learn the hard way may end up with a very difficult to overcome moat around their skills, especially if the AIs plateau for any period of time. I am concerned that we are pulling up the ladder in a way the ladder has never been pulled up before.
Supply and demand. There will continue to be a need for engineers to manage these systems and get them to do the thing you actually want, to understand implications of design tradeoffs and help stakeholders weigh the pros and cons. Some people will be better at it than others. Companies will continue to pay high premiums for such people if their business depends on quality software.
I think to give yourself more context you should ask about the patterns that led to SWEs having such high pay in the last 10-15 years and why it is you expected it to stay that way.
I personally think the barrier is going to get higher, not lower. And we will be back expected to do more.
I think the pay is going to skyrocket for senior devs within a few years, as training juniors that can graduate past pure LLM usage becomes more and more difficult.
Day after day the global quality of software and learning resources will degrade as LLM grey goo consumes every single nook and cranny of the Internet. We will soon see the first signs of pure cargo cult design patterns, conventions and schemes that LLMs made up and then regurgitated. Only people who learned before LLMs became popular will know that they are not to be followed.
People who aren't learning to program without LLMs today are getting left behind.
Yeah, all of this. Plus companies have avoided hiring and training juniors for 3 or 4 years now (which is more related to interest rates than AI). Plus existing seniors who deskill themselves by outsourcing their brain to AI. Seniors who know actually what they're doing are going to be in greater demand.
That is assuming that LLMs plateau in capability, if they haven't already, which I think is highly likely.
Unfortunately it seems the incentive structures are ultimately set by global competition and the security dilemma. So short of total world disarmament and UBI, fundamentally the incentive will always be to work harder than your opponent to preserve your own security.
reply