some people discuss these dynamics as sheep versus goats. Social stability was more precious due to scarcity, while goat behavior included 40 armed men killing their rivals with swords (and better if the rivals do not have their own swords). Many, many parallels exist in mammals that live in groups. You might be surprised at the details of how some mammals actually behave in real life!
I find these throws of passionate despondency similar to the 1980s personal computing revolution. Oh dear. Giving mere mortals the power of computing?! How many people would abandon their computers or phones.
It’s not like it changes our industry’s overall flavour.
How many SaaS apps are excel spreadsheets made production grade?
It’s like every engineer forgets that humans have been building a Tower of Babel for 300000 years. And somehow there is always work to do.
People like vibe coding and will do more of it. Then make money fixing the problems the world will still have when you wake up in the morning.
I am not against vibe coding at all, I just don't think people understand how shaky the foundation is. Software wants to be modified. With enough modifications the disconnect between the code as it is imagined and the code in reality becomes too arduous of a distance to bridge.
The current solution is to simply reroll the whole project and let the LLM rebuild everything with new knowledge. This is fine until you have real data, users and processes built on top of your project.
Maybe you can get away with doing that for a while, but tech debt needs to be paid down one way or another. Either someone makes sense of the code, or you build so much natural language scaffolding to keep the ship afloat that you end up putting in more human effort than just having someone codify it.
We are definitely headed toward a future where we have lots of these Frankenstein projects in the wild, pulling down millions in ARR but teetering in the breeze. You can definitely do this, but "a codebase always pays its debts."
Yea, the more things change the more they stay the same. This latest AI hype cycle seems to be no different. Which I think will become more widely accepted over the next couple of years as creating deployable, production-ready, maintainable, sellable, profitable software remains difficult for all the reasons besides the hands-to-keyboard writing of code.
Or drop the price to $20 a year instead of $20 a month and and focus on small software updated infrequently. Software as a service has a dirty secret that it was more service than software. The companies became larded with payroll and most never had great gross margins.
Pretty much. A lot of software is just good enough already, just keep security updates going and fix occasional bug people complain for too long.
But that might require just firing some people because that amount of man-hours is not needed any more or moving them to make something new and no investor likes it
It’s a simple math problem. And it is also Conway’s law that says all software design follows the organization that built it—that is all software design is political.
A framework calls you. You call a library.
A framework constrains the program. A library expands the program.
It’s easier to write a library that is future proofed because it just needs to satisfy its contract.
It’s harder to write a framework because it imposes a contract on everything that depends on it.
Just like it is hard to write tort law without a lot of jurisprudence to build out experience and test cases, it is hard to write a framework from only one use case.
No one likes lawyers because they block you from doing what you want. This is the problem with frameworks.
However the government likes laws because they block you from doing what you want. Same with whomever is directing engineering that wants all other programmers to work in a consistent way.
Technically everything you have written is true. But the proliferation of frameworks is almost a self-reinforcing antipattern.
> No one likes lawyers because they block you from doing what you want.
Or even doing what you need to do.
Certainly, to the extent that a mini-framework is composed of more constraints piled on top of an extant bigger framework, mini-frameworks are, like swimming pools, attractive nuisances. "Hey, look, guys! This is so much simpler!"
> It’s harder to write a framework because it imposes a contract on everything that depends on it.
Judging by what people write and use, I'm not sure this is _exactly_ true. Sure, writing a _good_ framework or library is hard, but people accept piss-poor frameworks, and accept libraries that were designed to work in conjunction with a single framework.
> It’s easier to write a library that is future proofed because it just needs to satisfy its contract.
But the thing is that the library itself defines the contract, and it might be a piss-poor one for many applications.
There is some excellent code out there, and there is a lot of shitty code out there. I think the problem is social; too many people want to write code that is in charge. Now, maybe it's somewhat technical, in that they have used things that are in charge, and they were too big (leading to the mini-framework of the article) or they were otherwise not great, so this leads to yet another framework (cue standards xkcd cartoon) because they realize they need something in charge, but aren't happy with their current options.
And, of course, since the frameworks they know take a kitchen sink mentality, their new framework does as well. (Maybe it's a smaller sink, but everything needed is still shoved in there.) So there are yet more libraries that are tied to yet another framework.
Because writing good libraries that are completely framework independent _can_ be as challenging as writing a good framework. And when someone decides they need a new framework, they are focused on that, and making it work well, and since that drives their thought process, everything else the write gets shoved into the framework.
Thank you. I thought I was going crazy reading the article which doesn’t connect open and close parenthesis :: higher and lower precedence :: indent and outdent :: +1 and -1 and just flip it around to get the opposing polarity.
You’re right and wrong at the same time. A quantum superposition of validity.
The word thinking is going too much work in your argument, but arguably “assume it’s thinking” is not doing enough work.
The models do compute and can reduce entropy; however, they don’t match the way we presume things do this because we assume every intelligence is human or more accurately the same as our own mind.
To see the algorithm for what it is, you can make it work through a logical set of steps from input to output but it requires multiple passes. The models use a heuristic pattern matching approach to reasoning instead of a computational one like symbolic logic.
While the algorithms are computed, the virtual space the input is transformed to the output is not computational.
The models remain incredible and remarkable but they are incomplete.
Further there is a huge garbage in garbage out problem as often the input to the model lacks enough information to decide on the next transformation to the code base. That’s part of the illusion of conversationality that tricks us into thinking the algorithm is like a human.
AI has always had human reactions like this. Eliza was surprisingly effective, right?
It may be that average humans are not capable of interacting with an AI reliably because the illusion is overwhelming for instinctive reasons.
As engineers we should try to accurately assess and measure what is actually happening so we can predict and reason about how the models fit into systems.
Growth curves mean nothing if you're selling $0.90 dollars. You have to show a growth curve when price > cost. It's not even clear that value > cost.
I absolutely love Anthropic; but I am worried about the fiscal wall they will hit that will ratchet up my opex as they will need to steeply raise prices.
You have to include the carrying cost per customer as well which is mostly labour. Most of SaaS undercounts the payroll attached to a subscription which is why it is so hard to get to positive net margins and maintain lifetime value.
I am sceptical an LLM foundation model company can get away with low human services either directly on its own payroll or by giving up margin to a channel of implementation partners. Thats because the go to market requires organizational change on the customer sites. That is a lot of human surface area.
Even if their introspection within the inference step is limited, by looping over a core set of documents that the agent considers itself, it can observe changes in the output and analyze those changes to deduce facts about its internal state.
You may have experienced this when the llms get hopelessly confused and then you ask it what happened. The llm reads the chat transcript and gives an answer as consistent with the text as it can.
The model isn’t the active part of the mind. The artifacts are.
This is the same as Searles Chinese room. The intelligence isn’t in the clerk but the book. However the thinking is in the paper.
The Turing machine equivalent is the state table (book, model), the read/write/move head (clerk, inference) and the tape (paper, artifact).
Thus it isn’t mystical that the AIs can introspect. It’s routine and frequently observed in my estimation.
This seems to be missing the point? What you're describing is the obvious form of introspection that makes sense for a word predictor to be capable of. It's the type of introspection that we consider easy to fake, the same way split-brained patients confabulate reasons why the other side of their body did something. Once anomalous output has been fed back into itself, we can't prove that it didn't just confabulate an explanation. But what seemingly happened here is the model making a determination (yes or no) on whether a concept was injected in just a single token. It didn't do this by detecting an anomaly in its output, because up until that point it hadn't output anything - instead, the determination was derived from its internal state.
I think it’s perfectly clear: the model must know it’s been tampered with because it reports tampering before it reports which concept has been injected into its internal state. It can only do this if it has introspection capabilities.
Sure I agree what I am talking about is different in some important ways; I am “yes and”ing here. It’s an interesting space for sure.
Internal vs external in this case is a subjective decision. Where there is a boundary, within it is the model. If you draw the boundary outside the texts then the complete system of model, inference, text documents form the agent.
I liken this to a “text wave” by metaphor. If you keep feeding in the same text into the model and have the model emit updates to the same text, then there is continuity. The text wave propagates forward and can react and learn and adapt.
The introspection within the neural net is similar except over an internal representation. Our human system is similar I believe as a layer observing another layer.
I think that is really interesting as well.
The “yes and” part is you can have more fun playing with the models ability to analyze their own thinking by using the “text wave” idea.
> This is the same as Searles Chinese room. The intelligence isn’t in the clerk but the book. However the thinking is in the paper.
This feels like a misrepresentation of the "Chinese Room" thought experiment. That the "thinking" isn't the clerk nor the book; it's the entire room itself.
And in this case actually correct! Decimate is often used to mean “almost wipe out”, but the word actually comes from “killing every 10th person”.. I.e 10% of a group.
The basic social skill is to avoid conflict and seek acceptance. Go along to get along.
One wouldn’t rewrite the app on one’s on recognizance without peer approval first if this is your vibe.
reply