I think that describing the MCAS disaster using predominantly SW Engineering language is misleading.
The substantial change that might have averted the catastrophe is having qualified engineering oversight integrated into the MCAS project management structure.
MCAS is a flight control application; its defining discipline is control theory. Thus the hypothetical engineer who could have averted the catastrophe would have had to be a controls, rather than a SW person.
I have read much about MCAS, but no detailed narrative ever mentioned a SW bug, which implies that MCAS SW has apparently functioned according to given specifications.
Thus, while $9/hr SW engineers is a choice correlated with an inadequate safety culture, I fail to see the casual link between that and the tragic outcomes; I cannot condone asserting such causation.
And the fact that they wanted to spend nothing on the software and then rush to production (this was exposed in this article) is some cluster-b antisocial personality disorder shit...The bean counters had the stock churning at all time highs and they were going to do things their own way (trading at 450 per share shortly before these tragedies).
So it's HIGHLY unlikely that Boeings software for the 737 Max would have been operating at spec given the use-case (overcoming hardware problems with software in a way that's never been done before). MCAS was not designed to take into account that external sesors could be out of wack (which happens all the time). MCAS took liberties and had opinions that ran counter to the norms of aviation. And this is the hallmark of poor software engineers that have no domain expertise and no ability to push back against anti-social personalities masquerading as managers.
"
When the two computers disagree, the solution for the humans in the cockpit is to look across the control panel to see what the other instruments are saying and then sort it out. In the Boeing system, the flight management computer does not “look across” at the other instruments. It believes only the instruments on its side. It doesn’t go old-school. It’s modern. It’s software.
This means that if a particular angle-of-attack sensor goes haywire—which happens all the time in a machine that alternates from one extreme environment to another, vibrating and shaking all the way—the flight management computer just believes it. "
If you want to have the title of SR. Engineer and lives are in the palm of your hand you better be prepared to be the bad guy and take on management when they're being driven by motivations that run counter towards the quality and the usecases for your code. VW had a similar culture and those "senior software engineers" went to prison, not the managers (remember that).
This. On top of the MCAS modification being necessary in the first place.
Higher torque generated because of more off-center thrusters surely raised eyebrows in the stall risk mitigation team.
In the end, that's 3 teams where management made the topic being fixed by another team. That's systemic. And with a single confounder: cost measures being prioritized over safety.
With the current QA problems, it cannot be made more clear that that single culture element is the root cause at Boeing. And ethics dictate that no more experiments be made, and the top 200 of management be forcefully removed.
The other problem from what I can see was a lot of frog-boiling. MCAS was actually originally designed for a slightly different aerodynamic situation, but in flight testing, they discovered a different and more serious instability.
The engineers determined that they could use the MCAS to fix this problem too, but the amount of control input had to be about 4x as much for this situation (2.5 degrees vs. 0.6 degrees of stabilizer movement). They also made it so MCAS could activate multiple times. The original safety analysis was based on the 0.6 degrees for one shot, which wouldn't have put the plane in a situation where the pilots couldn't overpower it, like what happened in the two crashes, so the system was put in a lower safety category that didn't require the same redundancy. Dominic Gates wrote about this in [1]
Given that the plane was already built and in flight test, the production lines were ready to start, and there were massive contract penalties for either late delivery, or additional pilot training, the pressure to hack something in and ship must have been enormous.
The only place to make a change and still hit the deadline was in the software, and unfortunately even the software was limited by Boeing's redundancy strategy of having two completely independent sets of flight computers and sensors. Having one computer look at sensors from both the left and right sides compromises the concept of having them be completely independent, and so it wasn't done, even though ultimately that was what was used as the fix.
Apparently 1/3 of all software vulnerabilities represent design weaknesses which were introduced in the requirements phase. The MCAS flaw seems to belong to this category which you describe.
Separating controls from SW is a strange thing to do. The issue here is a culture of shortcuts and awful money-driven decisions, not any particular discipline, though delayed software fixes did cause more accidents, and it was poorly designed overall.
Ultimately, the MCAS relied on a single sensor, one which was known to fail, and only displayed the redundant sensor data - get this - if they bought the additional option to show when the sensor failed.
> However, whereas MCAS was activated automatically, without pilot action, the cockpit crew would have to notice and act on an AOA DISAGREE alert. Further, the AOA indicator and disagree alert were not standard equipment on the 737 MAX, although the AOA indicator had been on earlier models. Boeing offered them as “add ons” at additional cost
Without that "add on" you'd never know the MCAS was acting on faulty data. And all the while management worked overtime to mislead regulators on the potential impact of MCAS to avoid additional scrutiny and training requirements. So nobody knew the safety of their aircraft depended on that non-standard equipment package.
Yeah - total business and design mismanagement driven by greed.
> When I say I changed the culture of Boeing, that was the intent, so it’s run like a business rather than a great engineering firm. It is a great engineering firm, but people invest in a company because they want to make money. --Harry Stonecipher, 2004
I think that outsourcing was a symptom of this disease. Not MBA cancer, not poor software, not anything other than a deliberate cultural shift which led to all those other things.
>> When I say I changed the culture of Boeing, that was the intent, so it’s run like a business rather than a great engineering firm. It is a great engineering firm, but people invest in a company because they want to make money. --Harry Stonecipher, 2004
> I think that outsourcing was a symptom of this disease. Not MBA cancer, not poor software, not anything other than a deliberate cultural shift which led to all those other things.
That is the MBA cancer. Focusing on shareholder value instead of making a great product.
I've come to the conclusion that just like we require licenses and certain degrees to work on safety critical fields we should extend that to things you can't have in order to work for companies that do manufacturing, starting with MBA's are legally ineligible for hire at companies like Boeing.
Yeah, I suppose like the biological cancer, there's not a clear cause effect relationship between a reshaping culture, and then MBAs or "Ship-it" mentality taking over engineering oversight and scheduling. It's a runaway effect.
I tend to place more blame on a deliberate reshaping by the top individual, rather than some accidental metastasizing of the problem. I'm straining the analogy.
I've always seen the cancer as MBAs only hire MBAs, and that's how it grows. Its very difficult to get to a senior level in most organizations without an MBA.
Maybe the CEO in such cases often is an MBA and you are a bit talking about the same thing, just from different perspectives? (Looking at the one person, vs the people, at the top?)
> I have compiled the following for the 2021 Fortune 500 US companies (the last global one I've seen is from FT in 2015 https://ig.ft.com/sites/mba-to-ceo/):
> 43% of CEOs have an MBA
Anyway,what happens if the CEO is an engineer, and everyone reporting to him/her is an MBA :-)
Not arguing anything in particular, just want to point out it's not completely unusual to be both. Many of the top "Chief" positions I've worked under were former PhD engineers and researchers who went and got an MBA to move up. The legendary director of JPL, Charles Elachi, for example. https://en.wikipedia.org/wiki/Charles_Elachi
And many of my former bosses.
Maybe that's a symptom of the same disease ("without mba you cannot rule"), but I think it doesn't necessarily mean an MBA holder is a bad candidate for leadership.
The substantial change that might have averted the catastrophe is having qualified engineering oversight integrated into the MCAS project management structure.
MCAS is a flight control application; its defining discipline is control theory. Thus the hypothetical engineer who could have averted the catastrophe would have had to be a controls, rather than a SW person.
I have read much about MCAS, but no detailed narrative ever mentioned a SW bug, which implies that MCAS SW has apparently functioned according to given specifications.
Thus, while $9/hr SW engineers is a choice correlated with an inadequate safety culture, I fail to see the casual link between that and the tragic outcomes; I cannot condone asserting such causation.