▲ 249 points
back
129 comments
Is someone dividing by zero somewhere?
"All six display units (DUs) blanked with a selected instrument approach to a runway with a 270-degree true heading, and all six DUs stayed blank until a different runway was selected."
cos(270°) = 0
Points out how you should use vector coordinates in navigation, instead of trig. All those pesky divides happen in one place, are obvious, and handling them is trivial.
But cos(90°) = 0 too, unless they thought to check for one case but they did check for the other, or something more complex is happening.
Next week un elreg : HN fixes 20 years old bug in Boeing 737
That would also cause a divide by zero at a 90 degree heading
Likewise cos 90° (all runways designated 27 are also designated 9) but those haven't reported these problems, have they?
angle = arctan(dy/dx) where dx=0.
I guarantee this is the mistake they made.
In some languages you can use Math.atan2(dy,dx) to easily avoid this.
That reminds me of the C-130 avionics failure when flying at negative altitude.
https://www.popularmechanics.com/military/aviation/a26598/c-...
Or the flight computer inverting the plane on flying over the equator (changing the sign of the latitude), luckily only in the simulator [1].
Reminds me of a story about the Black Buck raids by Vulcan bombers during the Falklands Conflict.
Because they didn't have any maps of the southern hemisphere, they just turned a map of the Atlantic upside down.
Source: https://youtu.be/rpRtEABdxNc?t=359 (worth listening to the whole thing!)
> The lowest airfield in North America, for the record, is Furnace Creek Airport, Death Valley at minus 210 feet below sea level.
So, then I guess it's 210 ft _above_ sea level? (SCNR)
While it is easy to blame Boeing these days, I suspect we have survivorship bias here. Dig into Airbus software and all sorts of similar scary bugs will pop up. For instance, one may remember inconsistencies between airspeed measurements that resulted in the loss of AF 447 on June 1, 2009. Overall, it's great that at least some airliner's closed software finally gets more eyes looking at it.
Frozen pitot tubes causing an airspeed disagree is something we train for. The issue there isn't software, it's mechanical, and what the Airbus did is exactly what it's supposed to do according to the manual.
The problem there was the junior co-pilot responding all wrong and the other pilot not noticing because Airbus sticks don't move together. The captain woke up, came to check and realized what was wrong but did so too late to fix it.
I seem to recall Airbus having different engineering teams across EU using different versions of the CAD tool and then when they went to build the plane the wires were too short because one version of the CAD tool accounted for the radius of wires turning a corner while the other didn't.
https://www.nytimes.com/2006/12/11/business/worldbusiness/11...
It is certain that under close scrutiny one can find bugs as well in Airbus airplanes. Bugs are inherent to software production, all processes are just there to minimizes the risks posed by critical bugs.
However, as airbus released the first fly by wire airliner (a320), they have more experience in software over boeing. Boeing was always afraid to give too much power to the software, a boeing plane has more mechanical backup than an airbus.
I sincerely hope that they are not fucking around by following the agile trend from the sw industry, and instead stay focused on old and proven engineering development practices.
I must say that this whole boeing show makes me scared to hop in a boeing, and is another reason why i will now seek flight in a a320 instead of a 737. The first reason is simply comfort, the 737 cabin size is really too small.
Edit : the a350 had a bug where the pilots needed to switch off the plane every 149h, and recently they discovered a bug in the a220 P&W engine (not airbus developped) leading to engine shuting down in flight. They also found a kind of mcas like bug the a321neo, but I think it happens in extreme manoeuver which can be only replicated in simulators
This anecdote was valid until the MAX:
A Boeing with an engine fire will pop up a dialog saying “flight computer detected a possible engine fire, would you like to shutdown?”. An Airbus with an engine fire will pop a dialog saying “flight computer detected an engine fire, shutting down engines”.
The joke being Boeing pilots have a chance of saving the plane in the event it’s not an engine fire, Airbus will just do what it thinks is best and let the pilot sort out the aftermath. MCAS changed all that
I've wondered if someone in Airbus is currently investigating how they would fare under similar scrutiny, but I suspect the answer is no.
The first F-22 they delivered to japan had serious trouble when it crossed the international date line
Sounds like nobody was hurt & they didn't lose any planes, but there was some period of time where the onboard computers were not good
https://www.defenseindustrydaily.com/f22-squadron-shot-down-...
Yet another reason to exercise low-level math functions with a wide range of inputs -- maybe not on every run, but at least in pre-release integration tests
There are only 4 billion floats
https://randomascii.wordpress.com/2014/01/27/theres-only-fou...
F-22 weren't exported, it was probably just at Okinawa
I'd like to learn how Boeing's software delivery practice has changed. Sounds like there may be some lessons learned we can all learn from.
I would hazard a guess: their process became "agile."
> I'd like to learn how Boeing's software delivery practice has changed
To something like this perhaps?
- Make it cheap. Now!
> I'd like to learn how Boeing's software delivery practice has changed. Sounds like there may be some lessons learned we can all learn from.
Does Boeing build its own cockpit displays and avionics? I was under the impression that that stuff was developed by companies like Rockwell Collins and Honeywell.
Outsourced to India!
Edit: the airports noted aren’t along the same magnetic declination, so maybe not that.
Given that this applies to a few US airports and some S. American ones, maybe it has to do with magnetic declination calculations gone awry.
Maybe only airports along one of these lines gets impacted:
https://en.m.wikipedia.org/wiki/Magnetic_declination#/media/...
And magnetic declination changes all the time. Maybe different systems are using different sources of truth?
Maybe when they did initial testing, they included some airports at the extremes, but when the declinations got updated, they were no longer good test cases?
I guess the automated test suite just got larger. Testing all approaches on all runways on all airports worldwide...
I think you meant to say, "I guess the automated test suite should get larger". :) Otherwise, I am afraid your optimism might be misplaced.
In the automotive and aerospace world, of course tests in the real world cannot cover all cases (cost, time make it impossible).
Even most (not all, but almost none for meaningful applications) computer program cannot be proved to come to an halt [0], so complete testing is impossible by essence. We can only use more restrictive rules for programming but cannot formally guarantee anything.
As those systems are tied to the physical world, a whole lot of complexity is added by uncontrolled parameters.
Yet we love testing things. So a lot of techniques exist, such as SIL [1] and HIL [2].
So you could imagine using a real dashboard hooked up to a plane simulator. Which would enable testing the device in a wide array of conditions.
[0] https://en.wikipedia.org/wiki/Halting_problem [1] https://www.quora.com/What-is-Software-in-the-Loop-SIL [2] https://en.wikipedia.org/wiki/Hardware-in-the-loop_simulatio...
Given these planes can take off and land on autopilot (I'm not sure that's true, just what I hear) then surely it's easy to run simulations of all those events; so if it was in scope for simulation it could be caught?
There's about 20k commercial airfields, if you ran 20mins of simulation for each one then you're done within the year. Presumably Boeing can run in parallel and above realtime.
Are their watchdog systems for everything, or do they primarily rely on pilots noticing problems?
Wild speculation: Perhaps it's a power issue, something about the conditions causes a power demand that exceeds specs (or makes an LED? voltage drop too low). What else links all the screens?
Apparently another plane went down in Ukraine according to this article. That seems like a much bigger story. Maybe I happened to miss that one yesterday?
Edit: A Ukrainian airline in Iran.
A 737 malfunctioning and inadvertently setting off WW3 is very on-brand for Boeing right now.
A Ukrainian airline but it occurred in Tehran, Iran a few hours after the missile attack. It was taking off, not landing.
It could be a big story, but unless it's a discussion on a technical failure, it doesn't really belong on HN. Ever since the last big election cycle people have been more strict about not allowing political and world-news posts that aren't related to tech or hacking culture.
It’s a different model it’s 737-800. Different plane model while still within the 737 family.
Thanks for mentioning it; I hadn't seen it elsewhere.
Looks like it's being blamed on a mechanical failure at the moment. Could be a big story.
Here was already a link on HN and a discussion, which just vanished for some reason (most likely removed by invisible admins). Hate this on HN.
Isn't this something that would be discovered through flight simulation? Does Boeing run their software and hardware through automated testing?
That is an awfully specific bug. I can see how it went undetected. But what could trigger this?
From reading the article, it seems like it occurs when the runway is precisely running east to west and the airplane is coming in westward. And also the latitude/longitude is somehow involved. You can see one of the airports here. Notice how it is exactly horizontal, i.e parallel of latitude. https://www.google.com/maps/place/Wiley+Post-Will+Rogers+Mem...
The affected runways have fairly similar directions (between RW25 and RW29. Runway names follow compass directions. See https://en.wikipedia.org/wiki/Runway#Naming). I don’t know what the distribution of runways in the USA is, but that feels too similar to me to be a coincidence.
Even if it is, I think it wouldn’t be the whole story, as there must be zillions of such runways in the USA.
don't land on those runways!
I’m getting happier and happier that my usual airline (Swiss) has a mostly-Airbus fleet.
I for one will never ride on a 737 again, even if they manage to get them back off the ground. For every problem that's come to light there are probably two more than haven't been discovered (or disclosed).
Oh, it is going to be fun to see them get re-certified.
Did someone forget to run the fuzzer, maybe?