My dad headed up the redesign effort on the Lockheed Martin side to remove the foam PAL ramps (where the chunk of foam that broke off and hit the orbiter came from) from the external tank, as part of return-to-flight after the Columbia disaster. At the time he was the last one left at the company from when they had previously investigated removing those ramps from the design. He told me how he went from basically working on this project off in a corner on his own, to suddenly having millions of dollars in funding and flying all over for wind tunnel tests when it became clear to NASA that return-to-flight couldn't happen without removing the ramps.
I don't think his name has ever come up in all the histories of this—some Lockheed policy about not letting their employees be publicly credited in papers—but he's got an array of internal awards from this time around his desk at home (he's now retired). I've always been proud of him for this.
I worked in security for a while, but luckily on the vendor side and not the consumer side. The old yarn in that area is when everything (security wise) is fine, management asks you "What are we paying you for?". When it inevitably turns pear shaped they ask, "What are we paying you for?"
Many years ago I had a fascination with security and fancied becoming the CISO for the multinational I was working for at the time - my boss at the time, the CIO, said the role would really have no power and would be there as a sacrificial lamb should there actually be a serious security breach. This rather put me off the idea.
On the flip side, some companies have gone to extremes. I now have to MFA and provide a pin-code to authenticate. I have to do this several times a day. It's fucking mind-boggling how I can get anything done in a day when I spend so much time verifying who I am. I'm waiting for the next innovation...require a drop of my blood to log in.
Why is that extreme? I have to provide a pin code using MFA to my bank to authenticate, and their sessions are a lot shorter than your average developer or operator session.
And their actions impact far more than just my own account. Is it inconvenient? Yes. Does it work? Yes. Is it perfect? No, absolutely not but it is a useful layer in the cake.
Requiring a user to MFA once per day per device is normal for a work account - but that's already a lot compared to services like gmail.
After all, workers are mostly working in an access-controlled office or their private home; and your endpoint protection will be ensuring they're connecting from a company-issued laptop and that they have screen lock on a timer and a strong password.
I'm already validating something-they-know (FDE password) and something-they-know (OS password) and something-they-know (SSO system password) and something-they-have (company laptop). And once a day I'm validating another something-they-have (TOTP code/Yubikey).
Asking people to provide the second something-they-have several times a day seems like security theatre to me.
In retrospect, after the 2008 crash in the finance world how the role of a CISO was described to me sounded an awful lot like risk officers in a lot of financial organisations.
I've seen this go from bad to worse. A company had a bad project manager they couldn't get rid of, and required a security person by law, so they promoted him. The idea was he would get kicked out of the company the next time a security boo boo happened.
It went a lot worse. The guy had no idea about security and no common sense, and did genius things like forbidding encryption in the name of security (so the network people would be able to do packet inspection for monitoring security). But he created a morass of paperwork, and made it impossible for any project to make any kind of progress without involving security. End user computers slowed to unusable speed as he threw in more and more snake oil security software. As his rules were vague, dumb, self-conflicting and very very time consuming, nobody followed them, so he could always point to someone not following the rules when a security boo boo happened. He grew his department like a mushroom, wasted huge amounts of money, and entrenched himself completely, all based on sweet talk and complete nonsense. I've learned a lot about office politics watching him.
People that have done QA have the best understand of this.
I was doing the QA on a life safety product. Any new hardware would mean pulling the specs for the ICs and verifying that the layout and pin-out where correct on the product designs. You don't need an electrical engineering degree to know that PIN-1 on IC-2 should be connected to PIN-A on IC-4 but deigns having it traced to PIN-C.
Not once was there ever a recall and all early product issues where just a firmware update. After no longer working at the company they stopped doing QA. Heard from a former coworker that they latest product releases have required a compete recall.
QA not only saves lives it also reduces service and support costs. It helps keep a good standing relationship with your clients. Good QA is about trying to brake the solution as a consumer not reading the manual.
This isn't a failure of PowerPoint. I work for NASA and we still use it all the time, and I'll assure anyone that the communication errors are rife regardless of what medium we're working in. The issue is differences in the way that in-the-weeds engineers and managers interpret technical information, which is alluded to in the article but the author still focuses on the bullets and the PowerPoint, as if rewriting similar facts in a technical paper would change everything.
My own colleagues fall victim to this all the time (luckily I do not work in any capacity where someone's life is directly on the line as a result of my work.) Recently, a colleague won an award for helping managers make a decision about a mission parameter, but he was confused because they chose a parameter value he didn't like. His problem is that, like many engineers, he thought that providing the technical context he discovered that led him to his conclusion was as effective as presenting his conclusion. It never is; if you want to be heard by managers, and really understood even by your colleagues, you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I've seen this over and over again, and I'm starting to think it's a personality trait. Engineers are gossiping among themselves, saying "X will never work". They get to the meeting with the managers and present "30 different analyses showing X is marginally less effective than Y and Z" instead of just throwing up a slide that says "X IS STUPID AND WE SHOULDN'T DO IT." Luckily for me, I'm not a very good engineer, so when I'm along for the ride I generally translate well into Managerese.
I love it when some company gets one of the engineers to do a demonstration, you know you got an actual engineer because it will be the worst sales pitch you ever received. They will tell you in excruciating detail all the problems with their product. Recognize and cherish these moments for what they are worth, despite the terrible presentation it is infinitely more valuable than yet another sales rat making untenable promises.
It is something to do with that being the engineers actual job, to find and understand the problems with the product. so when talking to a customer, that is what tends to come across, all the problematic stuff. The good stuff that works, not important to them.
As a former sales engineer - it's more about setting expectations correctly. If the customer knows the shortcomings (as well as the benefits) and signs anyway it's usually a good partnership. If the customer finds them out after the contract, then it's the opposite.
it's truly a horrible thing that hearing the facts as they are, is considered excruciating. i'm very lucky that the company i've been working for for 6 years takes care of me exactly because i'm detailed, i say what i think when i think it, and have built a cult following amongst our customer base for being able to get to the meat of problems and solve them, or get at least on the path to resolution.
I was just reading a great discussion about how "academics use qualifiers as to how confident they are in the information" and you can see similar trends in spaces like hacker news.
But using "uncertain" language seems unconvincing to people outside of these types of cultures.
- bad communication possible in any medium
- pptx in NASA even today!
- issue is managers/SMEs communication differences
- issues with technical papers
- long
- boring
- vs word, excel, pdf...
(Next slide please)
Manager/SME Differences
- context vs conclusion
- tell a compelling story
- but give away the ending FIRST
- inherent personality differences
- motivations/incentives/mindsets
(Next slide)
Learning from disasters
- medium guides message and messenger
- blame tool - binary choice?
- presentation aide vs distributed technical artifact
Blaming the audience makes sense because after all, they're the ones not getting the message right and not asking the presenter to explain it better. But it remains the presenter's failure to catch their attention better and try to deliver a clear message.
Every time I had a presentation, I tried to analyze the failures (including listening to me when it was recorded, a really painful experience). Certain mistakes such as like having slides on a white background that makes attendees look at the screen and read instead of watching the presenter and listening to him can be devastating. Just because attendees are naturally attracted by light. It's not the audience's fault, it's the presenter's fault (and to some extents the tools in use). A good exercise is to stop slides from time to time during the presentation (i.e. switch to a black one), you'll be amazed how much you suddenly catch the attention, you feel like you're at a theater. It even manages to catch attention of those who were looking at their smartphones because the light in the room suddenly changes.
Also another difficulty which is specific to English native speakers is that many of them initially underestimate the difficulties of the audience to catch certain expressions (with some people it's very hard to distinguish "can" from "can't" for example, which complicates the understanding), or idiomatic ones, or references to local culture, because such things are part of their daily vocabulary. Of course, after a few public talk, when they get questions at the end proving there were misunderstandings, they realize that speaking slower, articulating a bit more and avoiding such references does help with non-native listeners. Conversely, when you present in a language that is not yours, you stick to very simple vocabulary using longer sentences to assemble words that try to form a non-ambiguous meaning. It can probably sound boring for native speakers but the message probably reaches the audience better.
In any case, it definitely always is the presenter's failure when a message is poorly delivered and their responsibility to try to improve this, however difficult this is. It's just important never to give up.
I think the problem is that most people, especially non-engineers, are over-selling and over-promising all the time. Being honest about risks, issues, and short-comings makes a project or product look bad in comparison.
The most feasible way to get X done is saying "X is a great option, the risks are managable, and it's fairly quick". Then, it will unexpectedly take a bit longer, plus some unforeseen trade-offs will need to be made.
Agree and was going to say the same thing. Messages need to be created for a specific audience.
When I'm sending an email to non-tech mgrs that has a bunch of tech details like that slide, I typically separate more detailed stuff from the conceptual message:
Summary:
System performance is not good enough for go live.
If nothing else it's quite hard/uncommon to print out a PowerPoint and read it carefully in a quiet room by yourself, I do this with written stuff all the time.
The whole point of PowerPoint is to pander to people who can't or don't want to sit down and carefully read a report. They want to sit back and passively consume information like they were watching TV. The problem then isn't so much PowerPoint itself, but rather these quasi illiterate people being in decision making positions. That's the real problem, and PowerPoint is just a symptom.
Speaking in a meeting, or delivering a talk in a larger context, often works better with visuals. Delivering information in this way is not "pandering" to people who don't or won't read a detailed paper. They're different contexts with different goals.
Before Powerpoint, having any kind of visual aid to a talk was incredibly onerous. You had to print up transparancies, or literally have SLIDES made, and the whole thing was just an enormous pain in the ass.
The PROBLEM here isn't Powerpoint, or the existence of visuals during a talk. It's that humans are bad at communicating generally, and that the use of slides during a talk is something many folks absolutely do NOT understand or do well.
You've been to a talk where the speaker basically just reads the slides, right? That's pointless. What you want is slides that compliment and amplify what's actually being said, not duplicate it. You also want slides that "scan" well -- if your audience has to pause and read a 150 words on a slide, you've fucked up. (DoD and defense industry presentations are INFAMOUS for this, btw.)
Technical papers convey technical information more effectively than PowerPoint presentations, period. The only edge PowerPoint has comes from managers refusing to read technical papers, making the PowerPoint literally better than nothing.
>if you want to be heard... and really understood ... you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I question your premise. :J
I'm just kidding, that's interesting I'll have to think about applying that. I don't suppose that would translate over to blogging? The fear of course is that one makes a statement and the commentariat thinks the speaker is full of it for not having provided backup instead of questioning. Maybe it's dependent on what type of forum it is.
I feel that. I looked at the powerpoint from the post and I cringed because it looks just like the kind of thing I could have written, with half finished thoughts all over the place masquerading as something a decider could use...
This article (as it makes clear) owes it's analysis at least largely to what Tufte has written about the Challenger disaster (1986) and Columbia Disaster (2003). He wrote about the Columbia one more fully in the second edition of The Cognitive Style of Powerpoint.
Given that the link in the article to his report on his website is now broken, people might be interested in teh few page grabs that he has included in the "comments" on his site here[0].
See also the article that he has re-posted under the "comments" section on his page on the matter[1].
The full report (2003 edition, low-res) is available on ResearchGate. It appears to be a lawful copy, uploaded by the author himself. Fascinating reading, indeed.
That link is the chapter "The Cognitive Style of PowerPoint" from Tufte's book Beautiful Evidence, and it does mention Boeing's slides in the Columbia incident, but the main work that the author of this blog post cribbed (and failed to grasp) is a more detailed essay by Tufte called "PowerPoint Does Rocket Science: Assessing the Quality and Credibility of Technical Reports".
I don't see how this has anything to do with PowerPoint. There wasn't clear communication; the medium was completely incidental to that. They could have been writing on a chalkboard and had a communication failure, does that mean that chalkboards should be blamed in that case?
Because the medium is not conducive to dense amount of technical information that readers are expected to use to make or understand decisions. Other similar mediums like a chalkboard were not singled out because the problem was identified with PowerPoint specifically. And it wasn't a choice of mediums all with similar problems, but slides vs papers. From the article,
> “The Board views the endemic use of PowerPoint briefing slides instead of technical papers as an illustration of the problematic methods of technical communication at NASA.”
But the problem, if anything, was that too much dense information was conveyed at all. Based on the analysis in the post, of the engineers had replaced that slide with one that said "Don't go forward with reentry", that might have saved lives better than any change in medium. To be clear, I'm in favor of abolishing PowerPoint for any non-ephemeral use, but the problem here was focus and framing of the info.
* Our foam collision dataset from experimentation only included pieces below X cu in.
* Evidence points to this piece being at least Y cu in - 200 times more massive
* Catastrophic damage to the wing cannot be ruled out
This would have been a great PowerPoint, and I'm not convinced handing them only an academic paper with dozens of pages of facts and figures would have had the effect that my above deck would have had.
Yeah, a slide like that would have been fine; fundamentally the slide is exactly like an academic paper in that some of the most interesting implications are just banal statements of the numbers in the data and qualifiers. In fact, the slide is much better than the academic paper... it doesn't contain much in the way of irrelevant data and qualifiers.
In practice Tufte and bloggers and commenters are retconning messages engineers not possessing foreknowledge of what was going to happen didn't wish to convey. The slide isn't supposed to say "no reentry" not because engineers don't know how to say no using PowerPoint, but because what the engineers are actually saying by selecting those points for consideration is "damage is theoretically possible but not in our simulations which test data suggests are actually on the conservative side; the test data is only at a very small scale though". If they'd dumbed it down, the slide would have said "it could go wrong but the limited data we've got suggests it won't"
I'm basically with you and let's be clear, I know nothing about the domain. But the word "limited" is doing a lot of work in that last sentence. Maybe a diagram showing relative size of the biggest foam chunk they'd tested vs. the size of the actual one would have been useful.
Agree. Though to be honest I still think a paper with an executive summary that said "Don't go forward etc" would have probably been even better. Then the powerpoint slides can be illustrations of how far outside the testing data this is, simulations of possible damage, and other, you know, useful stuff.
The person I was replying to said they had no idea how it had anything to do with PowerPoint. The article quotes the report as identifying over-use of it as one of the problems that contributed to the communication break down.
I'm not making the argument and I'm not interested in engaging with this quibbling, I'm just explaining how the article said the expert who conducted the investigation found a problem with their use of PowerPoint. If you have a problem with that conclusion, then take it up with the investigation report, not me. I would be fascinated to see you provide a rebuttal of it.
The engineer in question was probably trained by the organization to leave decision-making to management. In organizations like that you refrain from statements that sound like decisions.
Saying “more testing must be done before deciding to re-enter” would be equally valid.
Would it be better if you sent them a PDF document instead? There seems to be an assumption here that if you send the stakeholders a larger volume of information they will take the time to read it. Is that a valid assumption?
Memos and reports also ask the author to try to explain things clearly and at length, a PowerPoint, even a technical one is usually figures and bullet points
Are you asking me what I (a layperson having no knowledge of the organizations or projects) think, or are you asking me to look up the recommendations in the report that found over use of PowerPoint to have contributed to the communication problems at NASA for you?
Feynman communicated the problem with the Challenger disaster using a rubber band and a glass of ice water.
I don't think PowerPoint is the problem in and of itself, but rather its use as a crutch to compensate for poor communication. Of course, even among scientists, few can count themselves at Feynman's level in terms of communication skills. Maybe this is a skill that NASA scientists need to brush up on, perhaps with Pluralsight courses or something? lol
Speaking of chalkboards, next time you have to give a presentation, bring a chalkboard and do your slides in realtime. Something about the visual show, auditory overload, and not least the novelty of the act makes it much more impactful and memorial than "another powerpoint that puts you to sleep"
White boards are... ok... better than powerpoint but still fail to sell it like a chalkboard does. I think it is the noise.
Yes, the noise (which I'd call "the sound") is a big factor.
I teach in a classroom that had blackboard that had stood the test of time for decades. When it was replaced with a whiteboard, things went downhill. The markers dry out quickly, without much notice, so that students often have trouble reading the material. And the whiteboards get harder to erase year after year.
I guess the advantage of whiteboards is that a variety of colours can be used. But some students have deficiencies of colour recognition, so that's not really helpful. (I never used coloured chalk, for the same reason. Maximal contrast is the key.)
And the noise. That click drag click of chalk. Students after the transition to whiteboards told me that they really missed that. It enlivened the lectures. And when students were writing down notes, they knew to look up when they heard the sound.
Back to the point about the "visual show" and doing slides in real time. Yes, yes, yes. Once in a while I need to show something on the projector. The moment I turn it one, I see students start to disengage.
Definitely on the people with color issues. I have some red-green issues, for most purposes it makes me horrible at choosing colors and horrible at choices involving subtle colors (picking the good produce) but rarely interferes otherwise. But a thin line of color is another matter. I generally can't tell the difference between a 1-pixel wide line of FF0000 vs 00FF00. Nor can I tell the difference of a few-pixel-wide indicator. I can't crimp Ethernet because the fine line of the coloring of the wires reduces it to not much better than a guess.
And a whiteboard from across the room is likely to be thin enough lines to give me big problems.
PowerPoint gets used because it requires less effort from the audience. They sit back and zone out like couch potatoes. Scrap the PowerPoint and throw the technical report at the managers. Any of them who complain or otherwise don't read it are incompetent and should be fired on the spot.
I think it's a lot harder to have this particular type of communications failure if you're writing on a chalkboard. Imagine trying to write out that whole slide, it would take forever. If you really did have to present that information on a chalkboard, you'd be significantly more likely to write something along the lines of:
"We checked the test data: possible to damage tiles significantly"
"Foam that hit wing was way bigger than the tests"
Obviously you can miscommunicate via any medium, but I think the author's point here (which I agree with) is some mediums lend themselves to specific types of miscommunication.
Yeah, the choice to gloss over the point "our tests are not relevant" was a deliberate one. If it was in a paper you'd have big fancy graphs of the tests and you'd have to do your own work to compare the x axis against a mention of the actual scale in question in another paragraph. It's not as if they started with "Warning: even the 600X smaller bits we tested can damage the wing" and microsoft just kind of spontaneously grew a bunch of random stuff above the fold. It's a kind of chickenshit communication which you can do in any medium. The point they ought to be making is not dense or technical, it is so simple a child could understand.
This was an interesting article but it doesn't really provide solutions. I watched a few tech talks teaching a new API. Most slides were split, left side bullet poitns, right side either code or an image. As I was watching I was thinking "isn't this supposed to be almost the worst style"? but I was also thinking "I can't think of any way to do this better". It's an API. It requires examples. And it requires something describing what to concentrate on, what the example or image is showing.
I've been the plenty of great talks with just images, no words. But they fit the type of talk. I'm not sure an API talk would be better without bullet points. If you know of some to reference, please post links.
Tufte did make specific recommendations that one should prepare a real document that your audience can and should read, and that they would have in front of them during the meeting. I'm not sure how best to translate that to your API example.
What would make the most important point of that slide stand out any more in a "real document" than in a slideshow? If anything, I would expect it to be buried even more - a slide and limited-time presentation forces you to be concise, while in a document there tend to be few limits on length.
I would say the disaster occurred despite PowerPoint, not because of it. It's not clear to me at all why the slide author thought all that text was needed, when it seems to communicate almost nothing. If anything I would blame it on the culture around "real documents", where having more information is treated as better (probably because they serve multiple functions - to educate, but also as a record of activities), even if it makes it bloated and hard to read.
I found it surprising that the slide in the article uses Calibri, a typeface that wasn’t publicly available at the time. The original discussion confirms that the slide in the article is a recreation of the original one:
> The slide in the article has the same text, but is a recreation of the original (The Calibri typeface used wasn't part of PowerPoint until 2007).
> The original slide can be seen in the full report linked in the article:
No, that's not how the physics works. The foam is moving at the same velocity as the shuttle when it breaks off, and had a short time to accelerate(decelerate) before hitting the shuttle.
THANK YOU. I've seen the velocity of the space shuttle quoted as the speed that the foam had when it hit Columbia's wing so many times, and it bugs the crap out of me.
If that was the actual impact velocity of the foam, there would not be any doubt about whether the shuttle would survive reentry, that is if it even managed to make it all the way into space.
The slide is so bad it might be intentional. The intent would be to be reassuring enough that no one will point at Boeing and say "Boeing technical failure strands astronauts in space", while providing just enough information that in the event (at the time of writing the slide considered very unlikely) things go bad, they can say that nothing was withheld.
Yeah, the problem here isn't Powerpoint, it's what it said. This is a case of shoot the messenger. Powerpoint doesn't hinder communication, it lets people think they understand when their actual knowledge is superficial and can easily be a case of the primrose path.
Some asked DoD to use satellites to see the damage. Linda Ham declined the request.
"The imagery request was soon rescinded by NASA Mission Management Team Chair Linda Ham after she investigated the origin of it. She had consulted with Flight Director Phil Engelauf and members of the Mission Management Team, who stated that they did not have a requirement for imagery of Columbia. Ham did not consult with the Debris Assessment Team, and cancelled the imagery request on the basis that it had not been made through official channels."
https://en.m.wikipedia.org/wiki/Space_Shuttle_Columbia_disas...
A 2008 episode of the PBS NOVA program covers the Columbia disaster. It does not get into the focus of the article posted here, but it does well covering how poorly the situation was handled (along with other things like the broad history of the Shuttle program).
Everyone should note that the slides in this blog post are fabricated, and the author of this post misrepresents Tufte and the conclusions of the review board—subtly pointing people in a direction that's the exact opposite of the real takeaways from the investigation. It's almost insidious.
The first time I came across this blog post a few years ago, I found it enraging, and I still do. As I said I said in my personal notes at the time, "The choice to use this fabricated material for this article and the false claims in it—claims not actually confirmed by the sources that it cites—amounts to what would be considered serious misconduct elsewhere." But there's no real accountability here, of course, because at the end of the day it's just some jerk with a blog serving up distorted pop science insights to aspiring entrepreneurs.
> Why, given that the foam strike had occurred at a force massively out of test conditions had NASA proceeded with re-entry?
What was the alternative?
Columbia could not have made it to ISS.
Columbia could not have repaired the damage in orbit.
Columbia could not have lasted, after two weeks in space, long enough to launch a rescue mission.
I know the "In Flight Options Assessment" said they could launch at an accelerated pace but the assessment assumes that it's ok to launch another vehicle with the same problem, no fix, and no completed analysis of the cause.
Yeah, they suspected the external tank bipod foam, but WHY did the foam come off? Was it a fluke? Had some unknown factor not present in previous external tank bipod foam applications but now present in all external tank bipod foam applications manifested?
>Two major assumptions, apart from the already stated assumption that the damage had to be visible, have to be recognized – the first is that there were no problems during the preparation and rollout of Atlantis, and the second is the question of whether NASA and the government would have deemed it acceptable to launch Atlantis with exposure to the same events that had damaged Columbia. At this point, at least two of the last three flights (STS-112 and STS-107) had bipod ramp foam problems, and the flight in-between these two, STS-113, was a night launch without adequate imaging of the External Tank during ascent.
This is from a pre-flight safety report for STS-113
>“More than 100 External Tanks have flown with only 3 documented instances of significant foam loss on a bipod ramp”
STS-1 through STS-111, April 1981 - June 2002: three "significant" bipod foam losses
STS-112, October 2002: significant foam loss
STS-113, November 2002: night time, but they saw 112 and went "oh shit" and wrote a report
STS-107, January 2003: yet another, fatal, significant foam loss
If two of the last three flights had foam problems and the one that didn't only didn't because you couldn't see if it did, and over 100 of the preceding flights only had three, you don't risk four more lives.
Apollo 13 probably seemed pretty hopeless. They didn't just say "sorry fellas, maybe you'll get lucky and make it, good luck."
The late great Bob Hoover said, "If you’re faced with a forced landing, fly the thing as far into the crash as possible." You keep trying until you can't anymore.
Saying "well, maybe they'll survive," and not even telling them, is not the move when you have nearly a month to figure something out.
They did a bunch of studies. While it was POSSIBLE to get a rescue shuttle up to them if they ignored a bunch of safety and refurbishment procedures, the sheer amount of complexities probably meant they would lose 2 shuttles and 2 crew.
The second shuttle would've flown with a crew of two, working with the knowledge that their ship was even more vulnerable than the one they intended to rescue.
> There was actually an exercise done to work this out, at the direction of the Columbia Accident Investigation Board (CAIB). [...] In the CAIB’s scenario, Atlantis would have launched with a four-person crew: two pilots, and two EVA mission specialists. [...]
> A Columbia rescue mission would have been the most monumentally difficult and epic space mission in history, and it would have required absolutely everything going right to bring the crew home safely. But NASA has shown time and again its ability to rise to the occasion and bring its formidable engineering and piloting expertise to bear. Instead, the worst instincts of the agency - to micromanage and engage in wishful thinking instead of clear-eyed analysis - doomed the crew.
The problem with sending up another shuttle is they hadn't actually fixed the underlying foam problem. If there was another foam strike on the rescue shuttle, now they have two craft in orbit with crew on that can't re-enter
One option would have been to place whatever high melting point metal tools they could spare into the hole, and freeze water around them to hold it in place. It also might have been possible to change the series of s-curves and other maneuvers done during re-entry, in order to lessen the heating on the left wing.
Source? It would be solid water ice, mostly surrounded by shuttle wing structure, in a low-g reentry. Not an "ice & volatiles & etc." meteor in a high-g situation.
- and noted no possibility of explosion, except in the case of a hypersonic tumble. A hypersonic tumble would shred the shuttle orbiter anyway, due to the extreme aerodynamic forces.
Note that the people making these slides are one the selling side, so they have a natural incentive to downplay the problem, and the slide smells a bit like that to me.
(And by selling side I don't necessarily mean sales people, but also the engineers involved in designing, building, or at least working on the product.)
I've been slowly refining a pitch deck over the past couple years and the feedback from reviews and test pitches has strongly reinforced for me just how important it is for slides to be short and laid out precisely.
You want the most important information in the right places, communicated with as few words as possible, using the most accurate words possible.
You want the key takeaways to be the things that people are most likely to remember from each slide.
You want to minimize distractions and try not to pollute slides with a bunch of vaguely related stuff. A crowded slide risks communicating nothing.
It's a real dramatic change compared to how I am used to using powerpoint for technical audiences or when I had to make presentations during school.
One of the dynamics is that if there is too much text they’ll read instead of listening to you. Another is that if the conclusion is on the deck, they’ll get bored waiting for you to get to the punchline they’ve already read, and that can lose your audience as well.
So you’re better off either opening with the conclusion or putting it on the next slide, but accidentally jumping two slides forward can still ruin your audience’s attention span. So sometimes it’s better for it not to be in the deck at all.
it's also important to know your audience. If they've spent their career reading technical papers or dense prose, that's one thing.
If they've largely grown up in the social media era and click away from reels/shorts that don't have animated captions, you'd design a very different deck.
Both Columbia and Challenger were quite preventable, but the problems were basically ignored because they clearly hadn't destroyed the orbiter. Never mind that both showed random behavior outside the design spec, sooner or later the orbiter was going to roll a 1. They had seen the blow-by that killed Challenger, they had seen orbiters scoured by the foam before.
However, it didn't really have a solution. The blow-by problem that destroyed Challenger was fixable, the foam problem was not. You could reduce the chance of a problem but nothing could be done about the fundamental problem of having parts of the heat shield aerodynamically behind other parts of the spacecraft, especially cryogenic parts of the spacecraft. (Ice could also do damage.)
The shuttle needed extensive refurbishment between flights. It wasn’t until after the loss of Columbia that they examined heat shield damage in orbit, assuming any damage observed on the ground would have occurred during reentry. For an organisation so risk averse as NASA, I was surprised how little focus was placed on the orbiters as experimental vehicles.
There is another point to be made about insisting on the reentry - Columbia didn’t have propellant to reach the ISS, and life support couldn’t be extended until a rescue mission could be mounted. A rescue mission would be further complicated because the docking hardware was not needed for this mission and was removed, and there was no airlock for EVAs (and no EVA suit on board), again, because it wasn’t part of the mission.
If a rescue mission could have been mounted, it’d be something straight out of science fiction - tying ships together with cables and shuttling astronauts in their orange pressure suits through the side doors of two disconnected shuttles.
For once I will defend PowerPoint: 1/ the slide is atrocious, even by 2003 standard 2/ you don't decide on the life or death based on a crappy slide you don't understand.
As an engineer, that slide looks completely reasonable to me. Its purpose was to communicate technical info, which it did adequately. Keep in mind that the subject matter is highly technical, given that we're literally talking about the Space Shuttle, and more than a high school level of reading comprehension is heavily implied. If the NASA personnel weren't competent enough to review technical data without a pithy summary, that's on them.
"That's on them" is not acceptable for an engineer when lives are on the line. Part of your job is making people understand what they need to understand. If their lack of understanding means people die, then you need to do your job and figure out how to communicate effectively to the audience you have, not the audience you want.
There’s a famous speech someone related from their civil engineering professor, where the professor basically said, if I pass you in this class I am effectively giving you a license to kill. So some of you will not be passing.
My CS undergrad was in the engineering college and so I had a mandatory engineering seminar that was basically "don't get people killed with your work." We covered Challenger, Hyatt Regency, and some other classic failures. I've mostly avoided working on life-critical software so it's not an immediate concern, but that sense of responsibility still stuck with me.
Also attended an engineering school. I had to take way too much chem and physics. It was weird.
I can be kind of a pain in the ass when it comes to details so I’ve worked on a couple such projects. It’s sobering, but also I think, “better me than” half a dozen corner cutters at my last two jobs. They could do much worse.
That said, I stayed on a commercial aerospace project about 14 months after I didn’t really want to be there because people kept saying the wrong things in meetings and thinking they sounded right.
The assumption of competency goes both ways. The NASA personnel should have been able to understand a very standard slide in their field, that any college-educated fluent English speaker would have been able to grasp.
First of all, is that actually their level of education or are you just making stuff up?
Second, that's irrelevant to my point that the engineer is responsible for communicating, not just figuring stuff out. You cannot say "if you don't get it, that's your problem" when their not getting it means people die.
The slide in the OP is a completely standard way of commumicating information in the aerospace industry. If the NASA personnel had problems understanding this slide, then they also had problems understanding virtually every other piece of technical info that was ever communicated to them by a third party. College level reading comprehension means being able to understand nuance, which this slide conveys.
Oh I thought you were talking about NASA and the Columbia disaster. The PowerPoint slide was poorly designed, as it says in the story. The problem was not PowerPoint though. You could summarize it using a PDF, and it will still have the problem if key issues don't grab attention.
The point that seems lost on HN, who evidently are incapable of consuming information that isn't presented in a format designed by Jony Ive himself, is that there's nothing wrong with the format of the slide to the kind of skilled subject matter expert in the aerospace industry that should have been reviewing it on NASA's behalf.
But that's not how it works in most large organizations. The person who has the authority to make the decision is not necessarily an SME in all of the areas they have to make decisions. Rather, they rely on SMEs to do the investigation and then... communicate properly.
For reference, NASA asked the group of aerospace engineers ("rocket scientists") who designed the Space Shuttle to provide them with a technical analysis of the effects of foam impacts, and then subsequently lost the Columbia with all hands because they couldn't exercise more than a high school level of reading comprehension. The engineers presented the facts as they were understood, only stopping short of doing NASA's job for them by deciding to abort reentry.
If you can't consume the PhD-level analysis that you requested because you only have a high school reading level, despite having been tasked with safely operating what was arguably the most technically complicated piece of technology in existence at the time, then there's really no other excuse for that level of incompetence.
Let's presume you are correct. The Boeing engineers who made this slide were geniuses. The NASA managers consuming it were dullards. If the Boeing engineers were so smart, why didn't they write for their audience?
I am consistently ranked as both one of the best engineers in my company and one of the best communicators. I would never make a slide like that for any audience. It's shit writing for non-engineers and it's shit writing for engineers, too.
You have got to get over whatever this hangup is about "PhD level analysis". That phrase doesn't mean anything.
I'm not arguing that it was a good slide: just extremely average in the context that it was created. If there was a problem with communication, then it was solely on the reader for not being fluent in their own field. The engineers were presenting a nuanced view of their data, which went above the heads of the NASA personnel who were evidently only capable of interpreting a simple "yes or no" answer.
They should have made it abundantly clear that they had no idea what was going to happen and that loss of the crew and shuttle was a real possibility, but I agree with you.
This slide was presented with a verbal talk track, and anyone who can't handle focusing on the topic because the slide is boring shouldn't be in a position of responsibility.
There's a near infinite quantity of technical information to choose from. The purpose is to emphasize what is important, and de-emphasize the unimportant. When I look at that slide, I see 95% utterly irrelevant information, and a teeny tiny note saying, in vague and indirect language, that the impact was 600x worse than any test.
So on the remote chance you're not just trolling: If you're doing anything safety critical, please quit your job before you kill someone. You vastly overestimate human's (including yours) ability to process information. I am being 100% sincere.
I can't remember the last time I saw a slide as mangled as the one in the article. It hurts my brain just reading it.
But you are right, most engineers would consider that reasonable, while complaining about the "muggles" that just don't get it.
As a Software Architect, one of my main responsibilities has been to take information presented like above and turn it into something that non-technical people can digest.
Being able to express a complex concept in simple terms is an invaluable skill.
Actually most engineers would complain about the slides shown here. The issue is not the technicality or depth of the content but on the way it is presented and shown. I'm
it depends, I noticed that many engineers will input information on a slide following their thought process closely, they rarely think about the audience's perspective, especially if the audience is less technical or not familiar with the domain.
Sure mate, because taking a messy list of confusing statements and turning it into something like: "High risk of failure on re-entry - foam strike more than 600 times bigger than test data - test data unfit to support decision as flight condition is significantly outside of test database" requires a PhD in Materials Science.
If you can't effectively communicate how the results (or lack of results) of your research will impact the outcome of a high-stakes space mission you have no business being in that room from the start.
The slide shown here is completely horrendous, even by 2003 standard. I've worked in a field as technical as this one for a long time, and this slide would not pass a review, even with people familiar with the content.
While it does not wash the responsibility of the executives, the engineers have also the responsibility to be clear in their communication
I wish this was satire. The slide is full of spelling mistakes and all kinds of horrible communication. Even when I know it documents an issue that killed people I still struggle to read it. Only the bottom part hints at anything relevant.
That's a completely unremarkable slide in the aerospace industry. If there was a communication problem, it was with the NASA personnel not being able to operate in their own field.
I spent too much if my early career dealing with the consequences of bad decisions made on “good data”. If the presentation was so good why did they come to the wrong conclusions? It became part of my overall thesis that good design invites you to use a thing the proper way even if you failed to read the instructions.
You can’t force someone to think but you can force a lot of people not to, or you can make it difficult to avoid. It’s worth investing the energy into stacking the deck the right way.
Another example of "guy can't spell" => "he's retarded". If you can't get everything right, you probably got lots of things wrong.
It also shows why software engineers are superior engineers. The history of software engineering has some single-digit kills, at most 20 in total. Meanwhile, shuttle engineers are supposed to be the best in the aerospace business and they've lost some 13 or so? All aerospace would be put in the thousands.
An old saying is "Any fool can build a bridge. It takes an engineer to build a bridge that barely stands". But these are fools who built bridges that didn't stand.
One day, I will teach them real engineering: It is Rails backend with React frontend. Zero kills. Life above all.
That's a good counterexample. We can count that in software engineering. Can't say I expected Boeing software engineers to be any good. Dregs of the field but they still count as in the field. Aerospace software engineering quality substantially lacking, it's true.
> One day, I will teach them real engineering: It is Rails backend with React frontend. Zero kills. Life above all.
Buddy, if you think that is "real engineering", you've got a lot of things to learn.
Even in software engineering, rails and react aren't really engineering. Real engineering includes calculations -- data structures, algorithms, data structures, and user interfaces designed to work with people who are deaf or blind. You wanna tell me that rails is performant? Have you done calculations? You wanna tell me react's algorithms are well-defined in memory and time? Have you ensured that your "software" is usable by people with disabilities?
My dad headed up the redesign effort on the Lockheed Martin side to remove the foam PAL ramps (where the chunk of foam that broke off and hit the orbiter came from) from the external tank, as part of return-to-flight after the Columbia disaster. At the time he was the last one left at the company from when they had previously investigated removing those ramps from the design. He told me how he went from basically working on this project off in a corner on his own, to suddenly having millions of dollars in funding and flying all over for wind tunnel tests when it became clear to NASA that return-to-flight couldn't happen without removing the ramps.
I don't think his name has ever come up in all the histories of this—some Lockheed policy about not letting their employees be publicly credited in papers—but he's got an array of internal awards from this time around his desk at home (he's now retired). I've always been proud of him for this.
It's funny how the thankless jobs of quality assurance become so critical so quickly. And I mean that ironically of course.
To folks out there: do the important work, not the glamorous work, and you'll not only sleep well, but you might actually matter as well.
Yes, but first it has to go horribly wrong. Same for security. After the breach there is plenty of budget.
I worked in security for a while, but luckily on the vendor side and not the consumer side. The old yarn in that area is when everything (security wise) is fine, management asks you "What are we paying you for?". When it inevitably turns pear shaped they ask, "What are we paying you for?"
Fun times.
It's a variation on the prevention paradox.
Many years ago I had a fascination with security and fancied becoming the CISO for the multinational I was working for at the time - my boss at the time, the CIO, said the role would really have no power and would be there as a sacrificial lamb should there actually be a serious security breach. This rather put me off the idea.
On the flip side, some companies have gone to extremes. I now have to MFA and provide a pin-code to authenticate. I have to do this several times a day. It's fucking mind-boggling how I can get anything done in a day when I spend so much time verifying who I am. I'm waiting for the next innovation...require a drop of my blood to log in.
Why is that extreme? I have to provide a pin code using MFA to my bank to authenticate, and their sessions are a lot shorter than your average developer or operator session.
And their actions impact far more than just my own account. Is it inconvenient? Yes. Does it work? Yes. Is it perfect? No, absolutely not but it is a useful layer in the cake.
Requiring a user to MFA once per day per device is normal for a work account - but that's already a lot compared to services like gmail.
After all, workers are mostly working in an access-controlled office or their private home; and your endpoint protection will be ensuring they're connecting from a company-issued laptop and that they have screen lock on a timer and a strong password.
I'm already validating something-they-know (FDE password) and something-they-know (OS password) and something-they-know (SSO system password) and something-they-have (company laptop). And once a day I'm validating another something-they-have (TOTP code/Yubikey).
Asking people to provide the second something-they-have several times a day seems like security theatre to me.
Your comment should be required reading for any CISO that finds themselves without mandate, budget or support from upper management.
In retrospect, after the 2008 crash in the finance world how the role of a CISO was described to me sounded an awful lot like risk officers in a lot of financial organisations.
Head of Quality Assurance is often also treated as ablative armor for existing management.
That's a very poetic description.
The Risk Management manager character played by Demi Moore in the "Margin Call" movie is another example of this in the financial industry.
Awesome movie.
I've seen this go from bad to worse. A company had a bad project manager they couldn't get rid of, and required a security person by law, so they promoted him. The idea was he would get kicked out of the company the next time a security boo boo happened.
It went a lot worse. The guy had no idea about security and no common sense, and did genius things like forbidding encryption in the name of security (so the network people would be able to do packet inspection for monitoring security). But he created a morass of paperwork, and made it impossible for any project to make any kind of progress without involving security. End user computers slowed to unusable speed as he threw in more and more snake oil security software. As his rules were vague, dumb, self-conflicting and very very time consuming, nobody followed them, so he could always point to someone not following the rules when a security boo boo happened. He grew his department like a mushroom, wasted huge amounts of money, and entrenched himself completely, all based on sweet talk and complete nonsense. I've learned a lot about office politics watching him.
We had a meaningful amount of {industrial accident happened} added to the pipeline every year. We made outdoor lighting.
Serious injuries or deaths is a terrible feeling, even if the end result was better safety for the rest of the workers.
Like the old saying, the firefighter is a hero but everybody is annoyed by the fire inspector, even though he saves way more lives.
People that have done QA have the best understand of this.
I was doing the QA on a life safety product. Any new hardware would mean pulling the specs for the ICs and verifying that the layout and pin-out where correct on the product designs. You don't need an electrical engineering degree to know that PIN-1 on IC-2 should be connected to PIN-A on IC-4 but deigns having it traced to PIN-C.
Not once was there ever a recall and all early product issues where just a firmware update. After no longer working at the company they stopped doing QA. Heard from a former coworker that they latest product releases have required a compete recall.
QA not only saves lives it also reduces service and support costs. It helps keep a good standing relationship with your clients. Good QA is about trying to brake the solution as a consumer not reading the manual.
[dead]
Well, I’m proud of him, too. Thank him for helping us return to the stars.
I hope he knows you are proud of him.
This isn't a failure of PowerPoint. I work for NASA and we still use it all the time, and I'll assure anyone that the communication errors are rife regardless of what medium we're working in. The issue is differences in the way that in-the-weeds engineers and managers interpret technical information, which is alluded to in the article but the author still focuses on the bullets and the PowerPoint, as if rewriting similar facts in a technical paper would change everything.
My own colleagues fall victim to this all the time (luckily I do not work in any capacity where someone's life is directly on the line as a result of my work.) Recently, a colleague won an award for helping managers make a decision about a mission parameter, but he was confused because they chose a parameter value he didn't like. His problem is that, like many engineers, he thought that providing the technical context he discovered that led him to his conclusion was as effective as presenting his conclusion. It never is; if you want to be heard by managers, and really understood even by your colleagues, you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I've seen this over and over again, and I'm starting to think it's a personality trait. Engineers are gossiping among themselves, saying "X will never work". They get to the meeting with the managers and present "30 different analyses showing X is marginally less effective than Y and Z" instead of just throwing up a slide that says "X IS STUPID AND WE SHOULDN'T DO IT." Luckily for me, I'm not a very good engineer, so when I'm along for the ride I generally translate well into Managerese.
I love it when some company gets one of the engineers to do a demonstration, you know you got an actual engineer because it will be the worst sales pitch you ever received. They will tell you in excruciating detail all the problems with their product. Recognize and cherish these moments for what they are worth, despite the terrible presentation it is infinitely more valuable than yet another sales rat making untenable promises.
It is something to do with that being the engineers actual job, to find and understand the problems with the product. so when talking to a customer, that is what tends to come across, all the problematic stuff. The good stuff that works, not important to them.
As a former sales engineer - it's more about setting expectations correctly. If the customer knows the shortcomings (as well as the benefits) and signs anyway it's usually a good partnership. If the customer finds them out after the contract, then it's the opposite.
Hence the role of "sales engineer" - who the customer things is from engineering but is really part of the sales team.
Right. They “engineer” “making the sale happen”.
it's truly a horrible thing that hearing the facts as they are, is considered excruciating. i'm very lucky that the company i've been working for for 6 years takes care of me exactly because i'm detailed, i say what i think when i think it, and have built a cult following amongst our customer base for being able to get to the meat of problems and solve them, or get at least on the path to resolution.
I was just reading a great discussion about how "academics use qualifiers as to how confident they are in the information" and you can see similar trends in spaces like hacker news.
But using "uncertain" language seems unconvincing to people outside of these types of cultures.
Also of course the power dynamics.
Oh yeah, I've certainly seen this before. I'll assess my ability to complete a project under a time frame as "reasonably confident."
In my mind, I'm thinking "so long as a meteor doesn't cause an extinction event," but the manager graciously pushes the target date back a week.
PowerPoint actually fine
(Next slide please)Manager/SME Differences
(Next slide)Learning from disasters
(Next slide)Questions?
Some time ago, I made up a PowerPoint show on effective communication[0].
I’ve found that most folks have no intention of improving their communication effectiveness. Everyone is much happier, blaming the audience.
[0] https://news.ycombinator.com/item?id=44202502
Blaming the audience makes sense because after all, they're the ones not getting the message right and not asking the presenter to explain it better. But it remains the presenter's failure to catch their attention better and try to deliver a clear message.
Every time I had a presentation, I tried to analyze the failures (including listening to me when it was recorded, a really painful experience). Certain mistakes such as like having slides on a white background that makes attendees look at the screen and read instead of watching the presenter and listening to him can be devastating. Just because attendees are naturally attracted by light. It's not the audience's fault, it's the presenter's fault (and to some extents the tools in use). A good exercise is to stop slides from time to time during the presentation (i.e. switch to a black one), you'll be amazed how much you suddenly catch the attention, you feel like you're at a theater. It even manages to catch attention of those who were looking at their smartphones because the light in the room suddenly changes.
Also another difficulty which is specific to English native speakers is that many of them initially underestimate the difficulties of the audience to catch certain expressions (with some people it's very hard to distinguish "can" from "can't" for example, which complicates the understanding), or idiomatic ones, or references to local culture, because such things are part of their daily vocabulary. Of course, after a few public talk, when they get questions at the end proving there were misunderstandings, they realize that speaking slower, articulating a bit more and avoiding such references does help with non-native listeners. Conversely, when you present in a language that is not yours, you stick to very simple vocabulary using longer sentences to assemble words that try to form a non-ambiguous meaning. It can probably sound boring for native speakers but the message probably reaches the audience better.
In any case, it definitely always is the presenter's failure when a message is poorly delivered and their responsibility to try to improve this, however difficult this is. It's just important never to give up.
I think the problem is that most people, especially non-engineers, are over-selling and over-promising all the time. Being honest about risks, issues, and short-comings makes a project or product look bad in comparison.
The most feasible way to get X done is saying "X is a great option, the risks are managable, and it's fairly quick". Then, it will unexpectedly take a bit longer, plus some unforeseen trade-offs will need to be made.
> This isn't a failure of PowerPoint.
Agree and was going to say the same thing. Messages need to be created for a specific audience.
When I'm sending an email to non-tech mgrs that has a bunch of tech details like that slide, I typically separate more detailed stuff from the conceptual message:
Summary:
System performance is not good enough for go live.
Working on a few possible solutions.
Details (for those interested):
System x is connected to ...blah blah blah...
If nothing else it's quite hard/uncommon to print out a PowerPoint and read it carefully in a quiet room by yourself, I do this with written stuff all the time.
The whole point of PowerPoint is to pander to people who can't or don't want to sit down and carefully read a report. They want to sit back and passively consume information like they were watching TV. The problem then isn't so much PowerPoint itself, but rather these quasi illiterate people being in decision making positions. That's the real problem, and PowerPoint is just a symptom.
That's not really true.
Speaking in a meeting, or delivering a talk in a larger context, often works better with visuals. Delivering information in this way is not "pandering" to people who don't or won't read a detailed paper. They're different contexts with different goals.
Before Powerpoint, having any kind of visual aid to a talk was incredibly onerous. You had to print up transparancies, or literally have SLIDES made, and the whole thing was just an enormous pain in the ass.
The PROBLEM here isn't Powerpoint, or the existence of visuals during a talk. It's that humans are bad at communicating generally, and that the use of slides during a talk is something many folks absolutely do NOT understand or do well.
You've been to a talk where the speaker basically just reads the slides, right? That's pointless. What you want is slides that compliment and amplify what's actually being said, not duplicate it. You also want slides that "scan" well -- if your audience has to pause and read a 150 words on a slide, you've fucked up. (DoD and defense industry presentations are INFAMOUS for this, btw.)
Technical papers convey technical information more effectively than PowerPoint presentations, period. The only edge PowerPoint has comes from managers refusing to read technical papers, making the PowerPoint literally better than nothing.
>if you want to be heard... and really understood ... you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I question your premise. :J
I'm just kidding, that's interesting I'll have to think about applying that. I don't suppose that would translate over to blogging? The fear of course is that one makes a statement and the commentariat thinks the speaker is full of it for not having provided backup instead of questioning. Maybe it's dependent on what type of forum it is.
>if you want to be heard
It would have been nicer if that had been the first sentence of that (interesting) comment.
I feel that. I looked at the powerpoint from the post and I cringed because it looks just like the kind of thing I could have written, with half finished thoughts all over the place masquerading as something a decider could use...
Yah they buried the lead on this one. This foam is 100x bigger than our tests so this must be manually verified
“The medium is the massage”
This article (as it makes clear) owes it's analysis at least largely to what Tufte has written about the Challenger disaster (1986) and Columbia Disaster (2003). He wrote about the Columbia one more fully in the second edition of The Cognitive Style of Powerpoint.
Given that the link in the article to his report on his website is now broken, people might be interested in teh few page grabs that he has included in the "comments" on his site here[0].
See also the article that he has re-posted under the "comments" section on his page on the matter[1].
[0]: https://www.edwardtufte.com/notebook/new-edition-of-the-cogn... [1]: https://www.edwardtufte.com/notebook/the-columbia-evidence/
If you haven't read it, I highly suggest you read Feynman's addendum to the Challenger disaster report:
https://www.nasa.gov/history/rogersrep/v2appf.htm
The words "a safety factor of three" will live with me for every day of my life.
"For a successful technology, reality must take precedence over public relations, for nature cannot be fooled."
> the few page grabs
The full report (2003 edition, low-res) is available on ResearchGate. It appears to be a lawful copy, uploaded by the author himself. Fascinating reading, indeed.
https://www.researchgate.net/publication/208575160_The_Cogni...
That link is the chapter "The Cognitive Style of PowerPoint" from Tufte's book Beautiful Evidence, and it does mention Boeing's slides in the Columbia incident, but the main work that the author of this blog post cribbed (and failed to grasp) is a more detailed essay by Tufte called "PowerPoint Does Rocket Science: Assessing the Quality and Credibility of Technical Reports".
<https://www.edwardtufte.com/notebook/powerpoint-does-rocket-...>
I don't see how this has anything to do with PowerPoint. There wasn't clear communication; the medium was completely incidental to that. They could have been writing on a chalkboard and had a communication failure, does that mean that chalkboards should be blamed in that case?
Because the medium is not conducive to dense amount of technical information that readers are expected to use to make or understand decisions. Other similar mediums like a chalkboard were not singled out because the problem was identified with PowerPoint specifically. And it wasn't a choice of mediums all with similar problems, but slides vs papers. From the article,
> “The Board views the endemic use of PowerPoint briefing slides instead of technical papers as an illustration of the problematic methods of technical communication at NASA.”
But the problem, if anything, was that too much dense information was conveyed at all. Based on the analysis in the post, of the engineers had replaced that slide with one that said "Don't go forward with reentry", that might have saved lives better than any change in medium. To be clear, I'm in favor of abolishing PowerPoint for any non-ephemeral use, but the problem here was focus and framing of the info.
I agree completely. My deck would have been:
Slide 1: 48-point font
Slide 2: 24-point font This would have been a great PowerPoint, and I'm not convinced handing them only an academic paper with dozens of pages of facts and figures would have had the effect that my above deck would have had.Yeah, a slide like that would have been fine; fundamentally the slide is exactly like an academic paper in that some of the most interesting implications are just banal statements of the numbers in the data and qualifiers. In fact, the slide is much better than the academic paper... it doesn't contain much in the way of irrelevant data and qualifiers.
In practice Tufte and bloggers and commenters are retconning messages engineers not possessing foreknowledge of what was going to happen didn't wish to convey. The slide isn't supposed to say "no reentry" not because engineers don't know how to say no using PowerPoint, but because what the engineers are actually saying by selecting those points for consideration is "damage is theoretically possible but not in our simulations which test data suggests are actually on the conservative side; the test data is only at a very small scale though". If they'd dumbed it down, the slide would have said "it could go wrong but the limited data we've got suggests it won't"
I'm basically with you and let's be clear, I know nothing about the domain. But the word "limited" is doing a lot of work in that last sentence. Maybe a diagram showing relative size of the biggest foam chunk they'd tested vs. the size of the actual one would have been useful.
Agree. Though to be honest I still think a paper with an executive summary that said "Don't go forward etc" would have probably been even better. Then the powerpoint slides can be illustrations of how far outside the testing data this is, simulations of possible damage, and other, you know, useful stuff.
> Based on the analysis in the post
The analysis in the post is dogshit and misrepresents the review board's actual conclusions.
> But the problem, if anything, was that too much dense information was conveyed at all
That's totally opposite to what the members of the review board identified as the problem.
The person I was replying to said they had no idea how it had anything to do with PowerPoint. The article quotes the report as identifying over-use of it as one of the problems that contributed to the communication break down.
I'm not making the argument and I'm not interested in engaging with this quibbling, I'm just explaining how the article said the expert who conducted the investigation found a problem with their use of PowerPoint. If you have a problem with that conclusion, then take it up with the investigation report, not me. I would be fascinated to see you provide a rebuttal of it.
The engineer in question was probably trained by the organization to leave decision-making to management. In organizations like that you refrain from statements that sound like decisions.
Saying “more testing must be done before deciding to re-enter” would be equally valid.
Would it be better if you sent them a PDF document instead? There seems to be an assumption here that if you send the stakeholders a larger volume of information they will take the time to read it. Is that a valid assumption?
Memos and reports also ask the author to try to explain things clearly and at length, a PowerPoint, even a technical one is usually figures and bullet points
Jeff bezos iirc speaks at length about this.
Are you asking me what I (a layperson having no knowledge of the organizations or projects) think, or are you asking me to look up the recommendations in the report that found over use of PowerPoint to have contributed to the communication problems at NASA for you?
Feynman communicated the problem with the Challenger disaster using a rubber band and a glass of ice water.
I don't think PowerPoint is the problem in and of itself, but rather its use as a crutch to compensate for poor communication. Of course, even among scientists, few can count themselves at Feynman's level in terms of communication skills. Maybe this is a skill that NASA scientists need to brush up on, perhaps with Pluralsight courses or something? lol
Speaking of chalkboards, next time you have to give a presentation, bring a chalkboard and do your slides in realtime. Something about the visual show, auditory overload, and not least the novelty of the act makes it much more impactful and memorial than "another powerpoint that puts you to sleep"
White boards are... ok... better than powerpoint but still fail to sell it like a chalkboard does. I think it is the noise.
Yes, the noise (which I'd call "the sound") is a big factor.
I teach in a classroom that had blackboard that had stood the test of time for decades. When it was replaced with a whiteboard, things went downhill. The markers dry out quickly, without much notice, so that students often have trouble reading the material. And the whiteboards get harder to erase year after year.
I guess the advantage of whiteboards is that a variety of colours can be used. But some students have deficiencies of colour recognition, so that's not really helpful. (I never used coloured chalk, for the same reason. Maximal contrast is the key.)
And the noise. That click drag click of chalk. Students after the transition to whiteboards told me that they really missed that. It enlivened the lectures. And when students were writing down notes, they knew to look up when they heard the sound.
Back to the point about the "visual show" and doing slides in real time. Yes, yes, yes. Once in a while I need to show something on the projector. The moment I turn it one, I see students start to disengage.
Definitely on the people with color issues. I have some red-green issues, for most purposes it makes me horrible at choosing colors and horrible at choices involving subtle colors (picking the good produce) but rarely interferes otherwise. But a thin line of color is another matter. I generally can't tell the difference between a 1-pixel wide line of FF0000 vs 00FF00. Nor can I tell the difference of a few-pixel-wide indicator. I can't crimp Ethernet because the fine line of the coloring of the wires reduces it to not much better than a guess.
And a whiteboard from across the room is likely to be thin enough lines to give me big problems.
The medium is the message.
PowerPoint gets used because it requires less effort from the audience. They sit back and zone out like couch potatoes. Scrap the PowerPoint and throw the technical report at the managers. Any of them who complain or otherwise don't read it are incompetent and should be fired on the spot.
I think it's a lot harder to have this particular type of communications failure if you're writing on a chalkboard. Imagine trying to write out that whole slide, it would take forever. If you really did have to present that information on a chalkboard, you'd be significantly more likely to write something along the lines of:
"We checked the test data: possible to damage tiles significantly" "Foam that hit wing was way bigger than the tests"
Obviously you can miscommunicate via any medium, but I think the author's point here (which I agree with) is some mediums lend themselves to specific types of miscommunication.
Yeah, the choice to gloss over the point "our tests are not relevant" was a deliberate one. If it was in a paper you'd have big fancy graphs of the tests and you'd have to do your own work to compare the x axis against a mention of the actual scale in question in another paragraph. It's not as if they started with "Warning: even the 600X smaller bits we tested can damage the wing" and microsoft just kind of spontaneously grew a bunch of random stuff above the fold. It's a kind of chickenshit communication which you can do in any medium. The point they ought to be making is not dense or technical, it is so simple a child could understand.
the medium is the message
Peter Norvig's Gettysburg Address powerpoint is a great illustration of how Powerpoint makes things worse.
https://norvig.com/Gettysburg/
This was an interesting article but it doesn't really provide solutions. I watched a few tech talks teaching a new API. Most slides were split, left side bullet poitns, right side either code or an image. As I was watching I was thinking "isn't this supposed to be almost the worst style"? but I was also thinking "I can't think of any way to do this better". It's an API. It requires examples. And it requires something describing what to concentrate on, what the example or image is showing.
I've been the plenty of great talks with just images, no words. But they fit the type of talk. I'm not sure an API talk would be better without bullet points. If you know of some to reference, please post links.
Tufte did make specific recommendations that one should prepare a real document that your audience can and should read, and that they would have in front of them during the meeting. I'm not sure how best to translate that to your API example.
What would make the most important point of that slide stand out any more in a "real document" than in a slideshow? If anything, I would expect it to be buried even more - a slide and limited-time presentation forces you to be concise, while in a document there tend to be few limits on length.
I would say the disaster occurred despite PowerPoint, not because of it. It's not clear to me at all why the slide author thought all that text was needed, when it seems to communicate almost nothing. If anything I would blame it on the culture around "real documents", where having more information is treated as better (probably because they serve multiple functions - to educate, but also as a record of activities), even if it makes it bloated and hard to read.
A document generally has an abstract at the front and/or a conclusion at the back where the important information goes.
A presentation accompanied by a document can be more easily done with punchy slides because the detail is in the document.
Discussed at the time: https://news.ycombinator.com/item?id=19668161
I found it surprising that the slide in the article uses Calibri, a typeface that wasn’t publicly available at the time. The original discussion confirms that the slide in the article is a recreation of the original one:
> The slide in the article has the same text, but is a recreation of the original (The Calibri typeface used wasn't part of PowerPoint until 2007).
> The original slide can be seen in the full report linked in the article:
> https://www.edwardtufte.com
falling nine times faster than a fired bullet
No, that's not how the physics works. The foam is moving at the same velocity as the shuttle when it breaks off, and had a short time to accelerate(decelerate) before hitting the shuttle.
THANK YOU. I've seen the velocity of the space shuttle quoted as the speed that the foam had when it hit Columbia's wing so many times, and it bugs the crap out of me.
If that was the actual impact velocity of the foam, there would not be any doubt about whether the shuttle would survive reentry, that is if it even managed to make it all the way into space.
I think Edward Tufte was involved in the investigation on that disaster.
He has a legendary hatred of PowerPoint.
https://www.edwardtufte.com/product/the-cognitive-style-of-p...
This is mentioned in the article, and linked to.
Yes. You are correct.
I took his course, a number of years ago, and he mentioned it.
The slide is so bad it might be intentional. The intent would be to be reassuring enough that no one will point at Boeing and say "Boeing technical failure strands astronauts in space", while providing just enough information that in the event (at the time of writing the slide considered very unlikely) things go bad, they can say that nothing was withheld.
Yeah, the problem here isn't Powerpoint, it's what it said. This is a case of shoot the messenger. Powerpoint doesn't hinder communication, it lets people think they understand when their actual knowledge is superficial and can easily be a case of the primrose path.
Some asked DoD to use satellites to see the damage. Linda Ham declined the request.
"The imagery request was soon rescinded by NASA Mission Management Team Chair Linda Ham after she investigated the origin of it. She had consulted with Flight Director Phil Engelauf and members of the Mission Management Team, who stated that they did not have a requirement for imagery of Columbia. Ham did not consult with the Debris Assessment Team, and cancelled the imagery request on the basis that it had not been made through official channels." https://en.m.wikipedia.org/wiki/Space_Shuttle_Columbia_disas...
A 2008 episode of the PBS NOVA program covers the Columbia disaster. It does not get into the focus of the article posted here, but it does well covering how poorly the situation was handled (along with other things like the broad history of the Shuttle program).
https://www.youtube.com/watch?v=6t48bc2dyzo
Interesting article. Nice to see Tufte quoted. I took his class about the visual display of information. It was very informative.
Tufte also had a lot to say about the Challenger disaster, which predates Powerpoint but not the visual display of information.
Found the chapter here: https://williamwolff.org/wp-content/uploads/2013/01/tufte-ch...
Take the text and send it to ChatGPT. Then, use this self doubt prompt and you get some alarming results.
https://justin.searls.co/posts/sprinkling-self-doubt-on-chat...
(2019)
Some previous discussions:
2022: https://news.ycombinator.com/item?id=30615114
2019: https://news.ycombinator.com/item?id=19668161
Related:
Death by PowerPoint: the slide that killed seven people (2019) - https://news.ycombinator.com/item?id=30615114 - March 2022 (197 comments)
Death by PowerPoint: The slide that killed seven people - https://news.ycombinator.com/item?id=19668161 - April 2019 (127 comments)
Everyone should note that the slides in this blog post are fabricated, and the author of this post misrepresents Tufte and the conclusions of the review board—subtly pointing people in a direction that's the exact opposite of the real takeaways from the investigation. It's almost insidious.
The first time I came across this blog post a few years ago, I found it enraging, and I still do. As I said I said in my personal notes at the time, "The choice to use this fabricated material for this article and the false claims in it—claims not actually confirmed by the sources that it cites—amounts to what would be considered serious misconduct elsewhere." But there's no real accountability here, of course, because at the end of the day it's just some jerk with a blog serving up distorted pop science insights to aspiring entrepreneurs.
> Why, given that the foam strike had occurred at a force massively out of test conditions had NASA proceeded with re-entry?
What was the alternative?
Columbia could not have made it to ISS.
Columbia could not have repaired the damage in orbit.
Columbia could not have lasted, after two weeks in space, long enough to launch a rescue mission.
I know the "In Flight Options Assessment" said they could launch at an accelerated pace but the assessment assumes that it's ok to launch another vehicle with the same problem, no fix, and no completed analysis of the cause.
Yeah, they suspected the external tank bipod foam, but WHY did the foam come off? Was it a fluke? Had some unknown factor not present in previous external tank bipod foam applications but now present in all external tank bipod foam applications manifested?
>Two major assumptions, apart from the already stated assumption that the damage had to be visible, have to be recognized – the first is that there were no problems during the preparation and rollout of Atlantis, and the second is the question of whether NASA and the government would have deemed it acceptable to launch Atlantis with exposure to the same events that had damaged Columbia. At this point, at least two of the last three flights (STS-112 and STS-107) had bipod ramp foam problems, and the flight in-between these two, STS-113, was a night launch without adequate imaging of the External Tank during ascent.
https://govinfo.library.unt.edu/caib/news/report/pdf/vol2/pa... (page 397)
That's not a valid assumption.
This is from a pre-flight safety report for STS-113
>“More than 100 External Tanks have flown with only 3 documented instances of significant foam loss on a bipod ramp”
STS-1 through STS-111, April 1981 - June 2002: three "significant" bipod foam losses
STS-112, October 2002: significant foam loss
STS-113, November 2002: night time, but they saw 112 and went "oh shit" and wrote a report
STS-107, January 2003: yet another, fatal, significant foam loss
If two of the last three flights had foam problems and the one that didn't only didn't because you couldn't see if it did, and over 100 of the preceding flights only had three, you don't risk four more lives.
You start designing a memorial at Arlington.
The alternative is to try.
Apollo 13 probably seemed pretty hopeless. They didn't just say "sorry fellas, maybe you'll get lucky and make it, good luck."
The late great Bob Hoover said, "If you’re faced with a forced landing, fly the thing as far into the crash as possible." You keep trying until you can't anymore.
Saying "well, maybe they'll survive," and not even telling them, is not the move when you have nearly a month to figure something out.
Nobody has made a decent proposal of how they could have been saved, the only "answers" being basically hail marys.
How realistic was the idea of sending another shuttle up to rescue them? Would they have had enough oxygen?
If they did a spacewalk and found the damage, what were their options?
They did a bunch of studies. While it was POSSIBLE to get a rescue shuttle up to them if they ignored a bunch of safety and refurbishment procedures, the sheer amount of complexities probably meant they would lose 2 shuttles and 2 crew.
The second shuttle would've flown with a crew of two, working with the knowledge that their ship was even more vulnerable than the one they intended to rescue.
> There was actually an exercise done to work this out, at the direction of the Columbia Accident Investigation Board (CAIB). [...] In the CAIB’s scenario, Atlantis would have launched with a four-person crew: two pilots, and two EVA mission specialists. [...]
> A Columbia rescue mission would have been the most monumentally difficult and epic space mission in history, and it would have required absolutely everything going right to bring the crew home safely. But NASA has shown time and again its ability to rise to the occasion and bring its formidable engineering and piloting expertise to bear. Instead, the worst instincts of the agency - to micromanage and engage in wishful thinking instead of clear-eyed analysis - doomed the crew.
https://www.quora.com/If-NASA-had-known-ahead-of-time-Columb...
A much more deeply researched article here:
https://arstechnica.com/science/2016/02/the-audacious-rescue...
I read that ars article every time it comes up, and I daydream of them making a movie about it.
I can’t imagine it will ever happen, it will be such a bad look for NASA
The problem with sending up another shuttle is they hadn't actually fixed the underlying foam problem. If there was another foam strike on the rescue shuttle, now they have two craft in orbit with crew on that can't re-enter
One option would have been to place whatever high melting point metal tools they could spare into the hole, and freeze water around them to hold it in place. It also might have been possible to change the series of s-curves and other maneuvers done during re-entry, in order to lessen the heating on the left wing.
Frozen water would have exploded at reentry
Source? It would be solid water ice, mostly surrounded by shuttle wing structure, in a low-g reentry. Not an "ice & volatiles & etc." meteor in a high-g situation.
the sources are the pressure and temperature conditions at reentry and the water phase diagram
Ablative heat shields work very reliably, even though their material has no substantial liquid phase between solid and vapor.
The function of the ice would be to act as an ersatz ablative heat shield.
Also a famous ex-NASA engineer analyzed a similar situation (water + protein) -
https://what-if.xkcd.com/28/
- and noted no possibility of explosion, except in the case of a hypersonic tumble. A hypersonic tumble would shred the shuttle orbiter anyway, due to the extreme aerodynamic forces.
Note that the people making these slides are one the selling side, so they have a natural incentive to downplay the problem, and the slide smells a bit like that to me. (And by selling side I don't necessarily mean sales people, but also the engineers involved in designing, building, or at least working on the product.)
FTA:
> Think about your message. Don’t let that message be lost
That was the important bit of the article. Regardless the medium, ensuring you deliver the message you want should always be the driving point.
Also: Yes to less slides and more Position Papers (but keep em brief please :-D)
I've been slowly refining a pitch deck over the past couple years and the feedback from reviews and test pitches has strongly reinforced for me just how important it is for slides to be short and laid out precisely.
You want the most important information in the right places, communicated with as few words as possible, using the most accurate words possible.
You want the key takeaways to be the things that people are most likely to remember from each slide.
You want to minimize distractions and try not to pollute slides with a bunch of vaguely related stuff. A crowded slide risks communicating nothing.
It's a real dramatic change compared to how I am used to using powerpoint for technical audiences or when I had to make presentations during school.
One of the dynamics is that if there is too much text they’ll read instead of listening to you. Another is that if the conclusion is on the deck, they’ll get bored waiting for you to get to the punchline they’ve already read, and that can lose your audience as well.
So you’re better off either opening with the conclusion or putting it on the next slide, but accidentally jumping two slides forward can still ruin your audience’s attention span. So sometimes it’s better for it not to be in the deck at all.
"Kings, heroes, and gods use a short and direct form of speech." -- theater maxim
it's also important to know your audience. If they've spent their career reading technical papers or dense prose, that's one thing.
If they've largely grown up in the social media era and click away from reels/shorts that don't have animated captions, you'd design a very different deck.
The main problem is they didn't use reusable rockets, so these couldn't be properly tested before putting a human crew in.
This problem seems mostly solved now.
What would "proper testing" look like, to have caught and corrected the fatal design flaw?
The shuttle was about as reusable as the Falcon 9 is now. Though at vastly higher expense.
All they actually needed to do is pay attention.
Both Columbia and Challenger were quite preventable, but the problems were basically ignored because they clearly hadn't destroyed the orbiter. Never mind that both showed random behavior outside the design spec, sooner or later the orbiter was going to roll a 1. They had seen the blow-by that killed Challenger, they had seen orbiters scoured by the foam before.
However, it didn't really have a solution. The blow-by problem that destroyed Challenger was fixable, the foam problem was not. You could reduce the chance of a problem but nothing could be done about the fundamental problem of having parts of the heat shield aerodynamically behind other parts of the spacecraft, especially cryogenic parts of the spacecraft. (Ice could also do damage.)
The shuttle needed extensive refurbishment between flights. It wasn’t until after the loss of Columbia that they examined heat shield damage in orbit, assuming any damage observed on the ground would have occurred during reentry. For an organisation so risk averse as NASA, I was surprised how little focus was placed on the orbiters as experimental vehicles.
Which parts weren't reusable?
The external fuel tank, whence came the foam that stuck the wing, leading to the loss of Columbia.
There is another point to be made about insisting on the reentry - Columbia didn’t have propellant to reach the ISS, and life support couldn’t be extended until a rescue mission could be mounted. A rescue mission would be further complicated because the docking hardware was not needed for this mission and was removed, and there was no airlock for EVAs (and no EVA suit on board), again, because it wasn’t part of the mission.
If a rescue mission could have been mounted, it’d be something straight out of science fiction - tying ships together with cables and shuttling astronauts in their orange pressure suits through the side doors of two disconnected shuttles.
For once I will defend PowerPoint: 1/ the slide is atrocious, even by 2003 standard 2/ you don't decide on the life or death based on a crappy slide you don't understand.
As an engineer, that slide looks completely reasonable to me. Its purpose was to communicate technical info, which it did adequately. Keep in mind that the subject matter is highly technical, given that we're literally talking about the Space Shuttle, and more than a high school level of reading comprehension is heavily implied. If the NASA personnel weren't competent enough to review technical data without a pithy summary, that's on them.
"That's on them" is not acceptable for an engineer when lives are on the line. Part of your job is making people understand what they need to understand. If their lack of understanding means people die, then you need to do your job and figure out how to communicate effectively to the audience you have, not the audience you want.
There’s a famous speech someone related from their civil engineering professor, where the professor basically said, if I pass you in this class I am effectively giving you a license to kill. So some of you will not be passing.
My CS undergrad was in the engineering college and so I had a mandatory engineering seminar that was basically "don't get people killed with your work." We covered Challenger, Hyatt Regency, and some other classic failures. I've mostly avoided working on life-critical software so it's not an immediate concern, but that sense of responsibility still stuck with me.
Also attended an engineering school. I had to take way too much chem and physics. It was weird.
I can be kind of a pain in the ass when it comes to details so I’ve worked on a couple such projects. It’s sobering, but also I think, “better me than” half a dozen corner cutters at my last two jobs. They could do much worse.
That said, I stayed on a commercial aerospace project about 14 months after I didn’t really want to be there because people kept saying the wrong things in meetings and thinking they sounded right.
The assumption of competency goes both ways. The NASA personnel should have been able to understand a very standard slide in their field, that any college-educated fluent English speaker would have been able to grasp.
People with a high school level education should not have been making life and death decisions about the Space Shuttle.
First of all, is that actually their level of education or are you just making stuff up?
Second, that's irrelevant to my point that the engineer is responsible for communicating, not just figuring stuff out. You cannot say "if you don't get it, that's your problem" when their not getting it means people die.
The slide in the OP is a completely standard way of commumicating information in the aerospace industry. If the NASA personnel had problems understanding this slide, then they also had problems understanding virtually every other piece of technical info that was ever communicated to them by a third party. College level reading comprehension means being able to understand nuance, which this slide conveys.
All you're doing here is convincing me that this wasn't a one-off and the aerospace industry has a pervasive problem with communication.
The average IQ and level of English proficiency is much higher in aerospace than it is building web apps.
Manifestly the high IQ and English proficiency in aerospace does not extend to the ability of making slides that are not completely a mess
> As an engineer, that slide looks completely reasonable to me.
Then you shouldn't be in charge of communicating highly technical subject matter to decision makers, especially if lives are at stake.
[flagged]
Are you saying NASA decision makers graduated from India?
[flagged]
Oh I thought you were talking about NASA and the Columbia disaster. The PowerPoint slide was poorly designed, as it says in the story. The problem was not PowerPoint though. You could summarize it using a PDF, and it will still have the problem if key issues don't grab attention.
The point that seems lost on HN, who evidently are incapable of consuming information that isn't presented in a format designed by Jony Ive himself, is that there's nothing wrong with the format of the slide to the kind of skilled subject matter expert in the aerospace industry that should have been reviewing it on NASA's behalf.
But that's not how it works in most large organizations. The person who has the authority to make the decision is not necessarily an SME in all of the areas they have to make decisions. Rather, they rely on SMEs to do the investigation and then... communicate properly.
For reference, NASA asked the group of aerospace engineers ("rocket scientists") who designed the Space Shuttle to provide them with a technical analysis of the effects of foam impacts, and then subsequently lost the Columbia with all hands because they couldn't exercise more than a high school level of reading comprehension. The engineers presented the facts as they were understood, only stopping short of doing NASA's job for them by deciding to abort reentry.
Why are you posting under three different accounts that you've made in the last 2 hours?
[flagged]
It wasn't communicated properly as it says in the story. It is typical engineer reaction to question the reading comprehension ability of the readers.
If you can't consume the PhD-level analysis that you requested because you only have a high school reading level, despite having been tasked with safely operating what was arguably the most technically complicated piece of technology in existence at the time, then there's really no other excuse for that level of incompetence.
Let's presume you are correct. The Boeing engineers who made this slide were geniuses. The NASA managers consuming it were dullards. If the Boeing engineers were so smart, why didn't they write for their audience?
I am consistently ranked as both one of the best engineers in my company and one of the best communicators. I would never make a slide like that for any audience. It's shit writing for non-engineers and it's shit writing for engineers, too.
You have got to get over whatever this hangup is about "PhD level analysis". That phrase doesn't mean anything.
I'm not arguing that it was a good slide: just extremely average in the context that it was created. If there was a problem with communication, then it was solely on the reader for not being fluent in their own field. The engineers were presenting a nuanced view of their data, which went above the heads of the NASA personnel who were evidently only capable of interpreting a simple "yes or no" answer.
They should have made it abundantly clear that they had no idea what was going to happen and that loss of the crew and shuttle was a real possibility, but I agree with you.
This slide was presented with a verbal talk track, and anyone who can't handle focusing on the topic because the slide is boring shouldn't be in a position of responsibility.
There really should have been one large bold font slide saying “we have no test data for a piece of foam this size”.
There's a near infinite quantity of technical information to choose from. The purpose is to emphasize what is important, and de-emphasize the unimportant. When I look at that slide, I see 95% utterly irrelevant information, and a teeny tiny note saying, in vague and indirect language, that the impact was 600x worse than any test.
So on the remote chance you're not just trolling: If you're doing anything safety critical, please quit your job before you kill someone. You vastly overestimate human's (including yours) ability to process information. I am being 100% sincere.
I can't remember the last time I saw a slide as mangled as the one in the article. It hurts my brain just reading it.
But you are right, most engineers would consider that reasonable, while complaining about the "muggles" that just don't get it.
As a Software Architect, one of my main responsibilities has been to take information presented like above and turn it into something that non-technical people can digest.
Being able to express a complex concept in simple terms is an invaluable skill.
It's really terrible. It's basically:
> Everything is fine.
> Stuff is good.
> There's no problem.
> It's all going great.
> Actually, everyone on board is likely to die.
Someone didn’t learn about anchoring in business 101.
Actually most engineers would complain about the slides shown here. The issue is not the technicality or depth of the content but on the way it is presented and shown. I'm
it depends, I noticed that many engineers will input information on a slide following their thought process closely, they rarely think about the audience's perspective, especially if the audience is less technical or not familiar with the domain.
We're talking about something a lot more technically sophisticated than a B2B SAAS CRUD web app. PhD level education is considered a prerequisite.
Sure mate, because taking a messy list of confusing statements and turning it into something like: "High risk of failure on re-entry - foam strike more than 600 times bigger than test data - test data unfit to support decision as flight condition is significantly outside of test database" requires a PhD in Materials Science.
If you can't effectively communicate how the results (or lack of results) of your research will impact the outcome of a high-stakes space mission you have no business being in that room from the start.
No PhD I know (several hundreds, in physics) would ever consider this slide remotely acceptable
Isn't this a stronger argument for making sure that things are communicated clearly?
PhD in what though?
If you want a slide to really hurt your brain search for “Iraq war PowerPoint slide”
The principles of that slide apply to a lot of other circumstances.
This?
https://armsandinfluence.typepad.com/photos/arms_and_influen...
I misremembered it’s the Afghanistan slide https://www.theguardian.com/news/datablog/2010/apr/29/mcchry...
If it weren't for the millions of lives ruined, that would be hilarious.
Has anyone checked in with Daesh about their Q3 OKRs?
The slide shown here is completely horrendous, even by 2003 standard. I've worked in a field as technical as this one for a long time, and this slide would not pass a review, even with people familiar with the content.
While it does not wash the responsibility of the executives, the engineers have also the responsibility to be clear in their communication
I wish this was satire. The slide is full of spelling mistakes and all kinds of horrible communication. Even when I know it documents an issue that killed people I still struggle to read it. Only the bottom part hints at anything relevant.
> which it did adequately
What makes you think this, given the subsequent events?
That's a completely unremarkable slide in the aerospace industry. If there was a communication problem, it was with the NASA personnel not being able to operate in their own field.
Putting the fact that you don't actually have any remotely suitable test data buried at the bottom is reasonable??
I spent too much if my early career dealing with the consequences of bad decisions made on “good data”. If the presentation was so good why did they come to the wrong conclusions? It became part of my overall thesis that good design invites you to use a thing the proper way even if you failed to read the instructions.
You can’t force someone to think but you can force a lot of people not to, or you can make it difficult to avoid. It’s worth investing the energy into stacking the deck the right way.
The NASA personnel gambled and it costed 7 lives. Somehow, a powerpoint slide was what caused it? Lol.
[dead]
TLDR; Bill Gates has blood on his hands.
Another example of "guy can't spell" => "he's retarded". If you can't get everything right, you probably got lots of things wrong.
It also shows why software engineers are superior engineers. The history of software engineering has some single-digit kills, at most 20 in total. Meanwhile, shuttle engineers are supposed to be the best in the aerospace business and they've lost some 13 or so? All aerospace would be put in the thousands.
An old saying is "Any fool can build a bridge. It takes an engineer to build a bridge that barely stands". But these are fools who built bridges that didn't stand.
One day, I will teach them real engineering: It is Rails backend with React frontend. Zero kills. Life above all.
BTW: Here's is the original content with Tufte review https://www.edwardtufte.com/wp-content/uploads/bboard/images...
One could argue otherwise such as the Boeing 787 MAX MCAS Software Design.
That's a good counterexample. We can count that in software engineering. Can't say I expected Boeing software engineers to be any good. Dregs of the field but they still count as in the field. Aerospace software engineering quality substantially lacking, it's true.
> One day, I will teach them real engineering: It is Rails backend with React frontend. Zero kills. Life above all.
Buddy, if you think that is "real engineering", you've got a lot of things to learn.
Even in software engineering, rails and react aren't really engineering. Real engineering includes calculations -- data structures, algorithms, data structures, and user interfaces designed to work with people who are deaf or blind. You wanna tell me that rails is performant? Have you done calculations? You wanna tell me react's algorithms are well-defined in memory and time? Have you ensured that your "software" is usable by people with disabilities?