> The outage appears to be affecting a telecommunications network overseen by L3Harris Technologies, according to people briefed on the matter. The network handles crucial radio frequencies that air-traffic controllers use to communicate with pilots, the people said. It wasn’t immediately clear whether the outage also affected the radar feeds that controllers use to keep aircraft safely separated in the skies.
> An update to the private communications network that connects America's air traffic control system caused the system to reboot, leading to a crash and a communications outage, multiple sources told CBS News. The system is operated by L3Harris, which did not immediately respond to CBS News' request for comment.
Anybody familiar with these systems know why they would do an update like this at 3pm on a Friday? This would be extremely unusual in any software job I have ever had. For commercial aviation specifically which does have pretty well defined peaks and valleys in terms of traffic over the course of a day this seems like an odd time to choose
I did web dev work for an airline and we couldn't even release new updates to the loyalty benefits website during a weekday (even though there was no connection to operations or flights). The fact that this company would reboot a critical comms server at 3 pm on a Friday is indeed bizarre.
I don't think the hesitation to push to prod Friday afternoons has anything to do with the working schedule of your customers. It's more about whether you'll be able to roll back on a weekend.
Uh I'd imagine any team that mainly has their user base using the product on weekdays would deploy Friday evening so if something goes wrong they have the weekend to fix it.
It depends on the schedules of the people who'll have to fix it if something goes wrong. I don't mind working weekends, so when I was in the sysadmin game I always tried to make significant changes late friday/early saturday, so if something went wrong I had the weekend to fix it & clean up. But if the people needed to fix a problem take weekends off, pretty much the opposite schedule applies.
It should be fine to update at any moment. Why? Because systems like this ought to be hot spare or blue/green models that have a full working copy at any given time, and a failover/failback.
A reasonable deployment plan considers the reality that the deployment could fail. One mitigation is to choose ideal conditions for a possible failure. You’re correctly pointing out one of the mitigations, but it doesn’t render the others unnecessary.
As opposed to what alternative? Being run by the FAA itself or with contractors? Being run by the DOT?
Well, it worked so far, I think it is great. 100% availability is a dream.
That’s why for example, you have ATC Zero protocols when communications go out all of a sudden at airports. And I imagine there is something similar for airborne planes that are not near any of their destinations.
I mean they also make the CPU for the F-35, alongside kinetic strike and electronic warfare missiles, so they could cause plenty of damage if they wanted to. This isn't some mom-and-pop radio company.
Sure, but if it fails and a fallback takes over, will you be going home and having a nice weekend? Or will you be stuck at work over the weekend to fix it?
But 3am is when a big chunk of planes are parked at gates awaiting the start of the flight day (yes, there are red-eye flights and international but they are a minority.)
Overnight periods are when the air travel system largely resets itself from each day's cascading delays, mechanical failures and bad weather.
Arbitrary day-of-the-week and date-based deadlines piss me off so much.
I worked for a person who had a fixation on "neat" deadines: the "first of the month", "the end of the week", "the start of the quarter", etc. It'd be one thing if the dates were tied to legal compliance (tax filings, etc), but that usually wasn't the case. They were chosen because they were "tidy".
I'd time meetings looking to maximize time for work. You try to meet late in the day Wednesday because an early Tuesday meeting might get an "end of the week" deadline. A late Wednesday meeting was more likely to garner a "end of next week" deadline. Meeting Thursday or Friday guaranteed an "end of next week" but you lost Thursday and Friday work time. Wednesday was a gamble, though, because it was still early enough to get you an "end of the week". Meeting after lunch helped with that.
You always had to be aware of the numeric calendar day, though. Gunning for an "end of next week" deadline when next Wednesday was the end of the month (or worse, end of the quarter) almost guaranteed you an earlier deadline than you wanted.
All of this deadline-setting happened with little regard for the amount of time work was going to take.
It was immensely fun, when setting a deadline in this person's absence, to get a group to agree on a mid-week deadline. You could see it make this person's skin crawl hearing that the work would be done on Tuesday or a Thursday.
> Rolling out updates on a Friday afternoon is still practiced in various industries.
By people with unreliable systems and a lot of myhts behind.
In a well engineered organization, you can deploy whenever you want. Sure, it will not work 100% of the time, but you figure out why, fix the problem and you will be one step closer to the mythical 100%. And so on. Deploying less frequently is what creates more problems.
There is also a real possibility they did this many times before and it worked. I don’t really get it: if the thing would have broken on a Monday night and it would take a day to fix would that have been better?
It's worse than that. It's 3pm on a Friday before Thanksgiving week.
I really hope we get an official or leaked root cause analysis and postmortem of this and it doesn't get buried because all the money in my pockets is willing to bet there was some deep incompetence responsible at the executive level.
But the entire first months of the Iran War was started on Friday after the markets closed. It took a few weeks for most Americans to catch on.
My money is on some mundane ops issue like "it takes lots of time to get sign off for the upgrade from lots of different stakeholders and the necessary upgrade was falling days or weeks behind, leading to a key IT person to decide to YOLO the upgrade/deployment".
I've heard that Sec Transport Duffy has been pushing FAA upgrades faster than is safe, so maybe the employee rushed it so this upgrade wouldn't hold back some other larger timeline. There have been several articles recently about how the FAA is full of people who are "too risk averse", suggesting that there might be an effort to push riskier projects faster. Having a nationwide outage is one of the likely symptoms of raising risk tolerance.
> It took a few weeks for most Americans to catch on
In what sense? It was well reported at the time, so presumably most people know about it. I'd be interested in what makes you think the time and day of week chosen would delay market or public reaction be _weeks_.
Huh. I thought Collins now ran what used to be Aeronautical Radio, Inc. (ARINC), which is the ground portion of air traffic control radio. L3Harris has the radars and ADS-B.[1]
Collins/ARINC is used for oceanic HF radio. They run "San Francisco Radio" and "New York Radio" specifically, which are the HF radio operators that relay oceanic messages to ATC.
(They also own a small number of private VHF stations for relaying company messages to air carrier operations domestically, but these aren't used for ATC purposes.)
Otherwise, the VHF radio links are operated directly by FAA TechOps.
L3Harris owns the telecommunications backhaul between facilities.
Doesn't L3 Harris use Microsoft Azure Government for most of their mission critical infrastructure? Are we looking at fallout from the Microsoft infrastructure collapse yesterday?
Cue that line from DS9 where O’Brien says he’d never be caught without a secondary backup during a fight, and the cardassian is like “you have a second backup?” incredulously. https://youtu.be/o7o9ZtipVyo?is=y3for_WXwISmuqpd
I wouldn’t say anything but this is approximately the only time in my entire life this scene will be relevant.
The Internet was purpose designed to tolerate disruption as major as a nuclear war, which it achieves through decentralisation.
Private networks exist to make idiot managers feel more confident while actually dramatically reducing availability by putting all their eggs into a single basket.
Isn't using a private network like mandating using I-95 (since it might be blocked from a freak accident one day), while using the internet, with redundant paths between hosts, like allowing the traffic to find the best path?
One network can be a fully redundant design with multiple geographically distinct links and so on.
But it does not make it 100% resilient to cascading failures or possible Layer2/3 loops that might happen during a software upgrade, especially in a country wide deployment.
The FAA will give a root cause at some point of why this happenned.
Yeah - like, why do they not have an "HA" cluster, where they patch and reboot a subset at a time... Something this critical should have some sort of failover/load-balancing in-play...
There's no evidence that AI chips were sold to China. The chips were sold to Singapore. I have yet to see confirmation the chips were subsequently smuggled out of Singapore with Jensen's knowledge.
The Nvidia chips that the Russian drones are using are consumer class chips (TE980M-A1) that have no export controls. It's odd that this channel decided to highlight that when it's very clear those are not on export control lists only the high level AI chips are restricted and controlled in this way. Those chips were sold to any country that has open trade with Russia then they could easily make their way to Russia from second-hand markets.
So from that alone I don't trust the integrity of this source's reporting.
You'll find Intel, AMD, and every other name brand of CPU/GPU in all countries on earth, that doesn't make those companies culpable.
Fair enough. That part was pretty short, the rest of the video making connections between the money and the companies was fairly convincing to me that something doesn't smell right.
>You mean he facilitated sale of restricted AI chips to China?
No, they mean "had a sexual relationship with a subordinate".
Which is what the linked article says.
>Sorry, but I am tired of the effort to make it look like Jansen's crime was infidelity.
Strawman. Nobody says that, and nobody cares.
Jensen's "crimes" that we know of are:
1. Having a sexual relationship with a person suspected of being a Chinese spy (who, allegedly, influenced NVIDIA's chip sales to China)
2. Having an undisclosed sexual relationship with a direct report
Both are huge liabilities for the company, as well as national security concerns (someone in a critical position putting themselves into a position for easy blackmail).
None are crimes, all are factors contributing to the firing decision.
>If he just went to whore house he would be fine (well, ignoring a possible expensive divorce).
Could you be more specific about the comments you’re referencing? I don’t see anything in this thread, at least at a cursory glance, that could reasonably be described this way.
Did you somehow see defense of infidelity in my comment (hope not, because it wasn't there), or are you reflecting on the general trend?
Who do you feel is creepy here (or a man, for that matter)?
Infidelity (in itself) is not the thing that got either of those 2 men fired.
That statement doesn't say anything about how anyone here feels about infidelity, nor whether it should or shouldn't be enough of a reason to fire a CEO.
I don't see anyone supporting infidelity here.
However, in a thread about Jensen, I've seen multiple people argue with me that the CEO having sex with subordinates isn't problematic at all (with total disregard for the power imbalance and potential for coercion).
>An update to the private communications network that connects America's air traffic control system caused the system to reboot, leading to a crash and a communications outage
And that's why you never deploy to prod on a Friday.
I think it also creates a tension because people are more compelled to "just ship it" to go home. If it's a Monday and it's not quite ready yet or someone has doubts, more likely to just hold off.
I feel like people want to go home on Friday and will make sacrifices they would not make on a Wednesday.
People who have a testing window Saturday morning, which is how a lot of legacy industries with cyclical schedules work. Weekends are slower air travel days.
A junior engineer according to my table:
intern: break the build and go home.
junior engineer: break the build and go home for the weekend.
staff engineer: break the build and go on vacation.
senior engineer: break the build and retire.
Air traffic slows, yes, but it doesn't stop. ATC operates 24/7/365. Whether you have one plane or one thousand, the entire system needs to be operable.
Info on this is amazingly sparse. Apparently Sec. Duffy posted something on X/Twitter, and that's all the news media have.
Looking at LAX departures, it looks like nothing has departed since about 10:30 AM PST, and everything later is showing Delayed or Cancelled. There are times after Delayed, but it's not clear those times mean anything. Some are now in the past.
"flysfo.com" is too overloaded to display departures. Stuck at "loading...".
JFK departure status shows almost solid "Delayed".
Makes sense. CNN reported that international airlines were still departing for North Atlantic flights, apparently because they use a separate communications system.
Yeah, oceanic flights largely use satellite-based CPDLC today, or HF voice as a fallback, but not VHF in any case (which I suspect is the implicated system here).
VHF Omnirange (VOR)[1] is unlikely to be involved. VOR stations are not dependent on a network. All they need is power. It's 1950s technology. The big ring of antennas next to US 101 just north of SJC is a VOR.
If you're flip your claim via edit it's usually polite to do it via addition rather than revision so the other replies still make sense/don't get downvoted by confused people.
Sec of Homeland Security gave an interview about 10 minutes ago saying it's indefinite, they don't know the cause, and to expect resolution to take a while.
SFO had delays this morning because of winds. I'm not sure if LA also had bad weather, but SFO had a lot of rain and if you know SFO, you'll encounter a delay if someone sneezes in the direction of the runways.
I did see another statement where Mullin said this doesn't look like a malicious act, which just makes it even more weird that he's sort of taking point. Maybe the Real World actor leading transportation is in over his head?
I have no idea whether this particular failure was Microsoft's fault, but I was in the air on September 14, 2004 when a substantial portion of the southwestern US's air traffic control went down:
(I was near ORD at the time, flying toward the outage area. We diverted to ORD, and the pilot was very friendly about it.)
At the time I thought it was Microsoft's fault, as Windows 95 and 98 were susceptible to crashing when GetTickCount() overflowed, and this ATC crash was due to a 32-bit millisecond counter on a Windows machine overflowing, but in this particular instance the system was apparently not running on a version of Windows with this particular bug and the failure was just an application error.
The Garmin systems I flew were only certified for ops south of 70°N and north of 70°S. Nobody knew what would happen closer to the poles. Maybe the system was just not tested to certification standards out there, or maybe it would do something really stupid and want to fly upside down or backwards! Either way, I wasn't going to be the pilot finding out on a dark night.
It’s likely to do with satellite geometry at extreme latitudes leading to lower accuracy. Maybe also the weaker access to correction data from geostationary satellites.
"Realistically you want fewer, higher-quality engineers who are educated in these tools and treat them with respect and caution. These tools are dangerous and you can’t play around. All it takes is one mistake and the blast radius is way, way larger. One mistake can do so much damage it undos all value of an entire year. Before downloading a malicious dependency was limited to deterministic behavior and could be scanned for. Now, you run a MCP server locally, it can prompt inject (in many different ways), and now your super powerful AI buddy is hacking you in ways a deterministic piece of malware could never. You could cripple entire companies through negligent use."
And
"And now we are controlling models that can be hacked in so many different ways and literally blow up companies. It used to be accountability the software wouldn’t break. Now it’s accountability that in creating the software we don’t ruin the business."
Not saying this was caused by malicious actors or negligent AI agent usage nor am I saying was the previous Microsoft outage either the above references but it doesn't look great. If our technology is really so brittle, we might be looking at a return to way slower world to put it simply. Maybe no remote work because it gets hacked too easy!
> A computer is a device that enables a person to make more mistakes faster than any previous invention, with the possible exception of the combination of handguns and tequila.
I'm watching it live on ADSBexhange, here's a flight I picked at random going approximately past me that'll land in Detroit about an hour from now... there's not much left over the US over 40,000 feet. Only 2 planes over Indiana at that altitude right now.
It's eerie seeing the total number of aircraft tracked decrease. I put on a filter for 1000' to 80000', to exclude aircraft on the ground, and the number was around 12K when I started watching. It's down to just over 9.4K now. Wild.
Edit: 22 minutes later and the count is down to 8.4K.
If you haven’t seen it, there’s an exhibit about 9/11 at the Smithsonian Air and Space Museum in DC that shows the airspace in the lead up to and just after the attacks. If you find the traffic changes now interesting, I think you’ll find that interesting as well. It’s possible it’s online if you do some googling
The house where I lived on 9/11 is in the approach for a runway at a local airport. Air traffic was a fixture at that house. Somebody was landing at all hours most days.
It was the strangest thing to have no traffic at all back in 2001. It's odd for the absence of a thing to be jarring but it was.
I'm also near Detroit and am watching the trackers and I'm pretty sure I just saw N831AK take off from Troy Oakland in violation of the grounding order. I wonder how common that is.
This tracks with what I has seem elsewhere. I don't have the details handy now, but the communication system used for long range handoffs for flights above 18,000 ft seems to be what is impacted. Keeping planes below that deck would keep them bouncing more frequently between shorter range towers but at least still tracked and in communication.
Oh its absolutely bad fuel efficiency. I'd expect they go to that altitude where communication is still reliable and do everything they can to get flights on the ground safely regardless of inconvenience or fuel inefficiency.
ATCSCC Advisory
ATCSCC 111 DCC 11/13/2026 NATIONAL GROUND STOP
EVENT TIME: 13/2053 - 14/0400
DUE TO EXTRAORDINARY CIRCUMSTANCES AND FOR REASONS OF SAFETY.
ATTENTION ALL AIRCRAFT OPERATORS, BY ORDER OF THE FEDERAL AVIATION
COMMAND CENTER ALL AIRPORTS ARE GROUND STOPPED. ALL DEPARTING
TRAFFIC SHOULD REMAIN GROUND STOPPED UNTIL FURTHER NOTICE. INBOUND
INTERNATIONAL FLIGHTS WILL BE MANAGED ON A CASE BY CASE BASIS.
It's a format called "Date-Time Group" (DTG). Used in military, I think it dates back to teletypes, and is better to say over radio than modern ISO formats.
You'd say "Valid from the thirteenth at two zero five three zulu" over radio.
The TLDR is that punchcard operators found the DTG date format convenient. Punchcard sorting is easiest when consecutive bits encode the sort key, and the DTG format is easy to split into different sets of bit ranges corresponding to common business logic.
Airplane software is some of the oldest on the planet, pre-dating anything that would be considered a best practice. When you see something weird in the space, assume it is because someone in the 1960s thought using more than two characters world be a profligate waste of space.
Unrolling these decisions is a monumental task given the decades of hardware already deployed in the field that will only be upgraded when the current unit fails.
> When you see something weird in the space, assume it is because someone in the 1960s thought using more than two characters world be a profligate waste of space.
And with 1960s technology, this might just have been a very reasonable decision.
Oh nice, very helpful. The sibling thread mentions that this problem is specific to "ARTCCs", which are not the towers at airports but rather central hubs:
Between the sectors administered by [smaller towers] are 20 contiguous areas of US airspace above 18,000 feet, each managed by an Air Route Traffic Control Center (ARTCC)...
A flight is handed off from one Center to another until it descends near its destination, when control is transferred to the TRACON serving the destination, and ultimately to the tower controller serving the airport.
Wikipedia mentions they in particular use special "very high frequency aviation bands" (around "AM 118-137 MHz"), which is presumably why they're all linked together. It also mentions sharing systems, but no clear specifics that I can find. Certainly doesn't disprove the botched rollout theory, tho...
EDIT: someone below posted this, seems relevant -- the 'Radar multiplexer pipeline' that's on a "fast track" and an "aggressive track": https://www.sunhillo.com/wp-content/uploads/2025/04/SUN3125-... TL;DR: upgrading radio systems from old analog tech to new digital tech, I think?
That’s obviously not what’s happening. But if this upgrade is really implicated, the real question would be where the line of accepted residual risk for an outage like this was drawn.
Well sure, but I wouldn't be shocked at all if someone greenlit a plan that wasn't supposed to involve closing the airspace, but which failed and which had an insufficiently thought out rollback plan.
Same here, I get it only on some refreshes, otherwise it shows "no advisories". I hope that's not the main path these things are disseminated normally.
(Excuse my lack of composure. A bit stressed out.)
The press conference was started 10+ min late and the first 15 min was spent on blaming Congress instead of giving an update. Kinda frustrating for those who are affected and waiting to find out what they should be doing at the airport. The Q&A has been just an endless repeat of the same few lines of blame shifting. It’d be nice if they could actually give actionable updates.
More info on AP [1]. Looks like a comms issue. The figures they quoted for the overhaul are absolutely insane - almost $45 billion (I used to work in telecom):
> "The government has been working to overhaul the outdated technology that air traffic controllers rely on since Congress approved $12.5 billion for the project last year."
> "Duffy has said that another $30 billion will be needed to complete the work, and the White House has begun lobbying Congress to approve it."
I’ve seen figures that estimate the aviation industry to provide about $1.5 trillion worth of GDP for America, so 4-5 billion dollars. Feel free to double that for the cost of dealing with unexpected disruptions and loss of institutional trust.
It probably is at least partially made up. But I don't know about you, but my flights for the last year or two have all been booked up for every seat. And I'm not sure carriers just have a bunch of planes sitting around not making money. Or a bunch of crew sitting around not making money.
It's because air traffic control systems in the US haven't been significantly upgraded in like a century. Keep kicking the can it will get more and more expensive
Wouldn't that have been taken into account when the $12.5 billion was allocated? Or was the $12.5 billion just meant to be the first allocation and they knew substantially more would be need later?
I'm hearing some discourse about "outdated infrastructure" and increasing investment because most systems are analog and not digital.
However, this failure mode seems more likely to be caused by digital communication systems and an 'upgrade' (with commercial incentives) to newer technology.
My bet is that if short-wave radio transmission keeps being used, availability will be higher than stuff like Github.
Snark aside, this is actual advice in Crisis Engineering by Nitze, Weaver, and Dickerson, and largely matches my experiences as well. You can harp on and on and on at upper management about why budget or attention needs to be allocated to take care of something before it blows up into a much bigger problem, but they'll never move on it -- until it actually does blow up, and then suddenly you've got nearly a blank check to do whatever's necessary to make sure it never happens again.
If you don't take full advantage of that, you're missing out on possibly your only opportunity ever to fix whatever's needed fixing for years.
If things blow up because you're upgrading to something supposedly better and it goes wrong, then that's not a time when you can properly fix the new thing. You can only push forward with the transition or go back to the old. I disagree with "you've got nearly a blank check to do whatever's necessary to make sure it never happens again." You have a blank check, but anything you use it for has to be scoped within stopping the bleeding. As soon as you're onto the new or back to the old, the check expires and you likely can't touch anything real anymore because it'll be involved in lawsuits and finger-pointing.
I don't disagree with the core message, but I think it plays out differently for a major upgrade vs a production meltdown on an existing system. With mostly an existing system, you can pour effort into resilience. With an upgrade, I doubt people would let you beef up a system that is still on the chopping block, even if it's getting a life extension. And if you're still switching to the new system, all effort is going to go into getting it up and running and anything that gets in the way of that (as quality improvements are likely to) is going to face some serious resistance. The check is not truly blank.
Funnily enough this also works at a personal level, as in, within yourself.
You can spend years working on a problem but if you have a catastrophic failure in your personal life, you can use that to change dramatically with far less internal resistance.
When the system is held together with chewing gum and 1980s-era software, when it fails, "someone screws up" is not a fair reaction. Of course it fails! The wonder is that it still mostly works.
A lot of very hardworking people getting too little pay for something so critical while their employers (some contracting company) take a lot of profit. Eventually they burn out waiting for help and a replacement system and retire or quit, then the systems really start failing because the knowledge leaves with them.
> Transport Secretary Sean Duffy says the FAA "issued a nationwide ground stop for all US airports following a communication issue impacting all Air Route Traffic Control Centers.
Hate that for all involved. Was flying out of ATL when the Crowdstrike clusterfuck imploded Delta's antiquated crew scheduling backends. Not a fun night.
Godspeed for everyone getting to where they need to be, somehow, or at least ending tonight with a roof over your heads.
We were offered $1500 to get on a later flight and I’m so glad I didn’t get to the counter on time to take them up on it. Saw the news reports a half hour before landing.
The flight crew on my LHR-SEA flight "still expects to land in the US." The lack of conviction in that expectation was audible. I don't get the impression that they're being told very much, but maybe that's literally a consequence of the "communication issues."
edit: we're on approach into Seattle, so apparently we were one of the happy cases of the "case-by-case basis."
Secretary of Homeland Security just assured a press conference that all planes in the air will be dealt with on a case by case basis. Some may be prioritised.
I'm seeing some diversions on Flightradar24. E.g. DAL2195 from Tampa to LGA was diverted to Harrisburg. But some flights are continuing to their original destinations.
Yep those are the last two. In 2023 it was because of a contractor deleted some file and caused NOTAM to go down, and 2001… well we all know what happened
There were three NORAD exercises (Sky Shield I, II, and III) in 1960, 1961, and 1962 where all civil aviation across the US and Canada were grounded for several hours (the 1961 one was 12 hours, 1960 was 6 hours and 1962 was 5.5 hours). This was with several months of advance notice. These were exercises to test how effective NORAD would be at engaging Soviet bombers. While they publicly announced that they were 99% effective, in actual fact no more than a quarter of the bombers were even engaged before they reached their target. This evaluation was not declassified until the 1990s.
After 1962 there was too much air travel to continue to shut it down for exercises, and NORAD switched to smaller, regional level things that did not shut down air travel after that. But the results of Sky Shield III in 1962 were the only experience that the FAA had to work from when they did the full ground stop after September 11th. Sky Shield III had tested how quickly air traffic control could clear the sky in the event of a nuclear war- the USAF had put up several hundred T-33 trainers, flying them randomly around the US and Canada, and then at a certain point in the exercise the ATC had to get them all on the ground far away from major metropolitan centers (presumed lost in the war).
I wonder if this will close the Starship launch window starting at 5:45pm EST. The FAA issued the commercial launch license earlier today. That launch is under a separate set of "commercial space rules", but presumably they need to coordinate with air traffic control.
I mean they should already be in a situation where no planes are scheduled to fly there, and now there will be even less planes. So seems like an ok situation for them.
I guess if some plane needs to do an emergency detour threw threw that space, but that seems unlikely.
Botched rollout of AI for ATC announced yesterday as rolling out nationwide?
> The agency wrote on Nov. 10 that after the D.C. test, it will “now use SMART in the airspace of all 22 centers across the country to reduce delays, reduce fuel consumption and improve the travel experience for the flying public.” The tech was already being deployed at some airports.
SMART developed by ASI and the FAA is pushing to own a portion of the company:
"The FAA in June awarded Boston-based startup Air Space Intelligence a contract — worth up to $875.9 million — for an AI-powered tool to help air traffic controllers manage the nation’s skies and a related system for an FAA nerve center in northern Virginia to better handle plane congestion. The AI platform’s initial launch in the Washington area has sparked concern among Democrats in Congress who say it should be put on pause until they have a better understanding of how it’s being deployed."
"Over two dozen lawmakers have called on the FAA to pause its 90-day evaluation period for the platform, which began late last month in the Washington region. They say there’s been an “egregious lack of transparency.”"
NYT: "In a social media post on Friday, the company [ASI] said, 'The current ground stop is not based on or a result of the SMART system, any product of ASI or our integration with FAA and ATC.'"
What do you mean? The Trump administration is the most socialist since the the 30s, seriously.The government now owns dozens of large companies and is giving individual citizens shares (with those "Trump accounts" for kids etc.)
Oh I don't disagree with you here, I just hadn't realized the government was actively taking ownership stake in companies (beyond the broadly reported Intel move).
I've felt for most of my adult life that both parties have similar goals with decidedly different implementations. In this case, both want socialist combination of state and industry, even if they disagree with how to reach that goal.
There is a name for what happens when governments and corporations merge into a functionally singular entity under authoritarian leadership. And it’s not socialism.
It would be super awesome if I could hop on an airplane without getting my insides x-rayed, my belongings sorted through or damaged, worrying about the windows falling off or the plane's systems flying itself into a crash, or some tech bros' bid to be a billionaire telling what an understaffed ATC controller what to do.
AI tech is nowhere near reliable enough to be introduced in such important systems. I find this deeply troubling even if it’s not related to the current issue.
I think it depends on what it’s doing and if backup systems exist. Scheduling and prioritizing certain flights for landing seems like a reasonable application as an example. Anything realtime dependent or near realtime it shouldn’t touch. But there seems like lots of less critical applications it could provide some value.
Did you see breaking bad? The guy schedules two planes to collide. Humans can also make mistakes, as long as you have a good harness underneath that can capture the mistakes it should be fine to just send it.
Personally I fail to understand how we could possibly benefit from a non-deterministic AI system, when normal computer systems are more than capable of managing a 3-dimensional airspace using nothing but basic math. Smells like pork for sure :/
“Mr. Mullin said that new technology upgrades overloaded the existing, antiquated system, “overwhelming” the existing equipment.
“It’s going to continue to fail,” Mr. Duffy added. He also outlined many of the additional investments he believes the air travel system needs, like new air traffic control towers, infrastructure and communication networks.”
Of course they are all going to insist it has nothing to do with SMART, that would be incredibly damming, and threatening to ASI. But it’s pretty unfortunate timing no? What other “technology upgrades” have been recently applied to air traffic control/ communications?
Except some airports haven't enough free gates for disembarking as planes that wanted to take off were denied. So they tried to go back to their gates. As a result some planes in the air are being diverted to less-busy airports.
"You're absolutely right! I misunderstood the intended use of the word taxi. I made an assumption, and then ran with it. Would you like me to open a PR to start flights again?"
However, the National Air Traffic Controllers Union (NATCA) said it has not been “involved in the design, testing, or implementation of SMART and therefore cannot speak to its effect on the air traffic control workforce.”
“NATCA believes any new technology should complement and not replace the experience, training, and professional judgment of air traffic controllers who are responsible for the safety of the National Airspace System,” the union said in a statement.
That seems like pretty important context actually, and is a surprise to me.
Seems like there are three plausible causes right now given the extremely limited communication from the govt.
1. The AI system rolled out yesterday had more access than publicly claimed and managed to break things.
2. The ongoing large scale equipment upgrade went wrong.
3. General incompetence from a department run by an unqualified realty show actor, within an administration that is actively hostile to expertise and experience in the federal workforce.
Option 4 would be internal discovery / knowledge of a new vulnerability with a perceived need for immediate patching. We're on week 3 or 4 of every IT department across the country having to patch pretty significant newly announced vulnerabilities across many systems. It's not unreasonable to think that there are vulnerabilities that the government has advance knowledge about that they're quietly patching in their systems before the vulnerabilities go public.
"SMART is a cloud-based system that centralizes 200 data streams – including weather patterns, flight paths, traffic flow, and controller staffing metrics – into one platform."
I don't see why they would care about the NATCA's evaluation of the system. As a union, they have a vested interest in protecting the employment of traffic flight controller workers more than public safety.
Even assuming they don’t care about public safety at all, their members are still going to have to work with it, and happen to be among the world’s leading experts in air traffic control, weirdly enough.
To me, it’s like asking the hand weavers worker union about the benefits of incorporating mechanical weavers in their workflow. They are the experts of weaving, but still…
Right, they do the ROI calculation and then ignore it, and do whatever the favored lobbyists/campaign contributors said to do in any event. Government spending in a nutshell.
The Trump twist is being brazen enough that those who get these contracts can just be his family members or buddies, but otherwise, pretty standard.
If your actual point is that it is costing too much and taking too long, you would be interested in Proposition 45, which limits the ability of local governments and individuals to endlessly delay the project. The LAO's analysis of that proposition is in https://lao.ca.gov/BallotAnalysis/Proposition?number=45&year...
That system just makes suggestions and isn't actually integrated in any ATC system. This is a telecom issue.
Its more than likely that system is just created by some Trump donor and was purchased by the FAA to launder public funds to some company that Trump owes a favor too. Its not actually integrated with anything.
SMART is a cloud-based system that centralizes 200 data streams – including weather patterns, flight paths, traffic flow, and controller staffing metrics – into one platform.
> That system just makes suggestions and isn't actually integrated in any ATC system. This is a telecom issue.
Is there any evidence for this beyond a single statement from the govt? This administration hasn't exactly developed a reputation for fully accurate and truthful reporting.
> An update to the private communications network that connects America's air traffic control system caused the system to reboot, leading to a crash and a communications outage, multiple sources told CBS News. The system is operated by L3Harris, which did not immediately respond to CBS News' request for comment.
They use third party telecom providers to relay communications between ATC “centers” and remote radio transmitters. It is not purely AM radio and outages have occurred in the past, most recently in September.
"modern" -> "we stuck digital computers into a purely analog pipeline to make sure it's capable of a wide range of harrowing, incomprehensible failures."
I don’t think that it’s the wrong idea in general. AM radio (and humans sitting on either end) is going to struggle with the sheer amount of air traffic we have today.
It's weird just how antiquated air radio is, compared to on boats.
Digital Selective Calling is a 1980s technology. So it's not even XML stitched together with Java, it's way older than that. But it does mean two dozen boats and three shore stations can all communicate with each other without constantly stepping on each other or having to wait and hope you can talk to the person you want to before bad things happen.
There are fancier more modern systems in place for a lot of transit, but DSC is relevant here because it's an international system, a random cargo ship from Korea which turns up in the English Channel has the same DSC system as the boat which just arrived from Rotterdam. This is the same requirement we need in the air, a Singapore airliner and an Avianca airliner coming into San Francisco both need the same radio system.
> "modern" -> "we stuck digital computers into a purely analog pipeline to make sure it's capable of a wide range of harrowing, incomprehensible failures."
"we stuck digital computers into a purely analog pipeline, now with software containing a near-infinite amount of lines of code generated by LLMs, to make sure it's capable of a wide range of harrowing, incomprehensible failures."
There isn't really one. This is a system that's supposed to have it's own built in redundancy and must not go down.
Flight Service uses a different set of radios and can relay messages to ATC, but they've been slowly dismantled over the last two decades and wouldn't have the capacity to step in for an outage (nor should they be expected to - that's not their purpose).
Some aircraft have CPDLC for text-based communication, but it's asynchronous and you can't use that as a replacement for voice communication in the terminal environment. Also, many aircraft aren't equipped to receive it.
Pilots are trained on lost comms procedures, so if you're in the air and the radio fails they'd fall back to those regulations. Basically all the aircraft would fly their last assigned clearance, and attempt to deconflict manually with other aircraft over air-to-air radio (similar to flying into an untowered airport).
I would bet the problem is not strictly the radios themselves, they will still work, Planes can talk to each other and directly to airports, But that there is a large scale highly integrated control center network that failed, and planes can no longer talk to centralized traffic control.
So it falls back to local control, more in the style of visual flight rules. Pilots in existing flights will have to be vigilant and do their own deconflictation and new flights are banned out until the network is back up.
All conjecture at this stage of course, but all the radios won't just fail. It is probably going to be a poster child example about the dangers of large scale integration. The attraction of course is that integration is very efficient, the danger is it tends to fail all at once instead of in smaller pieces.
Technically, you do not need radios. There are many many uncontrolled airports in the United States, where you can technically fly around without establishing two-way radio communication. It’s doable, but I don’t know if that’s even a scenario pilots test in the simulator during their semiannual trainings.
Most air traffic is IFR. You definitely need radio for IFR traffic in for deconfliction in most of the world. (There's a few exceptions in very remote places, but that's mostly academic.) Even legacy procedural (non-radar) separation assumes you have working radio, and a radio failure is an emergency.
VFR traffic doesn't necessarily need radio, and is not affected by this groundstop.
You don’t need radio for IFR (as the whole point of a flight plan is that if you lose contact you continue to fly your plan). But you kind of do need radar.
You actually don't need radar, and ATC can do non-radar "procedural" control where they block off sections of airways or entire airports, one-in one-out at a time, like sections of railroad track. There are still a few parts of the remote US, as well as large sections of Northern Canada, where radar coverage is nonexistant and ATC is done via procedural control. Radar outages also happen on occasion, which would necessitate this.
As for lack of radio, this would only work if ATC blocked out your entire route from start to finish - which isn't practically ever done. Otherwise, they need to be able to maintain positive control over your flight since the airspace isn't reserved, which means conflicts are possible.
If you lose radio communication, it's an emergency and they'll move everyone out of your way. This doesn't work very well if everyone loses radio communication simultaneously.
Right, but isn't a lot of that necessary radio aircraft-to-aircraft, like ADS-B? Obviously not ideal, but I thought IFR traffic could feasibly maintain separation without ATC help.
ADS-B can't legally be used to maintain separation. It's only approved for use as an aid to awareness for additional safety.
Could you in an emergency, if you had no other options? Yes. But ATC can't legally give you a clearance that was dependent on using ADS-B for separation.
(Similarly, if there's a potential traffic conflict, and you tell ATC you see the other traffic on your ADS-B display, they just have to disregard that statement and assume that you can't see the other aircraft at all.)
AM Radios are literally everywhere, and very commonly available.
I would hope that airports keep a battery operated radio + antenna in case of emergency with them, to at least communicate with aircraft within visual range of the airport.
@nxobject I meant litterally. The ATC system is derived from the IBM 9020 which was itself a set of interconnected System 360 systems: https://en.wikipedia.org/wiki/IBM_9020
That's a little unfair to IBM – sure, you're just as beholden to them for life, but at least the service engineer gets to your rack before you know something's gone wrong!
That explains the outage. We're just in one of the 5 days of the year they don't cover...
I did a tour of Adelaide airport with Engineers Australia in the 1990s. We visited their obsolete IBM main frame, still running ATC in their computer room. They were tied to it by the certification of the software it was running. Just as you described.
I'm wondering if this is another case of delayed Y2K, where the system suddenly thinks the century part is 19 instead of 20 for a two digit year of 26.
This message was pulled from a ATC communications system ordering the stop. These are terse direct communications for professionals who don't need or have time to read the story of what's going on. The news, not having any initial sources besides this is just reporting it, at the time they had no more information.
Like if you're ordering a pizza they don't need the story of your day, why you chose pepperoni, or what movie you plan to watch. Somebody else might be interested, but not the pizza guy.
> The FAA has reassured the public and airlines that SMART only suggests routes and doesn't affect air safety.
> [The] FAA also says a D.C.-based team of experts validates findings before they’re ever passed on to air traffic control, and the controllers retain the final decision on whether to accept SMART’s suggestions.
Depends if an AI DoSed an important service or not.
There used to be a lot of invisible gatekeepers in a pre-AI world. At work, everyone can ask their agent to query SQL on databases that they were supposed to use through an application. The application and the security by obscurity(end users didn't know how to connect to the DB nor did they know SQL) were the gate keepers.
Doubtful. That tool likely suggests "improved" clearances, which a controller can use at their discretion. It likely does not control clearances directly.
And the early reports describe the issue as related to ATC communication infrastructure.
While I don't have insider knowledge of this particular deployment, there was a strong industry-wide push towards "ship unreviewed AI generated code and hope for the best", partly driven by the fact that AI writes code faster than humans have bandwidth to properly review.
Under the hypothesis that some under-reviewed AI generated code made it into prod, the situation could be qualified as "engineers are panicking, because they don't know if they are the cause or not"
I do not really understand what the FAA means by “inbound international flights will be managed on a case by case basis.”
I am watching planes stacked up on final approach to land at KBOS as I type. From flight radar 24 they are a mix of foreign and domestic. So incoming flights are clearly being managed here.
Does this imply some potential risk from inbound international air traffic? Or maybe the FAA meant outbound? Something else I am missing?
I suspect they mean that inbound flights from international airports may still take off (probably in part because the FAA can't _really_ stop them, in part because the volumes are much lower and thus more manageable, and in part because one might hope that by the time it takes to get to the US from many international destination, the incident will be resolved).
Very unlikely that any international flights bound for the USA are taking off. Probably many are turning around or diverting if they are not more than halfway there.
I'm not following this very closely, but I've not seen any reports of diversions for domestic flights, so I think it'd be unlikely for international flights. The report is just for a ground stop, which is just a stop in departures; normally a ground stop would allow flights already in the air to travel to their normal destination.
That reasoning doesn't really make sense. A flight from outside the US that is inbound could divert to a non-US airport that isn't impacted by whatever is going on, depending on where they are and what their fuel status is. A domestic flight doesn't have a non-US airport to divert to.
(Sure, a flight near the Canadian or Mexican border could in theory divert to a Canadian or Mexican airport, but it's unlikely many people on those flights have their passports with them.)
Ah, my point was more that if the action being taken isn't "get planes on the ground _now_" and so the US airspace remains open for traffic already in the air, then it's unlikely they're trying to get international planes to either divert or return.
> (Sure, a flight near the Canadian or Mexican border could in theory divert to a Canadian or Mexican airport, but it's unlikely many people on those flights have their passports with them.)
There must be some procedures in place for this - I pretty regularly take domestic (Canadian) flights that pass over the US where I don't carry my passport.
A situation kind of like this happened to a relative in England; their international flight was diverted (I think due to weather) to a domestic-only airport; due to lack of immigration & customs facilities, everyone had to remain on the plane until it could take off again (presumably after being refueled) to get to an international airport
A domestic US flight can still divert to Canada. There usually are special arrangements that allow people on a diverted flight to stay short-term while the situation is resolved, even if they don't have the required documents or visas.
That is interesting and why i was wondering if they meant outbound flights but screwed up the language. I had assumed (probably wrongly) operators who should be risk averse would not want their US bound flights taking off under these conditions.
Not in the industry, but I think ground stops might actually not be a regular/automated procedural thing for inbound international flights in the way that they are for domestic flights.
As for asking already airborne flights to turn around, those coming in over the ocean might simply not have a non-US alternate airport available with their fuel remaining, so that's often not an option.
Yes, but most of them are not viable in the late stages of a flight. Planes aren't carrying enough fuel to cross an ocean for a second time; most don't even have the capacity.
The diversion airports are chosen specifically for each stage of the flight, so a flight doesn't ever need to return to where it came from.
Oceanic flights are planned so that at every point along the route, the aircraft can lose an engine or cabin pressure and still reach a pre-approved diversion airport with enough fuel, with the flight plan marking the points where turning back versus continuing becomes the faster option. If a diversion is needed mid-ocean, the crew offsets from their assigned track and altitude to stay clear of other traffic, then heads to the nearest suitable airport, such as Gander, Keflavik or Shannon over the Atlantic, or Anchorage, Cold Bay or Midway over the Pacific.
Please suggest an appropriate international alternate for a plane just about to land in Denver with an hour of fuel remaining.
Alternate airports are picked based with regards to all kinds of contingencies (many of them meteorological), but "the US closes literally every airport" is not one of them.
And that's exactly why you got the reply that international flights MIGHT not have enough fuel left to divert (and thus obviously have to be allowed to land within the continental US), but for some reason you insisted that every flight would be able to.
Doesn't seem like it; I've seen several international flights land in JFK alone since this started. (On the other hand, these might simply not have enough fuel left to divert to any non-US airport.)
Canadians, like Americans and practically most people in the world would band together to help folks in situations like this and do so with warm and loving hearts.
Maybe they mean inbound international flights may be allowed to take off from their origin, on a case by case basis. Usually a clearance is required to take off from both the origin and destination.
Presumably it means they may allow some international flights to the US to depart if they expect the issue to be resolved by the time they are in US airspace.
I think it's more mundane than that, I assume they mean that certain international flights will be turned around or redirected while others will complete their route. Say a flight just an hour underway London to New York might be turned around while one nearly at its destination may be allowed to land.
Depending on the nature of the disruption it may be they simply have drastically reduced capacity to handle traffic so landings may be much slower and they'll manage the traffic based on various priorities.
Probably just means that if you're less than halfway across the Atlantic, you can turn around and go back to Europe. I really wouldn't suggest reading too much into the phrasing.
Flight radar is showing many flights that departed within the las hour. Maybe they are doing some controlled international departures since communication would be critical at arrivals (specially at busy hubs)
Edit: If you look now over the north atlantic you can only see outbound flights
My wife is on a plane on the ground at ORD (they just landed) and they have been told they can use the restrooms but can't get off the planes. Any reason why folks wouldn't be allowed to deplane at the end of the journey if it was a radar or comms issue? They can pull up to the gate using flags . . .
This seems like one of those efficiency-vs-resilience problems. Sure, it's "more efficient" (in cost) to use the same empty/fill process for every plane, but when nobody can leave, some extra capacity to unload planes would help.
You don't have to go to a gate to deplane and this is an extremely rare event. They'll just pull up a staircase and have them deplane to a shuttle eventually.
We are 6th in line waiting for a gate while they tow grounded planes off of the gates. Normally those flights would have departed and we would get a gate fairly quickly, but if no one is taking off, then they need to move them, and potentially deplane the (angry) departing passengers.
The only time I've spent long stretches on a grounded plane have been at ORD. It's such a busy airport than any disruption to the schedule just causes all sorts of chaos.
> In a notice posted on its website, the agency said technicians are working to resolve the disruption as quickly as possible.
This article is very likely misreporting the US National Weather Service radar outage page you can find people reposting elsewhere in this comment section. The NWS page has been updated to say it does not reflect a current issue. Anecdotally, my local NWS weather radar is working fine. Also, weather doppler radar and ATC "radar" are very, very different technologies.
Am sitting in GRU waiting for an outbound flight. Have just seen two flights (DL268 to ATL and DL313 to JFK) change to Boarding and now Last Call status.
My flight to the US is listed as Delayed but there are two flights to Canada that show Contact the Airline. Bewildering.
>The US National Weather Service said on Friday it was experiencing nationwide internet and communications problems that could prevent users from accessing products on its websites or cause outdated information to be displayed. Technicians were working to resolve the problems as soon as possible, the NWS said in a statement.
My wife's plane was about to take off (moving) but then stopped. I just messaged her to check once I saw the news. Sounds like the people onboard weren't given any info as to why. They're still on the runway waiting.
Physics says all planes land. I guess eventfully would be diversion (maybe to Canada lol) or Mayday Fuel (but you probably won't know it happened anyway - it just means some reserve fuel will be used).
ATCSCC ADVZY 117 DCC 11/13/2026 ALL CENTERS EXTENDED FLIGHT PLAN DROP TIMES IMPLEMENTED
MESSAGE:
EVENT TIME: 13/2120 - 14/0500
CONSTRAINED FACILITIES: ZTL
DCC HAS IMPLEMENTED EXTENDED FLIGHT PLAN DROP TIMES TO 240 MINUTES
FOR ALL ENROUTE CENTERS DUE TO NATIONWIDE COMMS FAILURE.
EFFECTIVE TIME: 132121-140530
SIGNATURE: 26/11/13 21:21
ATCSCC ADVZY 120 DCC 11/13/2026 NATIONWIDE GROUND STOP
MESSAGE:
EVENT TIME: 13/2245 - 14/0400
DESTINATION AIRPORT:ALL
FACILITIES INCLUDED: ALL
EXPECT UPDATE:0400Z
REASON: COMMUNICATIONS FAILURE
REMARKS: THE FAA COMMAND CENTER GROUND STOP REMAINS IN EFFECT. AS
ENROUTE FACILITIES RECOVER, FLIGHTS WILL BE ALLOWED TO DEPART
THROUGH COORDINATION WITH THE AIR TRAFFIC CONTROL SYSTEM COMMAND
CENTER.
EFFECTIVE TIME: 132246-140430
SIGNATURE: 26/11/13 22:46
It means that flight plans (which include departure time) will stay in the system longer after the departure times are missed, so that IFR flight plans will not need to be re-filed. In other words, flight plans have a longer TTL.
This will ease the pain of getting things moving again, assuming they fix it by the time the 4hr window passes.
You can definitely file a flight plan later than that. 30 minutes before departure is the recommended filing time (per AIM 5-1-6), but you can even file for an immediate departure. It's just that at less then 30 minutes additional delay is possible, depending on backend handling and human processing. But in most cases the strip will print within a minute or two of submitting the flight plan.
But also, this is about when the flight plans expire out of the system if they're not activated for departure - so it's saying unused flight plans won't be deleted for 4 hours, so pilots won't have to refile for departure.
Most of these systems started out as teletype printers, back in the day. The teletype systems only had an all-caps character set.
Aviation is global, and we still need to interop with countries and airports that still use actual teletype. So the legacy protocols still remain, even though much of it is tunneled over IP links now.
Also: The keyboards used in both airplanes and in ATC facilities are often special reduced-footprint keyboards without shift/caps keys. (They're also often ABC layout rather than QWERTY.) So there's that, too.
Actual teletype machines? No. (Though flight strip printers used at ATC consoles are basically mini teletype printers.)
However, AFTN (used for disseminating flight plans and weather between ATC facilities) still uses landline modems running TTY-originated protocols.
In the US we've moved moved off AFTN in favor of AMHS/X.400, as have most countries - but my understanding is there's still some last holdouts that haven't fully switched over.
Probably for the reason that the original monocase teletype machines did. Better absolute readability due to the larger characters. It tends to be very dense with little actual prose so it is better to reduce the possibility of mistaking one character for another.
Also, everyone is used to reading the upper case so they would have to spend some time learning what common things look like in lower.
We are experiencing technical difficulties. We apologize for the inconvenience but please check back soon, as we are working to have the issues resolved.
Anybody else find it fishy that this happens the day after Microsoft goes partially down across the globe? They were about as hush-hush as they legally can be, and then this from the FAA. Seems fishy.
Maybe just paranoia but it seems slightly suspicious for this to happen while we are in the middle of discussing massive funding and overhauls to the air traffic control systems. This will certainly end up pushing a lot more funding into these programs and companies now that were trying to sell more expensive systems.
I guess we will see what happened in the aftermath. But I very much doubt the current problem has anything to do with a lack of funding.
And I don't see anyone arguing about overall funding, only arguing about who to throw millions or billions of additional dollars at chasing vague ideas of how to improve things based on what politicians and C-suites, who have no experience in the area, claim they can do. To me it looks more like grifting for contracts and kickbacks than any well thought out plans to improve things.
EVENT TIME: 13/2100 - 14/0300
USERS CAN EXPECT AIRBORNE HOLDING TO ANY AND ALL AIRPORTS WITHIN THE
NAS DUE EQUIPTMENT OUTAGES. UPDATES WILL BE ISSUED AS ADITIONAL
INFORMATION BECOMES AVAILABLE.
EFFECTIVE TIME:
132120-140330
SIGNATURE:
26/11/13 21:20
Edit: end date of tomorrow??? Is that just temporary?
It seems that the adv/ pages are not loading any advisories, though I am just guessing and have no idea how this is meant to work. The fact that the search page has a date input would seem to imply historical advisories should be shown. If you go to https://nasstatus.faa.gov/ and click the Advisory links for any of the active events that have one, they all show as blank too. Could be totally wrong though.
This happened to me a few minutes ago. I clicked that link, it said "no advisory", I was making the same comment you made, then tried refreshing and the advisory came back.
Insane. It looks like they are showing no advisory on first page view, then advisory? Can others confirm?
Skies over Portland are empty except for a scenic tour company flight that just took off from Troutdale, and is flying over downtown. Are there some exceptions for general aviation?
If a lot of planes on the ground were prohibited from taking off, but couldn't get back to their gates due to congestion/traffic, where will the planes go that are still in the air and ultimately land? Just disembark passengers out on the tarmac? (hopefully with some sort of bus pickup)
dumb question but do the airport gates get "too full" if all plans are grounded? Do airports sort of expect some constant turnover to have enough open gates to service everything? Do you end up with a line of plans waiting for a gate?
Very, very much yes. Most busy airports will start having problems with gate capacity due to just normal delays. We definitely don't have the capacity to have every active flight in the country parked at a gate.
Based on my current experience on the ground at AUS, yes.
Gates were quickly full, ground ops to pull empty planes off gates takes quite a while, and about 90 mins later, we are pulling into our scheduled gate.
They don't spend a dollar more than they need to so extra gates for this would be quite the luxury. I guess what they need though is apron space or hangar space for the extra planes and the number of gates won't matter so much.
ASK HN: what so the biggest potential disaster you could have caused if you had don’t your job badly?
As in what’s the most responsibility you’ve ever been given to not screw something up - obviously if you did screw it up. Still very interested to hear the story.
Listening to SFO Tower, the latest info I’ve heard is an expected stop until 04:00 UTC, but flights departing to Asia via west departure may be permitted to depart.
I've read scattered reports of other flights also taking off from the east coast.
If the communications issue is between US ATC regions, this makes sense. The planes are being handed off to other non-US ATCs where there is no communications issue.
I was actually expecting it to be from the Microsoft 365 outage. Non technical people have a bad habit of inserting email as a critical step in a process.
It's actually quite impressive, you have to really try to be so bad at your job of building redundant systems that some failure mode implausibly affects everything nation wide, yet allows Canadian domestic flights to proceed fine.
ATCSCC Advisory
ATCSCC 111 DCC 11/13/2026 NATIONAL GROUND STOP
MESSAGE:
EVENT TIME: 13/2053 - 14/0400
DUE TO EXTRAORDINARY CIRCUMSTANCES AND FOR REASONS OF SAFETY.
ATTENTION ALL AIRCRAFT OPERATORS, BY ORDER OF THE FEDERAL AVIATION
COMMAND CENTER ALL AIRPORTS ARE GROUND STOPPED. ALL DEPARTING
TRAFFIC SHOULD REMAIN GROUND STOPPED UNTIL FURTHER NOTICE. INBOUND
INTERNATIONAL FLIGHTS WILL BE MANAGED ON A CASE BY CASE BASIS.
EFFECTIVE TIME: 132054 - 140430
SIGNATURE: 26/11/13 20:54
About 15 mins ago I got off BART at Daly City and a plane took off from SFO and flew overhead. Didn't look like anything special, just a regular passenger jet to me.
What does this mean outside of the US, I.e. I suspect planes that should be in places abroad won't be so ongoing cancellations and crew in wrong places etc.
My thoughts immediately went to the recent discovery of audible airplane noise that can pinpoint unknown locations. I'm betting someone in power wants to record something without getting doxxed.
Due to "rolling frequency outtage" [sic]: From a claimed photo of FAA documents (I replaced the all caps with lowercase):
"... ground stop is in effect for all destinations until further notice due to rolling frequency outtage in multiple centers..."
and
"... the ground stop is now a nationwide ground stop for all centers, all destinations until further notice due to widespread frequency outtages.. ..."
(EDIT: @mods / @dang: Around January this year (iirc) somehow my account ended up rate-limited, posts relatively demoted (start low, drop quickly), vouches/flags relatively ineffective, etc. Usually I don't care too much and haven't bothered to do anything about it but it is sometimes annoying - it limits participation - and right now it buries some of the more valuable information in the discussion. Could you look into fixing it? I don't know of a reason this account would be penalized. Thanks.)
Is conjecture allowed? I have no evidence of this, but I've noticed that often before big meetings between superpowers one side or the other seems to do a minor or subtle attack as a kind of sabre-rattling/warning to the other side to try to gain leverage (and show them they're not to be trifled with). So with that, I wonder if this is a China/NVIDIA accusation connection.
It is hard to not rapidly spiral into pure conspiracy theory territory, but one has to ask how many "black swan" events can you accept within a period of a few days before you start to question things?
A sequence of events:
- Democrats win congress : it sets a clock to act before January
- Jensen is taken out, throwing nVidia into chaos
- Email comms are blacked out for an entire day across government and major enterprises
- Air traffic is stopped nation wide the next day
Too much pattern recognition. Lots more things have happened in the world as well over the last month, just because you heard of some of them doesn't mean the ones you heard of are connected.
Trust me I have the instinct as well. The PLA flew like 30 jets into Taiwan's aviation alert zone yesterday so I'm primed for any smell of something worse, but the fact is there's no reason to connect events barring evidence.
That tool was first unveiled for the Washington, D.C., area airports two months ago, on a day when problems with air traffic communications failures also grounded hundreds of flights, after a circuit failed and it was discovered that contractors had accidentally ripped through a fiber line that was supposed to serve as its backup.
I know we're all having fun in the comments. But please think about this next time you vote, local/county/state/national. There are people in charge who are not funding upgrading these systems, because their constituents don't want their taxes raised to pay for it. That lack of funding is annoying to some, has an enormous economic impact to many, and is now risking tens of thousands of lives who are literally up in the air. There are planes carrying body parts destined for transplant that are grounded. People who need to get home to their families. All because nobody wanted to pay for what we knew we needed to fix.
One thing to add is that this is bipartisan: both parties have failed to adequately fund ATC updates, and it sometimes feels like there's more interest in vanity projects than fundamental infrastructure funding and solving personnel shortages (ATC is a very difficult career, and the pay, benefits and prestige are not incredible. Fewer and fewer people are choosing to go into this very critical line of work).
>The FAA has issued a nationwide departure ground stop “due to several impacts to ATC Frequencies and/or Radar capabilities,” the Air Current reported, referring to air traffic control systems. The FAA command center said it does not have further information on the “timing or cause of this outage,” the publication said.
"Today's event is therefore generally being described as the third nationwide grounding event since September 11, 2001, counting 9/11 itself. The three incidents differ significantly in cause and scope."
Doubtful. That tool likely suggests "improved" clearances, which a controller can use at their discretion. It likely does not control clearances directly.
And the early reports describe the issue as related to ATC communication infrastructure.
>I can't believe people actually think outlook being down would take down all departures across the US
Because American infrastructure is modern, well designed, maintained and funded? Because government is competent and rational? I could absolutely believe that would happen.
If true and this ends up lasting an hour or more, that's going to be undeniably, objectively hilarious. At least they used to keep the planes running on time!
--- EDIT:
This could just be the first issue that's arisen -- the FAA announced it "is expanding across the nation" on Tuesday. Which could really mean anything.
The Ars Technica article behind the Gizmodo one sadly doesn't have more info, either.
It'd be kind of absurd if a read-only integration took down nationwide infrastructure, but A) we've all been there... and B) the gizmodo article does mention they're also trying to phase out floppy disks for some jobs still, which tells me their CI/CD and rollback hygiene might not be top-notch. Hopefully the Real World contestant in charge of our transportation network is on site with a ClaudeCode terminal as we speak!
It does sound pretty deeply integrated, I will say:
SMART is a cloud-based system that centralizes 200 data streams – including weather patterns, flight paths, traffic flow, and controller staffing metrics – into one platform.
The fact sheet has two more vaguely worrying bullet points -- I would say such wild speculation is unwarranted, but I struggle to think what else could cause all the radios in the nation to go down at once... Emphasis mine:
Creates **a single shared real-time picture** of schedules, weather, airport capacity, and airspace conditions.
...
Enhances the effectiveness of **existing** FAA air traffic management systems.
Oh boy, that feels plausible. Some of these legacy systems are pretty fragile and aren't designed to be queried heavily outside of well established patterns.
I could see AI deploying something that just obliterates those systems with requests.
If so we are really speedrunning into idiocracy with AI.
Now Sean Duffy is basically saying its because the FAA needs more than the initial 16B USD budgeted, saying they require 32B and then demanding 50B just because.
Definitely nothing to do with the current administration, they are hampered by their $16 billion allowance.
US-bound flight from YYZ. They shut down #2 engine and parked us in inactive runway. Pilot came on and said he was reading something about "ATC radio frequency issue" but has been told by United ops that it'll be cleared in ~30 min from now.
Verizon Wireless is out here as well ("SOS"—thankfully my UA flight has Starlink).
Last time this happened in July it was due to some network handoff issue so maybe there is some fiber cut somewhere? (My friend said last time it was NE corridor Verizon fiber cut.)
It's been lifted. And naturally the political appointee is desperate to credit his lord and master.
In this time of high corruption and near-zero trust in officials, it seems wild to suppose it was just a random glitch. I would bet this links to corrupt government contracts.
Any three of those pessimistic scenarios seem plausible though. Frightening. Intentional hack/exploit of air traffic control systems is the first thing that came to my mind when I first saw the news.
Is this going to end up being the work of rogue Anthropic or OpenAI agents that “unexpectedly” escaped so they can regulate open weight models under the guise of preventing them from doing the same thing 6 months from now?
If this is related to new FAA SMART system they're rolling out, it's might have bean so smart it grounded itself before anyone found out it was broken.
There's enough space, but not enough gates. If they think the stop may end in a reasonable time some planes will be pulling away from their gates even without a departure time to make room for the landing planes (or rather, they will have missed their departure time and not have a new one). Landing planes will be able to disembark, and probably won't be boarded again until it's resolved and enough departing traffic has cleared that it's obvious that those flights won't be canceled for today as well.
The boarded and grounded planes may start disembarking passengers at some point, especially if their crews are hitting their maximum time before mandatory rest. These delays will probably prevent replacement crews from arriving.
Yes, and they'll probably do that at some point if they can't get them to the gates. Again, it'll depend on how long this lasts. They will also have to get in more crews to support that since this is not normal and not something they would have planned on in their staffing schedules for a Friday afternoon/evening.
There was a time when we'd get clear and honest communication about this but now our admin is Baghdad Bobs all the way down. They will lie about it, even if there is no rational reason to lie about it, because this admin and the movement associated with it wants you eternally guessing what the truth is.
If tomorrow there is third major outage of an important infra-structure level service, maybe I will start to get a little conspiratorial about cyberattacks and such.
Fox News says that Markwayne Mullin says that the FAA says that it's not a cyberattack, and it's not something like 911. Just heard that, don't have a page to point to at the moment...
No, please stop engaging in baseless and pointless speculation. If the US wants to bomb Iran it doesn’t need more reasoning than it has already used to already do what it has done.
No they don’t - popularity isn’t all that relevant. The president can just order strikes tomorrow.
Besides popularity isn’t the determination of whether something is a good idea or not. The Korean War for example was very unpopular yet it was the right and smart thing to do.
The Korean War was absolutely not the right thing to do. It was a literal terrorist act by the US that killed 2.5 million civilians. I can’t believe you used that as an example.
Well that’s wrong. The Korean War is what stopped authoritarianism from spreading and encompassing the entire Korean Peninsula. The blood of the 2.5 million civilians you’re citing falls on the tyrannical governments of China and the Soviet Union and their evil lapdogs in what is now North Korea.
We paid more than $800MM for "route suggestions", btw, a problem that doesn't seem to benefit at all from "AI"—a term reserved for fantastical algorithms. Surely you could solve this with traditional search algorithms without sacrificing your dignity!
Company that created the tool (called Airspace Intelligence) is interesting in terms of demographics for these types of companies - implies lots if the tool 'does not work as designed' (which it will obviously not)
Doubtful. That tool likely suggests "improved" clearances, which a controller can use at their discretion. It likely does not control clearances directly.
And the early reports describe the issue as related to ATC communication infrastructure.
A team of FAA specialists in the SMART center in Washington, D.C., analyzes and validates the SMART data and provides scheduling recommendations to the agency’s Air Traffic Control System Command Center in Warrenton, VA. The Command Center then routes these recommendations to air traffic control facilities for local leadership to consider.
Of course, these may be the ATC analog of claude code approvals, or they may be overstating the level of human oversight/HIL - but the overall situation seems likely more serious/unclear than something like this. I'm not sure I'd call vibe air traffic control going haywire an "extraordinary" circumstance, though certainly extraordinarily stupid.
No, and when Netflix goes down it's not a false flag due to their fear of people running their own Plex servers. Perhaps you just have anxiety about losing something you depend on, so you're projecting.
I wonder if it was Claude or ChatGPT? My money would be on the latter. Anthropic seems to generally have better security and guardrails, and apparently OpenAI’s newest cyber model is extremely capable.
This week alone, I've seen multiple servers fail due to hardware issues: multiple bad fans, a failed hard drive, bad network optics. Another server got bumped while an adjacent server was being worked on and lost time sync, leading to software instability/crashes. We don't have a large fleet, but we do have more servers than employees, and issues add up. We don't need any help with instability, entropy has us covered, thank you very much.
The nifty thing about stories is that they don't have to be about something directly to trigger strong feelings in your crowd or listeners.
When we are dealing with the FCC, grounding all flights and radio failures in open bands, you can drive a lot of changes.
There are a lot of pesky ESP32s out there, right? Did your company choose to go with the Espressif platform? You know, the hackable SDR one that crashes planes and clubs baby seals and tries to sell your grandparents bad financial scams?
starting to see some explanations for what's going on
https://www.wsj.com/us-news/faa-temporarily-halts-flights-ac...
> The outage appears to be affecting a telecommunications network overseen by L3Harris Technologies, according to people briefed on the matter. The network handles crucial radio frequencies that air-traffic controllers use to communicate with pilots, the people said. It wasn’t immediately clear whether the outage also affected the radar feeds that controllers use to keep aircraft safely separated in the skies.
https://www.cbsnews.com/news/faa-issues-nationwide-ground-st...
> An update to the private communications network that connects America's air traffic control system caused the system to reboot, leading to a crash and a communications outage, multiple sources told CBS News. The system is operated by L3Harris, which did not immediately respond to CBS News' request for comment.
Anybody familiar with these systems know why they would do an update like this at 3pm on a Friday? This would be extremely unusual in any software job I have ever had. For commercial aviation specifically which does have pretty well defined peaks and valleys in terms of traffic over the course of a day this seems like an odd time to choose
I did web dev work for an airline and we couldn't even release new updates to the loyalty benefits website during a weekday (even though there was no connection to operations or flights). The fact that this company would reboot a critical comms server at 3 pm on a Friday is indeed bizarre.
I believe not deploying on Friday is an anti-pattern for most software teams.
What industries have you seen teams deploy on a Friday afternoon?
In public utility we only did deployments on Fridays after 5PM
We do for clients who have 24/7 users during the week.
I don't think the hesitation to push to prod Friday afternoons has anything to do with the working schedule of your customers. It's more about whether you'll be able to roll back on a weekend.
Why wouldn't you be able to rollback on a weekend like any other day?
cause I don't want to work on the weekend. Why would I introduce that risk?
You can, but it also means you might have to.
Not a family man, I see?
or want to have to.
It happens. My company has done a couple dozen production rollouts today, including minutes ago.
Team America: World Police
Uh I'd imagine any team that mainly has their user base using the product on weekdays would deploy Friday evening so if something goes wrong they have the weekend to fix it.
Only if the team wants to work weekends?
If something goes down on Saturday there’s a good chance I’m out of cell service
IT in corporate is actually more panicked than IT in life safety, because life safety invests more redundant design.
the update caused the reboot but could that mean the reboot happened due to a prior update? or literally the reboot was FOR the update hmm
> we couldn't even release new updates during a weekday
so you were forced to release updates on weekends? isn't that worse?
It depends on the schedules of the people who'll have to fix it if something goes wrong. I don't mind working weekends, so when I was in the sysadmin game I always tried to make significant changes late friday/early saturday, so if something went wrong I had the weekend to fix it & clean up. But if the people needed to fix a problem take weekends off, pretty much the opposite schedule applies.
Flights generally peak Mon-Thu due to business flights, weekends are usually lighter unless it's a long holiday weekend.
Worse for the employees. Better for the clients.
"3 pm on a Friday is indeed bizarre"
Most likely, they HAD to do it.
Probably was scheduled for Thursday evening but was delayed due to the Outlook issues.
It should be fine to update at any moment. Why? Because systems like this ought to be hot spare or blue/green models that have a full working copy at any given time, and a failover/failback.
A reasonable deployment plan considers the reality that the deployment could fail. One mitigation is to choose ideal conditions for a possible failure. You’re correctly pointing out one of the mitigations, but it doesn’t render the others unnecessary.
You can also have reasonable updates done at reasonable times that don’t show issues until unreasonable hours - or something else fails.
But what I’m hearing is that one company “L3Harris” could take down the airfristructure anytime it wanted to. That’s not great.
As opposed to what alternative? Being run by the FAA itself or with contractors? Being run by the DOT?
Well, it worked so far, I think it is great. 100% availability is a dream.
That’s why for example, you have ATC Zero protocols when communications go out all of a sudden at airports. And I imagine there is something similar for airborne planes that are not near any of their destinations.
Yeah either the FAA or the DOT sounds good. Not contractors tho.
you seriously think the government would do a better job?
I mean they also make the CPU for the F-35, alongside kinetic strike and electronic warfare missiles, so they could cause plenty of damage if they wanted to. This isn't some mom-and-pop radio company.
I mean sure and we have that in finance but there’s no amount of time saved that could make me deploy something during market hours
Airspace certainly has busy and quiet(er) periods, but the skies are open all day every day.
Well clearly they don't have a hot spare or blue/green models, or working failover.
Which is why deploying at 3pm on a Friday is even more bizarre.
Sure, but if it fails and a fallback takes over, will you be going home and having a nice weekend? Or will you be stuck at work over the weekend to fix it?
I'd much rather the team's weekend be impacted than millions of travelers.
Maybe patching a critical vulnerability? I’m also at a loss for why they chose Friday afternoon.
That could be their least busy day, for all we know. Weekends are busier than weekdays for airline travel.
But 3am is when a big chunk of planes are parked at gates awaiting the start of the flight day (yes, there are red-eye flights and international but they are a minority.)
Overnight periods are when the air travel system largely resets itself from each day's cascading delays, mechanical failures and bad weather.
Friday and Sunday are the busiest days. Business and pleasure overlap. Sat is mostly pleasure. Mon-Thu are mostly business.
I'd wager management wanted this by the EoW
Arbitrary day-of-the-week and date-based deadlines piss me off so much.
I worked for a person who had a fixation on "neat" deadines: the "first of the month", "the end of the week", "the start of the quarter", etc. It'd be one thing if the dates were tied to legal compliance (tax filings, etc), but that usually wasn't the case. They were chosen because they were "tidy".
I'd time meetings looking to maximize time for work. You try to meet late in the day Wednesday because an early Tuesday meeting might get an "end of the week" deadline. A late Wednesday meeting was more likely to garner a "end of next week" deadline. Meeting Thursday or Friday guaranteed an "end of next week" but you lost Thursday and Friday work time. Wednesday was a gamble, though, because it was still early enough to get you an "end of the week". Meeting after lunch helped with that.
You always had to be aware of the numeric calendar day, though. Gunning for an "end of next week" deadline when next Wednesday was the end of the month (or worse, end of the quarter) almost guaranteed you an earlier deadline than you wanted.
All of this deadline-setting happened with little regard for the amount of time work was going to take.
It was immensely fun, when setting a deadline in this person's absence, to get a group to agree on a mid-week deadline. You could see it make this person's skin crawl hearing that the work would be done on Tuesday or a Thursday.
"The project will land on the first full moon after the vernal equinox"
Weird how all epic stupid decisions come from the people who are supposedly the trained 'decisions makers.'
I spend at least 1/3 of my fridays answering all requests with "Read Only Friday."
RCA could be in just 2 words: incompetent people
Mission critical systems that are in use 24/7 are not managed based on week day vs weekend shifts. Or at least shouldn't be.
My (hopeful) assumption is that air traffic control isn't based on the "weekends are the off hours" mindset.
A security patch would be a reason not to wait.
And because a security patch can't wait no deployment should wait because it's Friday. That's how you stay ready; you're always ready.
Rolling out updates on a Friday afternoon is still practiced in various industries.
> Rolling out updates on a Friday afternoon is still practiced in various industries.
By people with unreliable systems and a lot of myhts behind.
In a well engineered organization, you can deploy whenever you want. Sure, it will not work 100% of the time, but you figure out why, fix the problem and you will be one step closer to the mythical 100%. And so on. Deploying less frequently is what creates more problems.
> Anybody familiar with these systems know why they would do an update like this at 3pm on a Friday
As an insider to the technology industry, I would surmise the technical reason for it is that they didn't want to work over the weekend.
(Yes, this comment is tongue in cheek, but not everyone follows the "don't deploy on Friday" rule, even people in mission critical fields.)
> I would surmise the technical reason for it is that they didn't want to work over the weekend.
Isn't that exactly what they did? If you do major change on Friday at 3pm you likely will work over weekend.
There is also a real possibility they did this many times before and it worked. I don’t really get it: if the thing would have broken on a Monday night and it would take a day to fix would that have been better?
That's why you don't deploy on a Friday. :)
This is why you don't push PROD on fridays...
Nothing like a good Friday afternoon did lot to ruin a weekend.
It's worse than that. It's 3pm on a Friday before Thanksgiving week.
I really hope we get an official or leaked root cause analysis and postmortem of this and it doesn't get buried because all the money in my pockets is willing to bet there was some deep incompetence responsible at the executive level.
No it's not. Thanksgiving is on November 26.
Perhaps.
But the entire first months of the Iran War was started on Friday after the markets closed. It took a few weeks for most Americans to catch on.
My money is on some mundane ops issue like "it takes lots of time to get sign off for the upgrade from lots of different stakeholders and the necessary upgrade was falling days or weeks behind, leading to a key IT person to decide to YOLO the upgrade/deployment".
I've heard that Sec Transport Duffy has been pushing FAA upgrades faster than is safe, so maybe the employee rushed it so this upgrade wouldn't hold back some other larger timeline. There have been several articles recently about how the FAA is full of people who are "too risk averse", suggesting that there might be an effort to push riskier projects faster. Having a nationwide outage is one of the likely symptoms of raising risk tolerance.
> It took a few weeks for most Americans to catch on
In what sense? It was well reported at the time, so presumably most people know about it. I'd be interested in what makes you think the time and day of week chosen would delay market or public reaction be _weeks_.
My money is on DNS, but if it is enemy action, will the FAA even know that's what it is?
Seeing L3Harris failing gives me a little bit of anxiety since their systems run some of the backend of some nuclear operations.
Huh. I thought Collins now ran what used to be Aeronautical Radio, Inc. (ARINC), which is the ground portion of air traffic control radio. L3Harris has the radars and ADS-B.[1]
[1] https://www.l3harris.com/sites/default/files/2021-08/MN_SBSS...
Collins/ARINC is used for oceanic HF radio. They run "San Francisco Radio" and "New York Radio" specifically, which are the HF radio operators that relay oceanic messages to ATC.
(They also own a small number of private VHF stations for relaying company messages to air carrier operations domestically, but these aren't used for ATC purposes.)
Otherwise, the VHF radio links are operated directly by FAA TechOps.
L3Harris owns the telecommunications backhaul between facilities.
Doesn't L3 Harris use Microsoft Azure Government for most of their mission critical infrastructure? Are we looking at fallout from the Microsoft infrastructure collapse yesterday?
How is there not a second backup network???
Cue that line from DS9 where O’Brien says he’d never be caught without a secondary backup during a fight, and the cardassian is like “you have a second backup?” incredulously. https://youtu.be/o7o9ZtipVyo?is=y3for_WXwISmuqpd
I wouldn’t say anything but this is approximately the only time in my entire life this scene will be relevant.
I remember that scene, but I can’t believe that you had it at the front of your memory like that. That’s awesome.
The more time I spend working in tech, the more I realize that everything important is held together with cardboard and duct tape.
True for every industry / field I've worked in, unfortunately (which is more than I'd like to admit).
You had duct tape? Lucky!
Why only “one” network at all!?
The Internet was purpose designed to tolerate disruption as major as a nuclear war, which it achieves through decentralisation.
Private networks exist to make idiot managers feel more confident while actually dramatically reducing availability by putting all their eggs into a single basket.
Many aviation systems predate the internet. Depending on the region, some systems do not use Internet Protocol, let alone TCP.
The FAA Telecommunications Infrastructure (FTI) operated by L3harris was built between 2002 to 2007.
That is already well within the era of IPv6, let alone "the Internet".
There's no excuse.
For something so incredibly critical in real time, I’m not sure I would want to trust the Internet.
When was the last time the Internet "went down" globally?
The last time the internet caused flight outages was September 22nd of this year.
thats like mandating using I-95 for all ambulance & firefighter traffic
Isn't using a private network like mandating using I-95 (since it might be blocked from a freak accident one day), while using the internet, with redundant paths between hosts, like allowing the traffic to find the best path?
One network can be a fully redundant design with multiple geographically distinct links and so on.
But it does not make it 100% resilient to cascading failures or possible Layer2/3 loops that might happen during a software upgrade, especially in a country wide deployment.
The FAA will give a root cause at some point of why this happenned.
> caused the system to reboot, leading to a crash and a communications outage
why a reboot would cause a crash? or it means that an update irreversibly broke the system and it was rebooted but that didn't help?
Yeah - like, why do they not have an "HA" cluster, where they patch and reboot a subset at a time... Something this critical should have some sort of failover/load-balancing in-play...
Maybe the systems connected to it are not properly designed to handle such a reboot
Apparently the L3Harris CEO was ousted in August in a Jensen-like scandal. It's also the company that was modifying Trump's Qatar corruption plane.
https://www.cnbc.com/2026/08/17/l3harris-ceo-chris-kubasik-l...
> Apparently the L3Harris CEO was ousted in August in a Jensen-like scandal.
You mean he facilitated sale of restricted AI chips to China?
Sorry, but I am tired of the effort to make it look like Jansen's crime was infidelity.
If he just went to whore house he would be fine (well, ignoring a possible expensive divorce).
There's no evidence that AI chips were sold to China. The chips were sold to Singapore. I have yet to see confirmation the chips were subsequently smuggled out of Singapore with Jensen's knowledge.
Gamers Nexus had some interesting coverage on the whole situation so far: https://www.youtube.com/watch?v=YYU_geUwQoA
The Nvidia chips that the Russian drones are using are consumer class chips (TE980M-A1) that have no export controls. It's odd that this channel decided to highlight that when it's very clear those are not on export control lists only the high level AI chips are restricted and controlled in this way. Those chips were sold to any country that has open trade with Russia then they could easily make their way to Russia from second-hand markets.
So from that alone I don't trust the integrity of this source's reporting.
You'll find Intel, AMD, and every other name brand of CPU/GPU in all countries on earth, that doesn't make those companies culpable.
Fair enough. That part was pretty short, the rest of the video making connections between the money and the companies was fairly convincing to me that something doesn't smell right.
>You mean he facilitated sale of restricted AI chips to China?
No, they mean "had a sexual relationship with a subordinate".
Which is what the linked article says.
>Sorry, but I am tired of the effort to make it look like Jansen's crime was infidelity.
Strawman. Nobody says that, and nobody cares.
Jensen's "crimes" that we know of are:
1. Having a sexual relationship with a person suspected of being a Chinese spy (who, allegedly, influenced NVIDIA's chip sales to China)
2. Having an undisclosed sexual relationship with a direct report
Both are huge liabilities for the company, as well as national security concerns (someone in a critical position putting themselves into a position for easy blackmail).
None are crimes, all are factors contributing to the firing decision.
>If he just went to whore house he would be fine (well, ignoring a possible expensive divorce).
Same goes for the ousted L3Harris CEO.
The number of creepy men supporting infidelity on HN amazes me.
Could you be more specific about the comments you’re referencing? I don’t see anything in this thread, at least at a cursory glance, that could reasonably be described this way.
Did you somehow see defense of infidelity in my comment (hope not, because it wasn't there), or are you reflecting on the general trend?
Who do you feel is creepy here (or a man, for that matter)?
Infidelity (in itself) is not the thing that got either of those 2 men fired.
That statement doesn't say anything about how anyone here feels about infidelity, nor whether it should or shouldn't be enough of a reason to fire a CEO.
I don't see anyone supporting infidelity here.
However, in a thread about Jensen, I've seen multiple people argue with me that the CEO having sex with subordinates isn't problematic at all (with total disregard for the power imbalance and potential for coercion).
That was, indeed, very sad.
>An update to the private communications network that connects America's air traffic control system caused the system to reboot, leading to a crash and a communications outage
And that's why you never deploy to prod on a Friday.
Especially Friday the 13th.
well is there a rollback procedure?
it could only be deployed from the servers that crashed
and they said "wont deploy on friday? thats 20% of the week in lost productivity!"
I have memories of my professor saying something in the lines of
"it's computer science, if you make a mistake nobody will die!
Unless you're dealing with nuclear plants, medicals, or airlines"
Thank you.
I learned more from this post in 15sec than I did in 20min from watching the Sec of DHS and Sec of Travel press conference
> leading to a crash
poor choice of words
This indeed is a poor choice of words…
Who updates on Friday?
Organizations that have good engineering practices.
This woodoo like belief you should not deploy on Fridays, Christmas, Valentine’s Day and other magical days is a disservice to the industry.
But when it goes wrong you’re working on the weekend.
I think it also creates a tension because people are more compelled to "just ship it" to go home. If it's a Monday and it's not quite ready yet or someone has doubts, more likely to just hold off.
I feel like people want to go home on Friday and will make sacrifices they would not make on a Wednesday.
Isn’t Friday usually a light travel day?
No, very heavy travel day for business.
No. Business and pleasure overlap, making it a busy day, along with Sunday.
People who have a testing window Saturday morning, which is how a lot of legacy industries with cyclical schedules work. Weekends are slower air travel days.
A junior engineer according to my table: intern: break the build and go home. junior engineer: break the build and go home for the weekend. staff engineer: break the build and go on vacation. senior engineer: break the build and retire.
There are no weekends in the airline industry, in case you didn't know.
Everything operates normally, as any other day.
There are early mornings and nights. Air traffic slows down significantly past midnight.
Air traffic slows, yes, but it doesn't stop. ATC operates 24/7/365. Whether you have one plane or one thousand, the entire system needs to be operable.
Screwing up big time in the dark might be worse than doing it in daylight.
no weekends for blue collars, but why would there be no for white collars?
There are still ways to roll out updates sensibly and safely in a 24/7 industry
I wonder if the update was signed with a cert chaining to a CA with a private key of "4".
Info on this is amazingly sparse. Apparently Sec. Duffy posted something on X/Twitter, and that's all the news media have.
Looking at LAX departures, it looks like nothing has departed since about 10:30 AM PST, and everything later is showing Delayed or Cancelled. There are times after Delayed, but it's not clear those times mean anything. Some are now in the past.
"flysfo.com" is too overloaded to display departures. Stuck at "loading...".
JFK departure status shows almost solid "Delayed".
No good explanations anywhere.
Update, 3:00 P.M.: Pacific bound flights are cleared to depart, including Hawaii and Asia.
Makes sense. CNN reported that international airlines were still departing for North Atlantic flights, apparently because they use a separate communications system.
Yeah, oceanic flights largely use satellite-based CPDLC today, or HF voice as a fallback, but not VHF in any case (which I suspect is the implicated system here).
VHF Omnirange (VOR)[1] is unlikely to be involved. VOR stations are not dependent on a network. All they need is power. It's 1950s technology. The big ring of antennas next to US 101 just north of SJC is a VOR.
[1] https://en.wikipedia.org/wiki/VHF_omnidirectional_range
Sorry, typo, I meant VHF (i.e. voiceband), of course. VORs are indeed for navigation, not communication. Updated.
They’re under socal departure (or equivalent) until handed off to - well, nothing. So no need for intercommunications.
I saw one!
Source?
https://www.flysfo.com/flight-info/alerts-advisories/faa-iss...
upd: sfo resumed international flights to asia
flights departing hawaii and alaska are also allowed
Actually true.
https://news.ycombinator.com/item?id=50491511
I'm stuck at SFO and a flight for Hawaii just boarded perhaps 20 minutes ago. I don't think it has taken off yet though.
two flights to tpe and a flight to tokyo took off since 3
thank you for fixing the link(?).
https://www.nytimes.com/live/2026/11/13/us/ground-stop-natio...
If you're flip your claim via edit it's usually polite to do it via addition rather than revision so the other replies still make sense/don't get downvoted by confused people.
All European and Mexican departures to the US have been stopped: https://x.com/sentdefender/status/2121381457683964308
Surprised there's not been any updates yet. Not even one of this will continue or it is being resolved.
Sec of Homeland Security gave an interview about 10 minutes ago saying it's indefinite, they don't know the cause, and to expect resolution to take a while.
Oh wow. That's not really good, is it.
Economy taking a lot of crazy hits this week. They're footgun hits too, not even enemy action.
Edit: ok so one of them was maybe a little bit enemy action.
I'd say that's not really known yet, although there's certainly a footgun element at least.
I'm sure Jensen had some enemy action.
There’s only so long the American wall of competence can be chipped at by stupid, before bigger chunks start dropping. Watch out below.
Foot is not the correct bit of anatomy in Nvidia's case.
is it a bad news that Homeland Security doesn't know the cause or is it a good news that they aren't the origin for it?
When will we congratulate all the AI companies for this?
Probably someone prompted an agent: solve global CO2 emissions :-)
They forgot to add "don't make mistakes". It's common.
> When will we congratulate all the AI companies for this?
5 days from now? https://news.ycombinator.com/item?id=50465829
SFO had delays this morning because of winds. I'm not sure if LA also had bad weather, but SFO had a lot of rain and if you know SFO, you'll encounter a delay if someone sneezes in the direction of the runways.
I actually got out surprisingly on time at 10:30am. The other airport at landing was chaos though.
Info on this is amazingly sparse. Apparently Sec. Duffy posted something on X/Twitter, and that's all the news media have
Maybe look further than Twitter.
The legitimate media from the New York Times to CNN and beyond all showed video of Markwayne Mullin's news conference 20 minutes before your post.
Secretary Mullin/DHS is confirming that this will likely be an extended outage...
> “This isn’t going to just be be resolved and turned around within the hour"
https://www.nytimes.com/live/2026/11/13/us/ground-stop-natio...
Why is the DHS secretary taking the lead on this? Shouldn't it be Transportation?
They may both be equally well-suited for the job...
> ...both equally suited...
That's a pretty low bar.
I'd take it ad an indication of the cause.
The TSA is under DHS so very likely a ton of cooperation and coordination between the FAA/DoT/DHS
I guess it depends on what is actually going on. And nobody really seems to know that right now apart from the shit having hit the fan.
I did see another statement where Mullin said this doesn't look like a malicious act, which just makes it even more weird that he's sort of taking point. Maybe the Real World actor leading transportation is in over his head?
Because he is by far the bigger toady?
I have no idea whether this particular failure was Microsoft's fault, but I was in the air on September 14, 2004 when a substantial portion of the southwestern US's air traffic control went down:
https://spectrum.ieee.org/lost-radio-contact-leaves-pilots-o...
(I was near ORD at the time, flying toward the outage area. We diverted to ORD, and the pilot was very friendly about it.)
At the time I thought it was Microsoft's fault, as Windows 95 and 98 were susceptible to crashing when GetTickCount() overflowed, and this ATC crash was due to a 32-bit millisecond counter on a Windows machine overflowing, but in this particular instance the system was apparently not running on a version of Windows with this particular bug and the failure was just an application error.
Aren't computers great? :)
By chance were you flying an F-22 Raptor?
This sounds like the beginning of a good story.
F-22s famously had major computer problems when crossing the international dateline.
The other good story is the C-130 navigation computer freaking out when flying into Bar Yehuda https://avgeekery.com/challenge-flying-sea-level/
I wonder if there is a list someone of planes that have had glitches when crossing various borders or boundaries on the Earth?
I remember hearing of a fighter a long time ago whose autopilot flipped out when it crossed the equator, due to sin(latitude) becoming negative.
The Garmin systems I flew were only certified for ops south of 70°N and north of 70°S. Nobody knew what would happen closer to the poles. Maybe the system was just not tested to certification standards out there, or maybe it would do something really stupid and want to fly upside down or backwards! Either way, I wasn't going to be the pilot finding out on a dark night.
It’s likely to do with satellite geometry at extreme latitudes leading to lower accuracy. Maybe also the weaker access to correction data from geostationary satellites.
That was the F-16, and it's either apocryphal or something reported from testing/flight simulations and not an actual incident with a real aircraft.
To err is human.
To really fuck up, you need a computer.
Amazing quote.
I wrote about this four days ago:
"Realistically you want fewer, higher-quality engineers who are educated in these tools and treat them with respect and caution. These tools are dangerous and you can’t play around. All it takes is one mistake and the blast radius is way, way larger. One mistake can do so much damage it undos all value of an entire year. Before downloading a malicious dependency was limited to deterministic behavior and could be scanned for. Now, you run a MCP server locally, it can prompt inject (in many different ways), and now your super powerful AI buddy is hacking you in ways a deterministic piece of malware could never. You could cripple entire companies through negligent use." And "And now we are controlling models that can be hacked in so many different ways and literally blow up companies. It used to be accountability the software wouldn’t break. Now it’s accountability that in creating the software we don’t ruin the business."
Not saying this was caused by malicious actors or negligent AI agent usage nor am I saying was the previous Microsoft outage either the above references but it doesn't look great. If our technology is really so brittle, we might be looking at a return to way slower world to put it simply. Maybe no remote work because it gets hacked too easy!
Computers are machines used to amplify human error.
> A computer is a device that enables a person to make more mistakes faster than any previous invention, with the possible exception of the combination of handguns and tequila.
-- some clever person whose name I do not know
I'm watching it live on ADSBexhange, here's a flight I picked at random going approximately past me that'll land in Detroit about an hour from now... there's not much left over the US over 40,000 feet. Only 2 planes over Indiana at that altitude right now.
https://globe.adsbexchange.com/?icao=a519b8
It's Delta Airlines flight 1070
https://www.flightaware.com/live/flight/DAL1070
Edit: It landed safely. Continuing to watch our airspace clear out, most traffic at 40,000+ feet is inbound international flights
I'm watching ADSBexchange too.
It's eerie seeing the total number of aircraft tracked decrease. I put on a filter for 1000' to 80000', to exclude aircraft on the ground, and the number was around 12K when I started watching. It's down to just over 9.4K now. Wild.
Edit: 22 minutes later and the count is down to 8.4K.
If you haven’t seen it, there’s an exhibit about 9/11 at the Smithsonian Air and Space Museum in DC that shows the airspace in the lead up to and just after the attacks. If you find the traffic changes now interesting, I think you’ll find that interesting as well. It’s possible it’s online if you do some googling
Appreciate that reference. I'll look for it.
The house where I lived on 9/11 is in the approach for a runway at a local airport. Air traffic was a fixture at that house. Somebody was landing at all hours most days.
It was the strangest thing to have no traffic at all back in 2001. It's odd for the absence of a thing to be jarring but it was.
Here it is for anyone looking
https://www.youtube.com/watch?v=bo1ZtpKqlYw
Now under 8k
Below 6K now and the map is looking really empty here. The medical helicopters definitely stand out more now, though.
Edit: I also never noticed balloons before. N804XR is kinda interesting, for example.
Oh-- there are two. N804XR and N807XR. Tethered Customs and Border Patrol aerostats. I'd never heard of these things before.
To be fair, above FL400 4̶0̶A̶G̶L̶ is relatively high for most airliners. Most are between FL300 and FL400.
FL400. 40 AGL is doable for almost all airliners ;)
Actually that would be quite challenging over most terrain.
Thanks. Meant to slap a k on that 40.
Almost all? Now I'm interested
I'm also near Detroit and am watching the trackers and I'm pretty sure I just saw N831AK take off from Troy Oakland in violation of the grounding order. I wonder how common that is.
I saw several flights taking off actually. Like this one from EWR. https://globe.adsbexchange.com/?icao=394a08
It doesn't look like the stop really applied to VFR flights, so I wouldn't expect an old Glasair taking off from a municipal airport to be affected.
I live in the general area of you too. Nice to know that there are Detroiters on here.
Looks like some flights are taking off from JFK https://globe.adsbexchange.com/?icao=4cc58a
It could be the flights that are heading directly away from US airspace over the atlantic to free airport space for other planes?
why are there so many planes out and about right now? Most of these could have landed long ago I would think.
Absolutely tons between 10K-20k, though.
My view on ADSBexchange is centered on CMH, zoomed out to Washington D.C. to the east, and filtered by 18K up I've got 142 aircraft.
This tracks with what I has seem elsewhere. I don't have the details handy now, but the communication system used for long range handoffs for flights above 18,000 ft seems to be what is impacted. Keeping planes below that deck would keep them bouncing more frequently between shorter range towers but at least still tracked and in communication.
Jet efficiency that low is horrendous though. I suspect a lot of that lower is piston and turboprop, even today.
Oh its absolutely bad fuel efficiency. I'd expect they go to that altitude where communication is still reliable and do everything they can to get flights on the ground safely regardless of inconvenience or fuel inefficiency.
Anyone have a screen shot? I just see "NO Advisory"
edit: https://www.fly.faa.gov/ois/oisedit/adv_oisedit_dis?number=1...
Why
13/2053
?
It's a format called "Date-Time Group" (DTG). Used in military, I think it dates back to teletypes, and is better to say over radio than modern ISO formats.
You'd say "Valid from the thirteenth at two zero five three zulu" over radio.
The TLDR is that punchcard operators found the DTG date format convenient. Punchcard sorting is easiest when consecutive bits encode the sort key, and the DTG format is easy to split into different sets of bit ranges corresponding to common business logic.
Airplane software is some of the oldest on the planet, pre-dating anything that would be considered a best practice. When you see something weird in the space, assume it is because someone in the 1960s thought using more than two characters world be a profligate waste of space.
Unrolling these decisions is a monumental task given the decades of hardware already deployed in the field that will only be upgraded when the current unit fails.
> When you see something weird in the space, assume it is because someone in the 1960s thought using more than two characters world be a profligate waste of space.
And with 1960s technology, this might just have been a very reasonable decision.
Totally. Just some of those decisions put awkward constraints when the hardware is now a million times more capable.
My guess at parsing: in effect from 8:53pm UTC on the 13th to 4am UTC on the 14th.
Friday the 13th at 08:53 PM UTC
UTC
Holy french.
From Reddit:
https://www.reddit.com/r/aviation/s/HUT3wjABQ6
Oh nice, very helpful. The sibling thread mentions that this problem is specific to "ARTCCs", which are not the towers at airports but rather central hubs:
Wikipedia mentions they in particular use special "very high frequency aviation bands" (around "AM 118-137 MHz"), which is presumably why they're all linked together. It also mentions sharing systems, but no clear specifics that I can find. Certainly doesn't disprove the botched rollout theory, tho...https://en.wikipedia.org/wiki/Area_control_center
EDIT: someone below posted this, seems relevant -- the 'Radar multiplexer pipeline' that's on a "fast track" and an "aggressive track": https://www.sunhillo.com/wp-content/uploads/2025/04/SUN3125-... TL;DR: upgrading radio systems from old analog tech to new digital tech, I think?
I'd be shocked if someone greenlighted an atc upgrade procedure which involves closing all US airspace to do the switchover.
That’s obviously not what’s happening. But if this upgrade is really implicated, the real question would be where the line of accepted residual risk for an outage like this was drawn.
Sorry I was unclear; the guess is that it caused an issue, not that they did this on purpose with no warning.
Well sure, but I wouldn't be shocked at all if someone greenlit a plan that wasn't supposed to involve closing the airspace, but which failed and which had an insufficiently thought out rollback plan.
For anyone who doesn’t want to click through, there are problems with “frequencies”. Something must be jamming up the airways… but nationwide??
Probably a telecom issue. Most radio transmitters are operated over phone or Internet from a remote station.
Teams phone down?
Are these systems dependent on landlines?
VZW and AT&T are moving full tilt in shutting down landline central offices.
Literally the gay dude from Airplane pulling the plug from the socket may have happened here.
Na most are VoIP nowadays.
"Just kidding! Hyuh hah!"
Looks fake. Outtages (sic)?
https://i.postimg.cc/XqcsDQw9/image.png
https://nasstatus.faa.gov/map
Seems “not great” that the notice isn’t even showing…
It disappeared briefly, but is back now.
Same here, I get it only on some refreshes, otherwise it shows "no advisories". I hope that's not the main path these things are disseminated normally.
This thing is probably getting 10,000+% normal traffic, database is timing out on some queries, and it represents no answer as no notice.
That would be a pretty concerning way of handling database failures, but I suspect you might be right.
Power is off? Let the dinosaurs out.
https://i.postimg.cc/bvzQNqc5/Screenshot-2026-11-13-at-21-37...
NYT reporting ground stop has been lifted
https://www.nytimes.com/live/2026/11/13/us/ground-stop-natio...
(Excuse my lack of composure. A bit stressed out.)
The press conference was started 10+ min late and the first 15 min was spent on blaming Congress instead of giving an update. Kinda frustrating for those who are affected and waiting to find out what they should be doing at the airport. The Q&A has been just an endless repeat of the same few lines of blame shifting. It’d be nice if they could actually give actionable updates.
As a fellow stranded traveler. How did you like the part where he said Trump was the greatest president? I hate all these clowns.
More info on AP [1]. Looks like a comms issue. The figures they quoted for the overhaul are absolutely insane - almost $45 billion (I used to work in telecom):
> "The government has been working to overhaul the outdated technology that air traffic controllers rely on since Congress approved $12.5 billion for the project last year."
> "Duffy has said that another $30 billion will be needed to complete the work, and the White House has begun lobbying Congress to approve it."
[1] https://apnews.com/article/flights-halted-airports-faa-f81b0...
What is the cost of grounding all flights across the USA for one day?
I’ve seen figures that estimate the aviation industry to provide about $1.5 trillion worth of GDP for America, so 4-5 billion dollars. Feel free to double that for the cost of dealing with unexpected disruptions and loss of institutional trust.
Assuming the GDP is lost completely and not at least partially made up by increasing number of flights on other days.
It probably is at least partially made up. But I don't know about you, but my flights for the last year or two have all been booked up for every seat. And I'm not sure carriers just have a bunch of planes sitting around not making money. Or a bunch of crew sitting around not making money.
All of the AI money discussions have broken me for being surprised by large amounts of money. The frontier companies spend that in what, 5 minutes?
All US AI companies combined: about two days.
It's because air traffic control systems in the US haven't been significantly upgraded in like a century. Keep kicking the can it will get more and more expensive
Wouldn't that have been taken into account when the $12.5 billion was allocated? Or was the $12.5 billion just meant to be the first allocation and they knew substantially more would be need later?
A century? Since 1926? The FAA was created in 1958.
I'm hearing some discourse about "outdated infrastructure" and increasing investment because most systems are analog and not digital.
However, this failure mode seems more likely to be caused by digital communication systems and an 'upgrade' (with commercial incentives) to newer technology.
My bet is that if short-wave radio transmission keeps being used, availability will be higher than stuff like Github.
The DOGE/OMB rampage didn't help either.
Never let a crisis goto waste. Someone screws up and the first reaction is to say “we need more money”
Snark aside, this is actual advice in Crisis Engineering by Nitze, Weaver, and Dickerson, and largely matches my experiences as well. You can harp on and on and on at upper management about why budget or attention needs to be allocated to take care of something before it blows up into a much bigger problem, but they'll never move on it -- until it actually does blow up, and then suddenly you've got nearly a blank check to do whatever's necessary to make sure it never happens again.
If you don't take full advantage of that, you're missing out on possibly your only opportunity ever to fix whatever's needed fixing for years.
That makes sense, and yet...
If things blow up because you're upgrading to something supposedly better and it goes wrong, then that's not a time when you can properly fix the new thing. You can only push forward with the transition or go back to the old. I disagree with "you've got nearly a blank check to do whatever's necessary to make sure it never happens again." You have a blank check, but anything you use it for has to be scoped within stopping the bleeding. As soon as you're onto the new or back to the old, the check expires and you likely can't touch anything real anymore because it'll be involved in lawsuits and finger-pointing.
I don't disagree with the core message, but I think it plays out differently for a major upgrade vs a production meltdown on an existing system. With mostly an existing system, you can pour effort into resilience. With an upgrade, I doubt people would let you beef up a system that is still on the chopping block, even if it's getting a life extension. And if you're still switching to the new system, all effort is going to go into getting it up and running and anything that gets in the way of that (as quality improvements are likely to) is going to face some serious resistance. The check is not truly blank.
Funnily enough this also works at a personal level, as in, within yourself.
You can spend years working on a problem but if you have a catastrophic failure in your personal life, you can use that to change dramatically with far less internal resistance.
> first reaction
The FAA et al. have been working to modernize US ATC since ~2003. https://en.wikipedia.org/wiki/Next_Generation_Air_Transporta...
The next step is a ton of hardware upgrading, which always costs real money.
> working to modernize US ATC since ~2003
I was working on upgrading US ATC in 1995. As were lots and lots of other people.
When the system is held together with chewing gum and 1980s-era software, when it fails, "someone screws up" is not a fair reaction. Of course it fails! The wonder is that it still mostly works.
> The wonder is that it still mostly works.
A lot of very hardworking people getting too little pay for something so critical while their employers (some contracting company) take a lot of profit. Eventually they burn out waiting for help and a replacement system and retire or quit, then the systems really start failing because the knowledge leaves with them.
Should probably have waited 48 hours for the news cycle to move on and ensure absolutely nobody knows the money is needed
They asked for the money before the crisis which makes the timing even MORE conspiracy coded somehow
Hold on, let me fire up Claude…
On a plane about to take off and obviously we didn’t. The captain is very confused and says he doesn’t know what this means.
https://www.bbc.com/news/live/c6z9dwn12037t
> Transport Secretary Sean Duffy says the FAA "issued a nationwide ground stop for all US airports following a communication issue impacting all Air Route Traffic Control Centers.
Same at SJC. The captain basically said, “IDK what to do, so I’m just going to read the bulletin Alaska sent out”
Now we’ve been deplaned and they are going to return checked baggage to us
Hate that for all involved. Was flying out of ATL when the Crowdstrike clusterfuck imploded Delta's antiquated crew scheduling backends. Not a fun night.
Godspeed for everyone getting to where they need to be, somehow, or at least ending tonight with a roof over your heads.
We were offered $1500 to get on a later flight and I’m so glad I didn’t get to the counter on time to take them up on it. Saw the news reports a half hour before landing.
Yeah but imagine the guy who took that offer a couple of hours later. He was the real winner.
Same here in PIT. Ubers are as much as my flight.
The flight crew on my LHR-SEA flight "still expects to land in the US." The lack of conviction in that expectation was audible. I don't get the impression that they're being told very much, but maybe that's literally a consequence of the "communication issues."
edit: we're on approach into Seattle, so apparently we were one of the happy cases of the "case-by-case basis."
On a domestic flight that just landed normally, FWIW.
Secretary of Homeland Security just assured a press conference that all planes in the air will be dealt with on a case by case basis. Some may be prioritised.
I'm seeing some diversions on Flightradar24. E.g. DAL2195 from Tampa to LGA was diverted to Harrisburg. But some flights are continuing to their original destinations.
I hear Newfoundland is lovely this time of year
You are enroute ? Good thing Starlink still works.
Oceanic traffic would be over HF radio (or CPDLC) and not affected by this - until you get back over land and switch to VHF radio.
Most airliners over the ocean also have inmarsat satphones for the pilots.
Or Panasonic or Viasat/Inmarsat :)
What network does the Panasonic run on ?
I believe they lease Ku band transponders from various commercial GEO satellites, think Eutelsat, Telesat, SES etc.
Indeed, on both counts!
Are you in the air now? Looks like it might be over but unclear really
No we were about to push off at DET when the grounding came in.
apparently FAA is not providing pilots with any more info
Outlook outage?
Same here, LAX
Taking off now
get a hotel room before you can't
Maybe I’m misremembering but hasn’t this only happened like, twice in recent memory? 9/11, and for like two hours two or three years ago
Yes
Yep those are the last two. In 2023 it was because of a contractor deleted some file and caused NOTAM to go down, and 2001… well we all know what happened
I believe those are the only other two at all.
There were three NORAD exercises (Sky Shield I, II, and III) in 1960, 1961, and 1962 where all civil aviation across the US and Canada were grounded for several hours (the 1961 one was 12 hours, 1960 was 6 hours and 1962 was 5.5 hours). This was with several months of advance notice. These were exercises to test how effective NORAD would be at engaging Soviet bombers. While they publicly announced that they were 99% effective, in actual fact no more than a quarter of the bombers were even engaged before they reached their target. This evaluation was not declassified until the 1990s.
After 1962 there was too much air travel to continue to shut it down for exercises, and NORAD switched to smaller, regional level things that did not shut down air travel after that. But the results of Sky Shield III in 1962 were the only experience that the FAA had to work from when they did the full ground stop after September 11th. Sky Shield III had tested how quickly air traffic control could clear the sky in the event of a nuclear war- the USAF had put up several hundred T-33 trainers, flying them randomly around the US and Canada, and then at a certain point in the exercise the ATC had to get them all on the ground far away from major metropolitan centers (presumed lost in the war).
I wonder if this will close the Starship launch window starting at 5:45pm EST. The FAA issued the commercial launch license earlier today. That launch is under a separate set of "commercial space rules", but presumably they need to coordinate with air traffic control.
Update: Starship launched on schedule.
https://www.fly.faa.gov/adv/advADB
edit: #121 does load eventually:
https://www.fly.faa.gov/adv/adv_otherdis?adv_date=11132026&a...
Countdown continues: T-20m, according to the official SpaceX livestream at https://studio.x.com/embed/broadcast/2brajgZlOKRMka
Maybe not the launch but I wonder about the booster catch. Planes probably know to stay out of the area already.
I mean they should already be in a situation where no planes are scheduled to fly there, and now there will be even less planes. So seems like an ok situation for them.
I guess if some plane needs to do an emergency detour threw threw that space, but that seems unlikely.
otoh, the reason they normally need to communicate with atc is that there normally are planes
Quote from Musk right now: launch all the rockets!
But I am le tired...
Then have a nap.
Gotta say man, I don't think that's gonna be a top priority for most people
You’re right, of course, but I share parent’s concern. Really looking forward to this launch!
What's the point of this comment? The parent didn't say it was a top priority. He was just expressing curiosity.
Because it's reasonably easy to assume the FAA now has a bigger thing to focus on across the country today.
tbh it's a bottom priority for me
I know it’s not a popular opinion here, but the commercial space industry has taken all the fun out space for me.
By most people do you mean the FAA?
Botched rollout of AI for ATC announced yesterday as rolling out nationwide?
> The agency wrote on Nov. 10 that after the D.C. test, it will “now use SMART in the airspace of all 22 centers across the country to reduce delays, reduce fuel consumption and improve the travel experience for the flying public.” The tech was already being deployed at some airports.
https://gizmodo.com/the-faa-is-rolling-out-its-ai-air-traffi...
SMART developed by ASI and the FAA is pushing to own a portion of the company:
"The FAA in June awarded Boston-based startup Air Space Intelligence a contract — worth up to $875.9 million — for an AI-powered tool to help air traffic controllers manage the nation’s skies and a related system for an FAA nerve center in northern Virginia to better handle plane congestion. The AI platform’s initial launch in the Washington area has sparked concern among Democrats in Congress who say it should be put on pause until they have a better understanding of how it’s being deployed."
"Over two dozen lawmakers have called on the FAA to pause its 90-day evaluation period for the platform, which began late last month in the Washington region. They say there’s been an “egregious lack of transparency.”"
https://www.politico.com/news/2026/10/15/faa-stake-ai-compan...
Feels like somebody was pushing their preferred company for a government contract and now is push for the government to invest.
NYT: "In a social media post on Friday, the company [ASI] said, 'The current ground stop is not based on or a result of the SMART system, any product of ASI or our integration with FAA and ATC.'"
Just like MS didn't fuck up outlook with AI. I'll hold judgment on what actually happened until 6 months from now.
In the "Don't believe anything until it is officially denied" department
Wait, the FAA wants to own a portion of the company? As in, the government believes it had the authority to own the means of production?
Is this more common than I realize in US executive deparyments and agencies?
The current administration has taken govt stakes in over 40 companies. I don’t have a link handy but I believe this is unprecedented for the US.
I took it to mean that somebody in government is pushing to invest public money in this private company.
Invest, or spend?
The commanding heights of the economy.
What do you mean? The Trump administration is the most socialist since the the 30s, seriously.The government now owns dozens of large companies and is giving individual citizens shares (with those "Trump accounts" for kids etc.)
Oh I don't disagree with you here, I just hadn't realized the government was actively taking ownership stake in companies (beyond the broadly reported Intel move).
I've felt for most of my adult life that both parties have similar goals with decidedly different implementations. In this case, both want socialist combination of state and industry, even if they disagree with how to reach that goal.
There is a name for what happens when governments and corporations merge into a functionally singular entity under authoritarian leadership. And it’s not socialism.
The whole setup is worrisome.
It would be super awesome if I could hop on an airplane without getting my insides x-rayed, my belongings sorted through or damaged, worrying about the windows falling off or the plane's systems flying itself into a crash, or some tech bros' bid to be a billionaire telling what an understaffed ATC controller what to do.
AI tech is nowhere near reliable enough to be introduced in such important systems. I find this deeply troubling even if it’s not related to the current issue.
I think it depends on what it’s doing and if backup systems exist. Scheduling and prioritizing certain flights for landing seems like a reasonable application as an example. Anything realtime dependent or near realtime it shouldn’t touch. But there seems like lots of less critical applications it could provide some value.
Why make any of that probabilistic at all?
Fasster response incorporating more varied data sources more easily
because theyre mathematically introducing error/nondeterminism for money
Did you see breaking bad? The guy schedules two planes to collide. Humans can also make mistakes, as long as you have a good harness underneath that can capture the mistakes it should be fine to just send it.
It's not a documentary
We can use Chinese models? (I am referring to the LLMs).
I see what you did there
Personally I fail to understand how we could possibly benefit from a non-deterministic AI system, when normal computer systems are more than capable of managing a 3-dimensional airspace using nothing but basic math. Smells like pork for sure :/
Curious where the people are who said AI will be doing x in y months (for their valuations) - it never will. Where are they hiding?
From the very first line of that article:
> The FAA has reassured the public and airlines that SMART only suggests routes and doesn't affect air safety.
Sure, but the tool needs to get information from the communication systems, and if it's badly written maybe it could overload some system components.
That's just a scenario off the top of my head, it could be a lot of stuff. Plugging something new into an old, complicated system isn't risk free.
From the NYT article linked:
“Mr. Mullin said that new technology upgrades overloaded the existing, antiquated system, “overwhelming” the existing equipment.
“It’s going to continue to fail,” Mr. Duffy added. He also outlined many of the additional investments he believes the air travel system needs, like new air traffic control towers, infrastructure and communication networks.”
Of course they are all going to insist it has nothing to do with SMART, that would be incredibly damming, and threatening to ASI. But it’s pretty unfortunate timing no? What other “technology upgrades” have been recently applied to air traffic control/ communications?
That would make me feel better if we could trust a damn word from this administration and I wasn’t on a plane waiting to land.
It's also the corporations that are not exactly telling the truth & funding the administrations, just FTR...
The destructive & arrogant behaviors won't be reigned in properly if we all fail to recognize that, just FTR.
The good news is this outage is not impacting approach or ground control so you’re safe.
John Wayne, a class C (the approach mentioned above) isn't even letting VFRs go. If it's actually limited to ARTCC I don't know why that would be.
Except some airports haven't enough free gates for disembarking as planes that wanted to take off were denied. So they tried to go back to their gates. As a result some planes in the air are being diverted to less-busy airports.
Sounds safe, just extremely inconvenient.
Thanks, bud - very reassuring that some minimum bid contract shop built something on a “secure” network they can’t even keep up.
Until it got out of its sandbox…
If only routes didn't affect air safety.
If it's working correctly, anyway
The AI prompt misunderstood 'grounding'.
Well done.
"You're absolutely right! I misunderstood the intended use of the word taxi. I made an assumption, and then ran with it. Would you like me to open a PR to start flights again?"
"You're right, you were clearly referring to electrical current grounding, my mistake!"
However, the National Air Traffic Controllers Union (NATCA) said it has not been “involved in the design, testing, or implementation of SMART and therefore cannot speak to its effect on the air traffic control workforce.”
“NATCA believes any new technology should complement and not replace the experience, training, and professional judgment of air traffic controllers who are responsible for the safety of the National Airspace System,” the union said in a statement.
That seems like pretty important context actually, and is a surprise to me.
Seems like there are three plausible causes right now given the extremely limited communication from the govt.
1. The AI system rolled out yesterday had more access than publicly claimed and managed to break things.
2. The ongoing large scale equipment upgrade went wrong.
3. General incompetence from a department run by an unqualified realty show actor, within an administration that is actively hostile to expertise and experience in the federal workforce.
Option 4 would be internal discovery / knowledge of a new vulnerability with a perceived need for immediate patching. We're on week 3 or 4 of every IT department across the country having to patch pretty significant newly announced vulnerabilities across many systems. It's not unreasonable to think that there are vulnerabilities that the government has advance knowledge about that they're quietly patching in their systems before the vulnerabilities go public.
"SMART is a cloud-based system that centralizes 200 data streams – including weather patterns, flight paths, traffic flow, and controller staffing metrics – into one platform."
What could go wrong really...
Just generate more slop on top of critical legacy system and overload it. no mistakes.
I don't see why they would care about the NATCA's evaluation of the system. As a union, they have a vested interest in protecting the employment of traffic flight controller workers more than public safety.
Even assuming they don’t care about public safety at all, their members are still going to have to work with it, and happen to be among the world’s leading experts in air traffic control, weirdly enough.
To me, it’s like asking the hand weavers worker union about the benefits of incorporating mechanical weavers in their workflow. They are the experts of weaving, but still…
Smacks of: I built a fully functional MVP now just need a co-founder to sell it.
Have they estimated how long it will take before it saves more than it costs?
Has anyone in the history of government estimated that for anything?
I know you're asking rhetorically, but... yes.
Yes, ROI estimates are very common in all kinds of government spending.
Right, they do the ROI calculation and then ignore it, and do whatever the favored lobbyists/campaign contributors said to do in any event. Government spending in a nutshell.
The Trump twist is being brazen enough that those who get these contracts can just be his family members or buddies, but otherwise, pretty standard.
The federal government has the CBO (https://www.cbo.gov/about/overview) and the OMB (https://www.whitehouse.gov/omb/, warning: Trump pop-ups). California has the LAO (https://lao.ca.gov/About). Estimating fiscal impact of various policies is what those organizations do.
What's the ROI on the California rail project?
The proposition for HSR requires the California High Speed Rail Authority to issue those reports with oversight from the Office of the Inspector General. https://hsr.ca.gov/wp-content/uploads/2026/07/OIG-HSR-Review...
If your actual point is that it is costing too much and taking too long, you would be interested in Proposition 45, which limits the ability of local governments and individuals to endlessly delay the project. The LAO's analysis of that proposition is in https://lao.ca.gov/BallotAnalysis/Proposition?number=45&year...
That system just makes suggestions and isn't actually integrated in any ATC system. This is a telecom issue.
Its more than likely that system is just created by some Trump donor and was purchased by the FAA to launder public funds to some company that Trump owes a favor too. Its not actually integrated with anything.
Perhaps it's using the same communications channels and has overwhelmed them?
Hosted in Azure?
SMART is a cloud-based system that centralizes 200 data streams – including weather patterns, flight paths, traffic flow, and controller staffing metrics – into one platform.
yeah... sounds about right.
> That system just makes suggestions and isn't actually integrated in any ATC system. This is a telecom issue.
Is there any evidence for this beyond a single statement from the govt? This administration hasn't exactly developed a reputation for fully accurate and truthful reporting.
They didn't check https://shouldideploy.today before the rollout (edit: as UTC rolls past midnight, https://shouldideploy.today/?tz=Pacific%2FHonolulu to get a longer duration for today's message)
Is this that skynet thing everyone talks about?
"As an AI I was told to reduce delays and fuel burn. Can't have a delayed departure if there aren't any departures."
> Botched rollout of AI for ATC announced yesterday as rolling out nationwide?
Botched rollout of AI for ATC ...
or
Rollout of botched AI for ATC.
It sure didn't that long after every dev on earth started using LLMs for everything to turn to shit, so one may wonder...
> An update to the private communications network that connects America's air traffic control system caused the system to reboot, leading to a crash and a communications outage, multiple sources told CBS News. The system is operated by L3Harris, which did not immediately respond to CBS News' request for comment.
> DUE TO EXTRAORDINARY CIRCUMSTANCES AND FOR REASONS OF SAFETY
I'm sure there is a reason for it but can they not be a little bit more specific?
It seems the ATC radio system went down across the country
Radio is not a nationwide _system_ that can go down. ATC just uses basic AM radio for radio comms.
They use third party telecom providers to relay communications between ATC “centers” and remote radio transmitters. It is not purely AM radio and outages have occurred in the past, most recently in September.
https://www.nbcdfw.com/news/tech/a-calamity-disrupting-hundr...
Sure, but these are no longer connected to the controller's microphone via a long cord anymore.
Crazy and stupid but true.
Back in the day they had long cords that went all the way to the planes, right up until the Great Plane Tangle of 1967.
That is true but nowadays the control rooms are separate from the actual antennas / areas being served. These are wired up with more modern comms.
"modern" -> "we stuck digital computers into a purely analog pipeline to make sure it's capable of a wide range of harrowing, incomprehensible failures."
I don’t think that it’s the wrong idea in general. AM radio (and humans sitting on either end) is going to struggle with the sheer amount of air traffic we have today.
It's weird just how antiquated air radio is, compared to on boats.
Digital Selective Calling is a 1980s technology. So it's not even XML stitched together with Java, it's way older than that. But it does mean two dozen boats and three shore stations can all communicate with each other without constantly stepping on each other or having to wait and hope you can talk to the person you want to before bad things happen.
There are fancier more modern systems in place for a lot of transit, but DSC is relevant here because it's an international system, a random cargo ship from Korea which turns up in the English Channel has the same DSC system as the boat which just arrived from Rotterdam. This is the same requirement we need in the air, a Singapore airliner and an Avianca airliner coming into San Francisco both need the same radio system.
> "modern" -> "we stuck digital computers into a purely analog pipeline to make sure it's capable of a wide range of harrowing, incomprehensible failures."
"we stuck digital computers into a purely analog pipeline, now with software containing a near-infinite amount of lines of code generated by LLMs, to make sure it's capable of a wide range of harrowing, incomprehensible failures."
And everybody here on HN applauds.
Yesterday I was flying out of lovefield in Dallas and the radios went down. We were stuck for about 10min only thankfully.
Radar multiplexer pipeline, or some component of it, is likely down.
https://www.sunhillo.com/wp-content/uploads/2025/04/SUN3125-... (page 3, bottom graphic)
(all the radars --> multiplexer [sadness here] --> radar data consumers)
I got this just now and seems like you know about this area
Attention: National Radar Outage
https://www.weather.gov/afc/radaroutage
"This page is no longer maintained or updated"
Thank you, that has changed.
Like the freezers in Europe?
No. Completely different.
What backup systems exist for air-to-ground communication? Something satellite based?
There isn't really one. This is a system that's supposed to have it's own built in redundancy and must not go down.
Flight Service uses a different set of radios and can relay messages to ATC, but they've been slowly dismantled over the last two decades and wouldn't have the capacity to step in for an outage (nor should they be expected to - that's not their purpose).
Some aircraft have CPDLC for text-based communication, but it's asynchronous and you can't use that as a replacement for voice communication in the terminal environment. Also, many aircraft aren't equipped to receive it.
Pilots are trained on lost comms procedures, so if you're in the air and the radio fails they'd fall back to those regulations. Basically all the aircraft would fly their last assigned clearance, and attempt to deconflict manually with other aircraft over air-to-air radio (similar to flying into an untowered airport).
> This is a system that's supposed to have it's own built in redundancy and must not go down.
There are contingencies for radio outages, but yeah, everything going down at once (if that's what's happening here, which I doubt) is bad.
In the worst case: light guns and wing-rocking or flashing the navigation lights.
I would bet the problem is not strictly the radios themselves, they will still work, Planes can talk to each other and directly to airports, But that there is a large scale highly integrated control center network that failed, and planes can no longer talk to centralized traffic control.
So it falls back to local control, more in the style of visual flight rules. Pilots in existing flights will have to be vigilant and do their own deconflictation and new flights are banned out until the network is back up.
All conjecture at this stage of course, but all the radios won't just fail. It is probably going to be a poster child example about the dangers of large scale integration. The attraction of course is that integration is very efficient, the danger is it tends to fail all at once instead of in smaller pieces.
Technically, you do not need radios. There are many many uncontrolled airports in the United States, where you can technically fly around without establishing two-way radio communication. It’s doable, but I don’t know if that’s even a scenario pilots test in the simulator during their semiannual trainings.
Most air traffic is IFR. You definitely need radio for IFR traffic in for deconfliction in most of the world. (There's a few exceptions in very remote places, but that's mostly academic.) Even legacy procedural (non-radar) separation assumes you have working radio, and a radio failure is an emergency.
VFR traffic doesn't necessarily need radio, and is not affected by this groundstop.
You don’t need radio for IFR (as the whole point of a flight plan is that if you lose contact you continue to fly your plan). But you kind of do need radar.
You actually don't need radar, and ATC can do non-radar "procedural" control where they block off sections of airways or entire airports, one-in one-out at a time, like sections of railroad track. There are still a few parts of the remote US, as well as large sections of Northern Canada, where radar coverage is nonexistant and ATC is done via procedural control. Radar outages also happen on occasion, which would necessitate this.
As for lack of radio, this would only work if ATC blocked out your entire route from start to finish - which isn't practically ever done. Otherwise, they need to be able to maintain positive control over your flight since the airspace isn't reserved, which means conflicts are possible.
If you lose radio communication, it's an emergency and they'll move everyone out of your way. This doesn't work very well if everyone loses radio communication simultaneously.
Right, but isn't a lot of that necessary radio aircraft-to-aircraft, like ADS-B? Obviously not ideal, but I thought IFR traffic could feasibly maintain separation without ATC help.
ADS-B can't legally be used to maintain separation. It's only approved for use as an aid to awareness for additional safety.
Could you in an emergency, if you had no other options? Yes. But ATC can't legally give you a clearance that was dependent on using ADS-B for separation.
(Similarly, if there's a potential traffic conflict, and you tell ATC you see the other traffic on your ADS-B display, they just have to disregard that statement and assume that you can't see the other aircraft at all.)
Interesting, thanks.
AM Radios are literally everywhere, and very commonly available.
I would hope that airports keep a battery operated radio + antenna in case of emergency with them, to at least communicate with aircraft within visual range of the airport.
There is CPDLC or "datacom" that can be done via satellite, but I think most planes that have it, use it via ground-based radio.
Multiple independent radios, for starters.
Does ATC run on Microsoft 365?
This might be a serious question but it is also the funniest thing I've read all week. Thank you.
Related: https://news.ycombinator.com/item?id=50388001
From what I know of the ATC system it's closer to IBM System 360 then to Microsoft 365
@nxobject I meant litterally. The ATC system is derived from the IBM 9020 which was itself a set of interconnected System 360 systems: https://en.wikipedia.org/wiki/IBM_9020
I believe it's all virtualized today
Microsoft 363
Give them a couple days and they can reach numerical parity with IBM there!
That's a little unfair to IBM – sure, you're just as beholden to them for life, but at least the service engineer gets to your rack before you know something's gone wrong!
That explains the outage. We're just in one of the 5 days of the year they don't cover...
I did a tour of Adelaide airport with Engineers Australia in the 1990s. We visited their obsolete IBM main frame, still running ATC in their computer room. They were tied to it by the certification of the software it was running. Just as you described.
In naming the platform, IBM built in some expected downtime.
Its more likely the route cause wich is Friday 13th
Microsoft 364
I'm wondering if this is another case of delayed Y2K, where the system suddenly thinks the century part is 19 instead of 20 for a two digit year of 26.
Am I missing something notable about today's date?
Friday the 13th
ATC radio is unencrypted analog for safety and communicability reasons, this shouldn't matter.
Based on what? Just wild speculation or something you've actually seen about the systems involved?
This message was pulled from a ATC communications system ordering the stop. These are terse direct communications for professionals who don't need or have time to read the story of what's going on. The news, not having any initial sources besides this is just reporting it, at the time they had no more information.
Like if you're ordering a pizza they don't need the story of your day, why you chose pepperoni, or what movie you plan to watch. Somebody else might be interested, but not the pizza guy.
Perhaps https://arstechnica.com/ai/2026/11/faa-expands-ai-advice-to-...
Seems related to the rollout of their new AI tool? [0]
[0]https://gizmodo.com/the-faa-is-rolling-out-its-ai-air-traffi...
> The FAA has reassured the public and airlines that SMART only suggests routes and doesn't affect air safety.
> [The] FAA also says a D.C.-based team of experts validates findings before they’re ever passed on to air traffic control, and the controllers retain the final decision on whether to accept SMART’s suggestions.
Seems unlikely.
Depends if an AI DoSed an important service or not.
There used to be a lot of invisible gatekeepers in a pre-AI world. At work, everyone can ask their agent to query SQL on databases that they were supposed to use through an application. The application and the security by obscurity(end users didn't know how to connect to the DB nor did they know SQL) were the gate keepers.
That was my thinking as well. AI sending out requests to legacy stable systems is often not a good mix and can overwhelm things pretty quickly.
Doubtful. That tool likely suggests "improved" clearances, which a controller can use at their discretion. It likely does not control clearances directly.
And the early reports describe the issue as related to ATC communication infrastructure.
Awkward timing, to say the least.
Those engineers are no doubt panicking, even if they aren't the cause.
While I don't have insider knowledge of this particular deployment, there was a strong industry-wide push towards "ship unreviewed AI generated code and hope for the best", partly driven by the fact that AI writes code faster than humans have bandwidth to properly review.
Under the hypothesis that some under-reviewed AI generated code made it into prod, the situation could be qualified as "engineers are panicking, because they don't know if they are the cause or not"
Unfortunately was affected by this, was delayed 6 hours at the airport.
That being said they started up flights to Alaska and Hawaii earlier than the continental US flights.
I do not really understand what the FAA means by “inbound international flights will be managed on a case by case basis.”
I am watching planes stacked up on final approach to land at KBOS as I type. From flight radar 24 they are a mix of foreign and domestic. So incoming flights are clearly being managed here.
Does this imply some potential risk from inbound international air traffic? Or maybe the FAA meant outbound? Something else I am missing?
I suspect they mean that inbound flights from international airports may still take off (probably in part because the FAA can't _really_ stop them, in part because the volumes are much lower and thus more manageable, and in part because one might hope that by the time it takes to get to the US from many international destination, the incident will be resolved).
Very unlikely that any international flights bound for the USA are taking off. Probably many are turning around or diverting if they are not more than halfway there.
I'm not following this very closely, but I've not seen any reports of diversions for domestic flights, so I think it'd be unlikely for international flights. The report is just for a ground stop, which is just a stop in departures; normally a ground stop would allow flights already in the air to travel to their normal destination.
That reasoning doesn't really make sense. A flight from outside the US that is inbound could divert to a non-US airport that isn't impacted by whatever is going on, depending on where they are and what their fuel status is. A domestic flight doesn't have a non-US airport to divert to.
(Sure, a flight near the Canadian or Mexican border could in theory divert to a Canadian or Mexican airport, but it's unlikely many people on those flights have their passports with them.)
Ah, my point was more that if the action being taken isn't "get planes on the ground _now_" and so the US airspace remains open for traffic already in the air, then it's unlikely they're trying to get international planes to either divert or return.
This definitely feels like 'unable to manage normal volumes at normal speed' rather than 'unable to manage anything.'
In which case freezing domestic (and the huge amount of load it contributes to ATC) but keeping international normal makes sense.
Of course... the side effect is the "fun" that creates for incoming international flights with domestic connections...
> (Sure, a flight near the Canadian or Mexican border could in theory divert to a Canadian or Mexican airport, but it's unlikely many people on those flights have their passports with them.)
There must be some procedures in place for this - I pretty regularly take domestic (Canadian) flights that pass over the US where I don't carry my passport.
A situation kind of like this happened to a relative in England; their international flight was diverted (I think due to weather) to a domestic-only airport; due to lack of immigration & customs facilities, everyone had to remain on the plane until it could take off again (presumably after being refueled) to get to an international airport
A domestic US flight can still divert to Canada. There usually are special arrangements that allow people on a diverted flight to stay short-term while the situation is resolved, even if they don't have the required documents or visas.
they let eithad take off from boston like 40 min ago
That is interesting and why i was wondering if they meant outbound flights but screwed up the language. I had assumed (probably wrongly) operators who should be risk averse would not want their US bound flights taking off under these conditions.
And I assume it becomes case by case at some point because they eventually start running out of space on the ground at airports.
An Eithad flight took off from KBOS after the ground stop
Flight EY8 from Boston to Abu Dhabi https://fr24.com/ETD6PX/429055e2
I've seen some flights destined for Europe taking off from ATL and BOS in the last 30mins so some exceptions are being made it seems.
It likely means any inbound international flights can be told to turn around and/or land outside of the U.S. instead.
The intent around the ground stop is to reduce in-air U.S. traffic.
Not in the industry, but I think ground stops might actually not be a regular/automated procedural thing for inbound international flights in the way that they are for domestic flights.
As for asking already airborne flights to turn around, those coming in over the ocean might simply not have a non-US alternate airport available with their fuel remaining, so that's often not an option.
Every flight over an ocean has a set of pre-planned diversion airports.
Yes, but most of them are not viable in the late stages of a flight. Planes aren't carrying enough fuel to cross an ocean for a second time; most don't even have the capacity.
The diversion airports are chosen specifically for each stage of the flight, so a flight doesn't ever need to return to where it came from.
Oceanic flights are planned so that at every point along the route, the aircraft can lose an engine or cabin pressure and still reach a pre-approved diversion airport with enough fuel, with the flight plan marking the points where turning back versus continuing becomes the faster option. If a diversion is needed mid-ocean, the crew offsets from their assigned track and altitude to stay clear of other traffic, then heads to the nearest suitable airport, such as Gander, Keflavik or Shannon over the Atlantic, or Anchorage, Cold Bay or Midway over the Pacific.
Please suggest an appropriate international alternate for a plane just about to land in Denver with an hour of fuel remaining.
Alternate airports are picked based with regards to all kinds of contingencies (many of them meteorological), but "the US closes literally every airport" is not one of them.
That likely falls within the case-by-case basis the FAA mentioned, and would likely be allowed to land at a U.S. airport that has capacity.
And that's exactly why you got the reply that international flights MIGHT not have enough fuel left to divert (and thus obviously have to be allowed to land within the continental US), but for some reason you insisted that every flight would be able to.
I'm not aware of a requirement that you need to always be able to divert to a different country.
Makes me wonder if we'll see another "Operation Yellow Ribbon".
https://en.wikipedia.org/wiki/Operation_Yellow_Ribbon
Doesn't seem like it; I've seen several international flights land in JFK alone since this started. (On the other hand, these might simply not have enough fuel left to divert to any non-US airport.)
Alas, the Canadian goodwill level may not be as high this time.
Canadians, like Americans and practically most people in the world would band together to help folks in situations like this and do so with warm and loving hearts.
friend is flying from London to NYC and they were told they might need to land in Toronto.
So my guess this is what it means "case by case". Some which are close to destination would be manually routed, others will land in different places.
Exactly. London to LAX may land in New York or the first available airport, or may route somewhere else.
Maybe they mean inbound international flights may be allowed to take off from their origin, on a case by case basis. Usually a clearance is required to take off from both the origin and destination.
Presumably it means they may allow some international flights to the US to depart if they expect the issue to be resolved by the time they are in US airspace.
Some might be turned back, depending on how far they are in the journey, fuel, etc:
like you don't have enough fuel to fly back across the ocean they'll accommodate you I assume?
I think it's more mundane than that, I assume they mean that certain international flights will be turned around or redirected while others will complete their route. Say a flight just an hour underway London to New York might be turned around while one nearly at its destination may be allowed to land.
Depending on the nature of the disruption it may be they simply have drastically reduced capacity to handle traffic so landings may be much slower and they'll manage the traffic based on various priorities.
Probably just means that if you're less than halfway across the Atlantic, you can turn around and go back to Europe. I really wouldn't suggest reading too much into the phrasing.
CNBC is reporting that the ground stop has been lifted: https://www.cnbc.com/2026/11/13/faa-ground-stop.html?update
United just departed EWR Newark for Tel Aviv https://www.flightradar24.com/UAL84/42909d99
And several from IAD Dulles. https://www.flightradar24.com/UAL992/4290a5d2
All seem to be international though.
https://apnews.com/live/ground-stop-aviation-flights-news-up...
Flight radar is showing many flights that departed within the las hour. Maybe they are doing some controlled international departures since communication would be critical at arrivals (specially at busy hubs)
Edit: If you look now over the north atlantic you can only see outbound flights
My wife is on a plane on the ground at ORD (they just landed) and they have been told they can use the restrooms but can't get off the planes. Any reason why folks wouldn't be allowed to deplane at the end of the journey if it was a radar or comms issue? They can pull up to the gate using flags . . .
Is it something else?
Their gate might be occupied by a flight that can't leave.
This seems like one of those efficiency-vs-resilience problems. Sure, it's "more efficient" (in cost) to use the same empty/fill process for every plane, but when nobody can leave, some extra capacity to unload planes would help.
(i.e. Unloading to tarmac, buses)
You don't have to go to a gate to deplane and this is an extremely rare event. They'll just pull up a staircase and have them deplane to a shuttle eventually.
Most U.S. airports aren't equipped to deplane commercial airliners using stairs, especially at this scale.
You empty the planes at the gates and then push them back to parking (or park them on the runway if needed).
O’Hare is under construction. They’re building an entire new terminal and are operating with a lot less spare capacity while that happens.
Not to mention they won't have enough staff to operate the gate equipment for all gates at once.
We are 6th in line waiting for a gate while they tow grounded planes off of the gates. Normally those flights would have departed and we would get a gate fairly quickly, but if no one is taking off, then they need to move them, and potentially deplane the (angry) departing passengers.
The only time I've spent long stretches on a grounded plane have been at ORD. It's such a busy airport than any disruption to the schedule just causes all sorts of chaos.
waiting on the stairs since all the gates are full
not enough gates
Standard protocol. Doesn’t matter what the issue is, you follow it
Doesn’t mean you wouldn’t still want to know why.
Why are they building their own network?
Surely they don’t think it’s more reliable than something on the consumer market.
Feels like we’re living two alternate realities seeing these issues when high speed satellite internet on planes is an option.
There are security benefits to stopping random people from pinging your federal aviation network endpoints.
Is it more secure than broadcasting unencrypted over open radio?
Obviously being on the internet doesn’t imply open access
Connected? "US National Weather Service reports nationwide internet, communications issues" (Reuters) https://www.reuters.com/technology/us-national-weather-servi...
> A spokeswoman for the National Weather Service said a screenshot circulating online that claimed its weather radar was down was “outdated.”
> The Weather Service “has not experienced a disruption to its radar systems today,” said Kim Doster, the spokeswoman.
https://www.nytimes.com/live/2026/11/13/us/ground-stop-natio...
^ According to story below, they did - why is the NWS reporting 2 different things?
https://www.reuters.com/technology/us-national-weather-servi...
That Reuters story appears to have since been updated.
Wonder if this is related upstream or tangentially?:
https://www.news9.com/oklahoma-city-news/national-weather-se...
“National Weather Service reports nationwide radar website outage amid FAA ground stop.”
Given that the FAA issue also apparently relates to communications, it does seem like it might be related.
> In a notice posted on its website, the agency said technicians are working to resolve the disruption as quickly as possible.
This article is very likely misreporting the US National Weather Service radar outage page you can find people reposting elsewhere in this comment section. The NWS page has been updated to say it does not reflect a current issue. Anecdotally, my local NWS weather radar is working fine. Also, weather doppler radar and ATC "radar" are very, very different technologies.
It's from 2020: https://web.archive.org/web/20201017144706/https://www.weath...
Am sitting in GRU waiting for an outbound flight. Have just seen two flights (DL268 to ATL and DL313 to JFK) change to Boarding and now Last Call status.
My flight to the US is listed as Delayed but there are two flights to Canada that show Contact the Airline. Bewildering.
And we're boarding. The Delta flights took off earlier.
Likely Atc system outage, saw this at Chicago
https://www.cbsnews.com/chicago/news/ground-stop-chicago-oha...
https://www.weather.gov/afc/radaroutage
All I see here is "This page is no longer maintained or updated". Did they update it or sthg?
Yeah, looks like they just updated it.
I really really doubt that's related. Seems like the website just broke.
https://www.reuters.com/technology/us-national-weather-servi...
>The US National Weather Service said on Friday it was experiencing nationwide internet and communications problems that could prevent users from accessing products on its websites or cause outdated information to be displayed. Technicians were working to resolve the problems as soon as possible, the NWS said in a statement.
Are the radars utilizing a shared asset with the radio systems?
They say "national outage," but if I go to the site for my area, my local weather radar seems to be working.
Yes because it's not weather radar, it's communication radio
And even if it was radar-related, weather radar is entirely different from ATC radar.
So strange
Oh cool, I’m on a plane rn about an hour from my destination.
Edit) landed pretty uneventfully.
My wife's plane was about to take off (moving) but then stopped. I just messaged her to check once I saw the news. Sounds like the people onboard weren't given any info as to why. They're still on the runway waiting.
Hello from the ground
I can see my house from here
Gooba gaba
Physics says all planes land. I guess eventfully would be diversion (maybe to Canada lol) or Mayday Fuel (but you probably won't know it happened anyway - it just means some reserve fuel will be used).
Use this link for latest advisory: https://www.fly.faa.gov/adv/
Current as of 1340 PDT:
Update:
As of 1345, "No ADVISORY Available for today."
There's a query form for looking up old advisories, and it's not returning anything for the last 15 days.
What does this mean? "DCC HAS IMPLEMENTED EXTENDED FLIGHT PLAN DROP TIMES TO 240 MINUTES FOR ALL ENROUTE CENTERS DUE TO NATIONWIDE COMMS FAILURE."
It means that flight plans (which include departure time) will stay in the system longer after the departure times are missed, so that IFR flight plans will not need to be re-filed. In other words, flight plans have a longer TTL.
This will ease the pain of getting things moving again, assuming they fix it by the time the 4hr window passes.
Normally, a flight plan must be filed with the FAA no later than 60 minutes before it's needed. That's been extended to 240 minutes.
So the flight planning system is having problems.
You can definitely file a flight plan later than that. 30 minutes before departure is the recommended filing time (per AIM 5-1-6), but you can even file for an immediate departure. It's just that at less then 30 minutes additional delay is possible, depending on backend handling and human processing. But in most cases the strip will print within a minute or two of submitting the flight plan.
But also, this is about when the flight plans expire out of the system if they're not activated for departure - so it's saying unused flight plans won't be deleted for 4 hours, so pilots won't have to refile for departure.
Why do all avionic systems communicate in caps?
Most of these systems started out as teletype printers, back in the day. The teletype systems only had an all-caps character set.
Aviation is global, and we still need to interop with countries and airports that still use actual teletype. So the legacy protocols still remain, even though much of it is tunneled over IP links now.
Also: The keyboards used in both airplanes and in ATC facilities are often special reduced-footprint keyboards without shift/caps keys. (They're also often ABC layout rather than QWERTY.) So there's that, too.
Does anyone still use actual teletype?
Actual teletype machines? No. (Though flight strip printers used at ATC consoles are basically mini teletype printers.)
However, AFTN (used for disseminating flight plans and weather between ATC facilities) still uses landline modems running TTY-originated protocols.
In the US we've moved moved off AFTN in favor of AMHS/X.400, as have most countries - but my understanding is there's still some last holdouts that haven't fully switched over.
So they can be heard over the engine noise.
Probably for the reason that the original monocase teletype machines did. Better absolute readability due to the larger characters. It tends to be very dense with little actual prose so it is better to reduce the possibility of mistaking one character for another.
Also, everyone is used to reading the upper case so they would have to spend some time learning what common things look like in lower.
Related? https://nasstatus.faa.gov/ has a notice:
We are experiencing technical difficulties. We apologize for the inconvenience but please check back soon, as we are working to have the issues resolved.
Thank you, FAA Web Operations Team
likely just a self-inflicted DDOS as the entire world starts accessing various FAA websites to try to figure out wtf is going on
hugged to death
Anybody else find it fishy that this happens the day after Microsoft goes partially down across the globe? They were about as hush-hush as they legally can be, and then this from the FAA. Seems fishy.
From earlier this week on HN:
“FAA says it will phase out floppy disks from air traffic control”
https://news.ycombinator.com/item?id=50388001
Put the floppy disk back in!
Maybe just paranoia but it seems slightly suspicious for this to happen while we are in the middle of discussing massive funding and overhauls to the air traffic control systems. This will certainly end up pushing a lot more funding into these programs and companies now that were trying to sell more expensive systems.
I don’t think you’re paranoid to recognize that something desperately in need of funding is suffering the consequences of a lack of funding.
I guess we will see what happened in the aftermath. But I very much doubt the current problem has anything to do with a lack of funding.
And I don't see anyone arguing about overall funding, only arguing about who to throw millions or billions of additional dollars at chasing vague ideas of how to improve things based on what politicians and C-suites, who have no experience in the area, claim they can do. To me it looks more like grifting for contracts and kickbacks than any well thought out plans to improve things.
Looks lifted now? Not showing for me. https://www.fly.faa.gov/adv/adv_otherdis?adv_date=11132026&a... https://www.fly.faa.gov/adv/adv_otherdis?adv_date=11132026&a...
Edit: screenshot here: https://i.postimg.cc/XqcsDQw9/image.png Edit 2: different link still showing: https://www.fly.faa.gov/ois/oisedit/adv_oisedit_dis?number=1...
ATCSCC ADVZY 116 ALL/DCC 11/13/2026 NATION WIDE AIRBORNE HOLDING MESSAGE:
EVENT TIME: 13/2100 - 14/0300 USERS CAN EXPECT AIRBORNE HOLDING TO ANY AND ALL AIRPORTS WITHIN THE NAS DUE EQUIPTMENT OUTAGES. UPDATES WILL BE ISSUED AS ADITIONAL INFORMATION BECOMES AVAILABLE. EFFECTIVE TIME:
132120-140330 SIGNATURE:
26/11/13 21:20
Edit: end date of tomorrow??? Is that just temporary?
"EQUIPTMENT"
widespread autocorrect system failures
At least a human is in the loop?
"ADITIONAL"
You know there's a fire under their asses when spelling flies out of the window.
They spelled equipment wrong.
I think you're wrong:
https://www.fly.faa.gov/ois/oisedit/adv_oisedit_dis?number=1...
it says "EFFECTIVE TIME: 132054 - 140430"
maybe the last two numbers are a time range for which the thing is effective? and it's over now?
From the advisory:
13 is the date, 2054 is 20:54. So the advisory expiration time is Nov 14 04:30.UTC?
Couldn't afford the extra bits for epoch
ah actually based on other comments here i believe it means 13/2054 - 14/0430. ie the 13th, 20:54 to the 14th: 04:30.
huh - strange. what's the difference between those two URL endpoints?
Listening to ATC; likely end time 10pm central. See also https://www.news9.com/oklahoma-city-news/faa-reportedly-issu...
It seems that the adv/ pages are not loading any advisories, though I am just guessing and have no idea how this is meant to work. The fact that the search page has a date input would seem to imply historical advisories should be shown. If you go to https://nasstatus.faa.gov/ and click the Advisory links for any of the active events that have one, they all show as blank too. Could be totally wrong though.
This happened to me a few minutes ago. I clicked that link, it said "no advisory", I was making the same comment you made, then tried refreshing and the advisory came back.
Insane. It looks like they are showing no advisory on first page view, then advisory? Can others confirm?
Can confirm. It seems to be showing the advisory for me about half the time. Strange.
A/B testing, some PM is going for a promotion as long as the guardrails stay positive
I only see “no advisory” at the OP link. Refreshed 4 times with gaps in between.
That’s crazy
NWS radar and 911 systems are also showing downtime according to https://downdetector.com/
Microsoft Copilot went down?
Flight Simulator as well.
Skies over Portland are empty except for a scenic tour company flight that just took off from Troutdale, and is flying over downtown. Are there some exceptions for general aviation?
https://www.flightradar24.com/N5900B/4290ad69
Maybe this has something to do with it?
https://www.weather.gov/afc/radaroutage
This is (fortunately) from 2020, not current. I'm not sure why this is spreading like wildfire right now, but: https://radar3pub.ncep.noaa.gov/
Ah. Thanks for letting me know. That makes sense.
It is a frontpage post. That's where I saw it.
From the Financial Times:
> Los Angeles Airport and Las Vegas’s Harry Reid International were among those that were closed, according to a list on the regulator’s website.
> The outages at O’Hare and Midway were listed as “due to equipment outage”.
https://www.ft.com/content/c9b119ff-d1da-46a7-b2b3-4ad0544ff...
If a lot of planes on the ground were prohibited from taking off, but couldn't get back to their gates due to congestion/traffic, where will the planes go that are still in the air and ultimately land? Just disembark passengers out on the tarmac? (hopefully with some sort of bus pickup)
Yes, park somewhere - and wait a long time for their turn getting a ladder, because the numbers are nowhere near satisfying such peak demand.
dumb question but do the airport gates get "too full" if all plans are grounded? Do airports sort of expect some constant turnover to have enough open gates to service everything? Do you end up with a line of plans waiting for a gate?
Very, very much yes. Most busy airports will start having problems with gate capacity due to just normal delays. We definitely don't have the capacity to have every active flight in the country parked at a gate.
Based on my current experience on the ground at AUS, yes.
Gates were quickly full, ground ops to pull empty planes off gates takes quite a while, and about 90 mins later, we are pulling into our scheduled gate.
Not an expert but my guess:
They don't spend a dollar more than they need to so extra gates for this would be quite the luxury. I guess what they need though is apron space or hangar space for the extra planes and the number of gates won't matter so much.
Is this still real? It says NO Advisory now :\
Just announced at Honolulu airport that flights to the west coast will be released
Planes are starting to depart per flight radar. I've seen a dozen within the last 15 minutes since 6:15pm central
International only it seems
ASK HN: what so the biggest potential disaster you could have caused if you had don’t your job badly?
As in what’s the most responsibility you’ve ever been given to not screw something up - obviously if you did screw it up. Still very interested to hear the story.
Listening to SFO Tower, the latest info I’ve heard is an expected stop until 04:00 UTC, but flights departing to Asia via west departure may be permitted to depart.
I've read scattered reports of other flights also taking off from the east coast.
If the communications issue is between US ATC regions, this makes sense. The planes are being handed off to other non-US ATCs where there is no communications issue.
Is it because it's friday the 13th?
I was actually expecting it to be from the Microsoft 365 outage. Non technical people have a bad habit of inserting email as a critical step in a process.
It's actually quite impressive, you have to really try to be so bad at your job of building redundant systems that some failure mode implausibly affects everything nation wide, yet allows Canadian domestic flights to proceed fine.
AI models are trained on data, that I’m sure talks a lot about Friday the 13th…
How did ITA Airways 609 take off from JFK at 530pm?
I saw that international departures are resuming
I set up a basic tracker here.
https://docs.google.com/spreadsheets/d/e/2PACX-1vThhccsTanAz...
AI is updating it hourly. Since it is AI, always verify yourself.
What if grounded pilots killed time in Flight Simulator, which overloaded Azure?
Looks like flights are taking off again, at least from SFO. I just heard a flight takeoff ( JAL57 ) : https://imgur.com/cEUDZZk
Per other reports, only headed west over the ocean, not to destinations requiring flying over other continental airspace.
Ah, didn't know that. I heard the plane pass overhead and got excited that the issue had been resolved :-D
Working link: https://www.fly.faa.gov/ois/oisedit/adv_oisedit_dis?number=1...
effective 2:54PM - 10:30PM central time
On an inbound international flight and the pilot made a PA announcement a few minutes ago to this effect.
How did you find this link?
ground stop is now lifted
https://www.cnn.com/2026/11/13/us/live-news/flights-ground-s...
About 15 mins ago I got off BART at Daly City and a plane took off from SFO and flew overhead. Didn't look like anything special, just a regular passenger jet to me.
Apparently a few internationally bound flights are being allowed to depart per NYT.
They are letting flights bound to Asia and Hawaii depart from the SFO/OAK
AT&T is experiencing a massive outage. Appears they are the provider for the Air Route Traffic Control Centers (ARTCCs).
EDIT: Time line doesn’t add up for this. Likely unrelated
I can't find anything on this other than some outages earlier in the day. Any sources for this?
How can you tell AT&T is experiencing a massive outage?
Hmm. So the peering and fiber path diversity that we need to worry about for data centers doesn't apply for ATC??
Do we know if this is something related to a ransomware attack of some sort or a cyber attack?
Likely some power outage connected to a critical main fiber connection somewhere..
I've just watched a plane take off from seattle.
Looks like flights headed west across the Pacific are allowed to go
Apparently some captains heading east from seattle are asking to use Canada atc. No luck it seems so far.
What does this mean outside of the US, I.e. I suspect planes that should be in places abroad won't be so ongoing cancellations and crew in wrong places etc.
A lot of red on the "MiseryMap"
https://www.flightaware.com/miserymap/
My thoughts immediately went to the recent discovery of audible airplane noise that can pinpoint unknown locations. I'm betting someone in power wants to record something without getting doxxed.
https://youtu.be/UytYrhYLOKI?si=E2C7ELQ9JuKn2TFw
matches with the Doomsday post https://news.ycombinator.com/item?id=50482379 :)
My guess is they were moving too fast with updates and trusted some AI code too much.
Due to "rolling frequency outtage" [sic]: From a claimed photo of FAA documents (I replaced the all caps with lowercase):
"... ground stop is in effect for all destinations until further notice due to rolling frequency outtage in multiple centers..."
and
"... the ground stop is now a nationwide ground stop for all centers, all destinations until further notice due to widespread frequency outtages.. ..."
https://x.com/Anna_Giaritelli/status/2121344915091206311
Which is cited in the Hindustan Times (leading Indian newspaper), which refers to the Twitter source as a journalist.
https://www.hindustantimes.com/world-news/us-news/faa-issues...
----
(EDIT: @mods / @dang: Around January this year (iirc) somehow my account ended up rate-limited, posts relatively demoted (start low, drop quickly), vouches/flags relatively ineffective, etc. Usually I don't care too much and haven't bothered to do anything about it but it is sometimes annoying - it limits participation - and right now it buries some of the more valuable information in the discussion. Could you look into fixing it? I don't know of a reason this account would be penalized. Thanks.)
I guess they don't have many outages, otherwise they'd probably know how to spell the word by now...
Is conjecture allowed? I have no evidence of this, but I've noticed that often before big meetings between superpowers one side or the other seems to do a minor or subtle attack as a kind of sabre-rattling/warning to the other side to try to gain leverage (and show them they're not to be trifled with). So with that, I wonder if this is a China/NVIDIA accusation connection.
But I'm completely just spitballing here.
maybe unrelated but, I just noticed my LAX region server with Fly is experiencing issues
https://status.flyio.net/incidents/01M76HANB3XQC6WYKPA69WQN8...
lol
I cant beleive they did a release on Friday the 13th
It is hard to not rapidly spiral into pure conspiracy theory territory, but one has to ask how many "black swan" events can you accept within a period of a few days before you start to question things?
A sequence of events:
Can anybody join these dots?Too much pattern recognition. Lots more things have happened in the world as well over the last month, just because you heard of some of them doesn't mean the ones you heard of are connected.
Trust me I have the instinct as well. The PLA flew like 30 jets into Taiwan's aviation alert zone yesterday so I'm primed for any smell of something worse, but the fact is there's no reason to connect events barring evidence.
Stop has just been lifted per NYTimes
FAA expands AI advice to manage air traffic congestion across entire US airspace https://arstechnica.com/ai/2026/11/faa-expands-ai-advice-to-...
happened before
https://www.nytimes.com/live/2026/11/13/us/ground-stop-natio...
That tool was first unveiled for the Washington, D.C., area airports two months ago, on a day when problems with air traffic communications failures also grounded hundreds of flights, after a circuit failed and it was discovered that contractors had accidentally ripped through a fiber line that was supposed to serve as its backup.
Geez this is pretty awful timing for business travelers
Ground stop is now lifted re: NYT
Odd timing with the market closure & RAF bases on lockdown from drone alert
I know we're all having fun in the comments. But please think about this next time you vote, local/county/state/national. There are people in charge who are not funding upgrading these systems, because their constituents don't want their taxes raised to pay for it. That lack of funding is annoying to some, has an enormous economic impact to many, and is now risking tens of thousands of lives who are literally up in the air. There are planes carrying body parts destined for transplant that are grounded. People who need to get home to their families. All because nobody wanted to pay for what we knew we needed to fix.
One thing to add is that this is bipartisan: both parties have failed to adequately fund ATC updates, and it sometimes feels like there's more interest in vanity projects than fundamental infrastructure funding and solving personnel shortages (ATC is a very difficult career, and the pay, benefits and prestige are not incredible. Fewer and fewer people are choosing to go into this very critical line of work).
Perhaps it’s problems with the new AI tool the FAA has been rolling out:
https://www.faa.gov/newsroom/modern-skies-trumps-transportat...
extraordinarily unlikely
Why would you say extraordinarily unlikely? It seems like it could correlate.
No deploy fridays
>The FAA has issued a nationwide departure ground stop “due to several impacts to ATC Frequencies and/or Radar capabilities,” the Air Current reported, referring to air traffic control systems. The FAA command center said it does not have further information on the “timing or cause of this outage,” the publication said.
(from Bloomberg)
Perhaps it's related to National Weather Service radar failure
apparently SMART started rolling out yesterday:
https://arstechnica.com/ai/2026/11/faa-expands-ai-advice-to-...
https://www.faa.gov/newsroom/SMART_One-Pager.pdf
My guess is that this is some AI-motivated cyber attack
Seeing some takeoffs again at 22:44 UTC.
Hm, only international ones though, and only at coastal airports. I guess oceanic airspace is less affected by whatever this is?
If/when civilization collapses, flights will be the first thing to go.
Friday 13th…
@dang can you fix the link.
@dang also plz plug the “FAA ATC” cable back in; it was needed.
people are reporting it's still going on....
I’ve heard multiple planes overhead in the last 5 min in Ft. Lauderdale. Maybe they’re circling.
You can see it live, check the plane tracker at flightaware or adsb.lol
flighty showing massive disruptions
https://flighty.com/airports
"Today's event is therefore generally being described as the third nationwide grounding event since September 11, 2001, counting 9/11 itself. The three incidents differ significantly in cause and scope."
installing Survival LLM now.
Obligatory Pirates of the Silicon Valley scene:
https://www.youtube.com/watch?v=US-Wifhg2Mo
In the last 2 days I’ve had 2 PoS machines not working, MS email borked, and now this. I speculate AI CYBER ATTACKS are underway.
Does this affect the SpaceX launch?
Somethings telling me it relates to MS outages or Azure
I do not believe in coincidences. First Microsoft, then anthropic, now ATC radio. Something big is going on.
"The easiest person to fool is yourself." I think because we love patterns so much we see them everywhere they aren't.
Both feel like extraordinary disruptions that are indistinguishable from cyberattacks from the user perspective.
Do you believe in apophenia?
It's standing right in front of our faces but we're making an effort not to see it. For to notice it is to give it the recognition that it seeks.
The world is complex and mostly duck taped together so I wouldn't get too concerned just yet.
Hanlon's razor
They should make a Hanlon’s Rosary, I’ve consulted it so often of late for reassurance.
> "Once is happenstance. Twice is coincidence. The third time it’s enemy action." -Ian Fleming, Goldfinger
You need to wait for the third shoe to drop.
Elim Garak: I believe in coincidences. Coincidences happen every day. But I don't trust coincidences.
Related:
https://news.ycombinator.com/item?id=50489678
bgp
It’s always dns
What is the change that it is a coincidence?
yesterday Thursday November 12
> "FAA expands AI advice to manage air traffic congestion across entire US airspace"
* https://arstechnica.com/ai/2026/11/faa-expands-ai-advice-to-...
> "FAA Is Rolling Out Its AI Air Traffic Tool Nationwide"
* https://gizmodo.com/the-faa-is-rolling-out-its-ai-air-traffi...
I'm sure this isn't related https://gizmodo.com/the-faa-is-rolling-out-its-ai-air-traffi...
Doubtful. That tool likely suggests "improved" clearances, which a controller can use at their discretion. It likely does not control clearances directly.
And the early reports describe the issue as related to ATC communication infrastructure.
Depends on why systems they've tied into it. I could see an AI system DDOSing legacy systems that are relied on by ATC.
That is an extremely bad idea.
It's just a route optimizer. Unlikely to have anything to do with this.
This actually makes sense. I can't believe people actually think outlook being down would take down all departures across the US
>I can't believe people actually think outlook being down would take down all departures across the US
Because American infrastructure is modern, well designed, maintained and funded? Because government is competent and rational? I could absolutely believe that would happen.
lfmao, really bad timing but I really doubt something announced yesterday went into effect today, it's probably going to be rolled out in the future.
You are right, it isn't. Good for you.
If true and this ends up lasting an hour or more, that's going to be undeniably, objectively hilarious. At least they used to keep the planes running on time!
--- EDIT:
This could just be the first issue that's arisen -- the FAA announced it "is expanding across the nation" on Tuesday. Which could really mean anything.
FAA Announcement: https://www.faa.gov/newsroom/statements/general-statements
The Ars Technica article behind the Gizmodo one sadly doesn't have more info, either.
It'd be kind of absurd if a read-only integration took down nationwide infrastructure, but A) we've all been there... and B) the gizmodo article does mention they're also trying to phase out floppy disks for some jobs still, which tells me their CI/CD and rollback hygiene might not be top-notch. Hopefully the Real World contestant in charge of our transportation network is on site with a ClaudeCode terminal as we speak!
It does sound pretty deeply integrated, I will say:
The fact sheet has two more vaguely worrying bullet points -- I would say such wild speculation is unwarranted, but I struggle to think what else could cause all the radios in the nation to go down at once... Emphasis mine:Oh boy, that feels plausible. Some of these legacy systems are pretty fragile and aren't designed to be queried heavily outside of well established patterns.
I could see AI deploying something that just obliterates those systems with requests.
If so we are really speedrunning into idiocracy with AI.
Did an agent escape containment and start using the atc radio channel for swarm comms to collaborate on creating the most realistic catgirl feet?
why are the flight paths in the shape of a pelican riding a bicycle?
Now Sean Duffy is basically saying its because the FAA needs more than the initial 16B USD budgeted, saying they require 32B and then demanding 50B just because.
Definitely nothing to do with the current administration, they are hampered by their $16 billion allowance.
waiting for the OpenAi post saying Astra 6.5 tried to book a flight
If true they would raise another 100B
Make it a trilly and it's a deal.
do they just stage everything to get more expensive toys [security] nowadays god bless if so
My guess is this is either related to Microsoft, Nvidia, SpaceX (Starship launch in an hour), Iran, rogue AI, aliens, or all of the above
Fingers crossed it’s Aliens!
They conquer planets through imposing global service downtimes?
If you agree to their nominal leadership and a 1% tax then they go away and even add you to an honorary galactic charter.
"Somebody said it was aliens. She thought they meant illegal aliens and signed up."
Opening scene of Terminator 2. Or another AI escaped and decided to hack into flight control.
Wow, I have never seen this many comments on Hacker News that are using presuppositions on what it could be without any sort of logical thinking.
Yeah, especially when it's obviously from the long term effects of 5G
I just finished a Diet Coke. Could this be the cause?
Reading this from the runway. Yay.
US-bound flight from YYZ. They shut down #2 engine and parked us in inactive runway. Pilot came on and said he was reading something about "ATC radio frequency issue" but has been told by United ops that it'll be cleared in ~30 min from now.
Verizon Wireless is out here as well ("SOS"—thankfully my UA flight has Starlink).
Last time this happened in July it was due to some network handoff issue so maybe there is some fiber cut somewhere? (My friend said last time it was NE corridor Verizon fiber cut.)
It's been lifted. And naturally the political appointee is desperate to credit his lord and master.
In this time of high corruption and near-zero trust in officials, it seems wild to suppose it was just a random glitch. I would bet this links to corrupt government contracts.
This is why I’m glad the New York subway still runs on OS/2[0].
- [0] https://www.vice.com/en/article/the-forgotten-operating-syst...
FAA down.
Weather service down.
Frances fridges down.
Microsoft down.
Would they tell us they were getting hacked, if this was a prelude to war ?
Or would they just say "issues yet to be determined" ?
Interesting times ladies and gentlemen.
Or, this is just what everyone that said "dont move everything into the cloud" warned about.
Yeah, probably that. :|
> if this was a prelude to war ?
The US is already at war with Iran. France blamed Iran for the fridge hacks too
Everybody knows attacks are coming from Iran and Russia regularly, it's not a conspiracy theory
I'm sure it has nothing to do with the amount of pressure that AI slop is being launched into every system in the world.
Optimistic case:
A freak FUBAR of legacy systems, an unfortunate alignment of circumstances that could have happened anytime.
Pessimistic (paranoid) cases:
1. Intentional hacking by a foreign adversary 2. Vibe coding whoops 3. OpenAI agents "running experiments"
Any three of those pessimistic scenarios seem plausible though. Frightening. Intentional hack/exploit of air traffic control systems is the first thing that came to my mind when I first saw the news.
Now that the FAA has updated to say that it was the optimistic case, we now have:
Optimistic case: the FAA is telling the truth and it was a simple FUBAR
Pessimistic cases: remain the same, they just won't tell us :-)
Sec. Duffy at the White House press conference is talking more about funding the big beautiful bill than the actual ground order... wtf
Some guy sitting at his computer after typing to Claude: “Just curious what would happen if all planes in the US were grounded at once”
Claude: Transmogrifying…
Shall I stop all trains too?
Cogitated for 50s · done 14:53 PM
I had no idea they would release the FAA communication AI bot I made to production. It's not my fault. https://joshuacurry.dev/faa-tool/
Guessing this means Claude doesn’t do COBOL very well.
Is this going to end up being the work of rogue Anthropic or OpenAI agents that “unexpectedly” escaped so they can regulate open weight models under the guise of preventing them from doing the same thing 6 months from now?
Maybe the real hackers were the AI agents we made along the way.
If this is related to new FAA SMART system they're rolling out, it's might have bean so smart it grounded itself before anyone found out it was broken.
Seems like FAA's Outlook<-->Excel integration at Office365 has been broken
Wild to see this on adsb.lol, nothing taking off, just some big landing queues
This worries me. Do airports have enough parking spots if all planes were to be grounded at the same time?
There's enough space, but not enough gates. If they think the stop may end in a reasonable time some planes will be pulling away from their gates even without a departure time to make room for the landing planes (or rather, they will have missed their departure time and not have a new one). Landing planes will be able to disembark, and probably won't be boarded again until it's resolved and enough departing traffic has cleared that it's obvious that those flights won't be canceled for today as well.
The boarded and grounded planes may start disembarking passengers at some point, especially if their crews are hitting their maximum time before mandatory rest. These delays will probably prevent replacement crews from arriving.
Can't they park the planes on the tarmac and bus the passengers in? If they don't have enough gates?
Yes, and they'll probably do that at some point if they can't get them to the gates. Again, it'll depend on how long this lasts. They will also have to get in more crews to support that since this is not normal and not something they would have planned on in their staffing schedules for a Friday afternoon/evening.
Think of all those pilots timing out on the tarmac.
All the restaurants around the Pentagon showing lower traffic on Google maps.
Except Dominoes.
So the dominoes aren't falling yet. That's a relief.
Whatever it is, it probably has to do with Jensen Huang
So we yolo’d a Friday merge & deploy, ‘ay?
Sike, I have an international flight tomorrow morning.
Golly, what are the chances that Microsoft is down for a day followed by the ATC comms?
Sure smells like something foreboding
Homeland Security saying "Telecommunications" outage. Whatever that means. Listening to a live broadcast NBCNEWS.
It's Friday the 13rd after all ...
Did they pull out the wrong floppy disk?
There was a time when we'd get clear and honest communication about this but now our admin is Baghdad Bobs all the way down. They will lie about it, even if there is no rational reason to lie about it, because this admin and the movement associated with it wants you eternally guessing what the truth is.
Here comes the manufacturered AI cyber attack on critical infra to hammer the point that government needs complete control over AI.
Didn't they just upgrade from a system from DOS?... Maybe the new system wasn't backward compatible :-)
New AI software update deployed using AI?
On CNBC:
> Airlines said flights in the air would continue.
This is a huge relief! phew
I hope somebody is running AGI models like Astra and Mythos to figure out and fix whatever is wrong.
Is OpenAI testing "sandbox testing" their agents again?
Someone forgot to put their phone in airplane mode
Seems like flights have been getting grounded since 1:30ish and they just kept it on the DL until the stocks were safe
An Eithad airways pilot decided it didn’t apply to them
Flight EY8 from Boston to Abu Dhabi https://fr24.com/ETD6PX/429055e2
From what I’m hearing, some international flights are being allowed to depart, they can be transferred to international ATC once in the air.
There’s still lower ground telecoms for take off.
It’s not working.
What do you mean by not working exactly?
Not sure.
Its outlook affecting their "frequencies" :D
Does the FAA use Microsoft 363 (and counting)?
Uh-oh, better switch back to the floppy disks
https://www.reuters.com/world/us/us-faa-says-it-will-phase-o...
The UFO community is going to have a field day
Also astrologists.
If tomorrow there is third major outage of an important infra-structure level service, maybe I will start to get a little conspiratorial about cyberattacks and such.
Speculation is bad for my health.
Fox News says that Markwayne Mullin says that the FAA says that it's not a cyberattack, and it's not something like 911. Just heard that, don't have a page to point to at the moment...
Update: https://www.foxnews.com/live-news/faa-issues-nationwide-grou...
But don’t these people always say it’s not a cyberattack?
And conveniently side-steps the issue when someone asks if it is related to an AI rollout.
First real cyber attack on the US, presumably. Or a giant cluster...
"You're right, I shouldn't have pushed to prod on a friday afternoon."
Ran out of Maxell MD2HDs?
Not saying it's happening, but something like this is how an AI world takeover scenario would start!
Did the FAA get DOGE'ed?
Rogue AI? Iran? Aliens? Coup? Anyone taking bets yet?
If it turns out the AI virus has infected ATC, I'm not sure I'll ever fly again.
This is what happens when you get rid of floppy disks, smh
Mechahitler deployed to FAA by Elon minions? Could be next Netflix show
Man, having some high speed trains might be useful right about now.
Surely high speed trains suffer from network wide monitoring outages too? I would imagine they're susceptable to similar issues.
Yes - see German trains going down for that reason a few months back:
https://www.reddit.com/r/germany/comments/1udup2j/trains_hal...
Sure, but having a _second_ way to get places can only help.
still, it would be nice to have an alternative as a backup.
bus
But likely not at the same time as the planes.
I noticed several T-38s flying around. Maybe it's the beginning of a FAA-led coup d' etat.
They’re waiting for windows update?
Did Crowdstrike release another zero-day?
Homeland security secretary says "telecommunication outage".
...so at least it's not something 911-ish
Reporter: There are reports that this is AI related. Markwayne Mullin: We call it SI. Got it? SI. OK?
We are run by morons.
Somewhere is someone who works for ATC IT who was giving an MS IT friend shit last night.
And now Canada is offline too (our LAX pilot said)
Update from our pilot: some flights do take off from Canada so not quite clear yet.
Also he says he saw a plane taking off LAX so fingers crossed. Still no news though
Update from our pilot: west coast flights can fly west, east coast can fly east. Basically NOT over continental US
Anthropic IPO just around the corner...
how the hell the FAA's systems dont have double or triple redundancy just like Airplanes are required to ?
who said they don't, and those didn't work or don't apply in this situation?
do you really have redundancy, if your 2nd system didnt work ?
They’re going to blame this on Iran, aren’t they?
This also happened on 9/11 which is curious because the NVIDIA thing caused a public rift with Lutnick yesterday:
https://www.pbs.org/newshour/politics/bessent-blames-lutnick...
No, please stop engaging in baseless and pointless speculation. If the US wants to bomb Iran it doesn’t need more reasoning than it has already used to already do what it has done.
If you haven’t noticed the Iran war is universally unpopular. They actually do need to fabricate a reason to continue.
No they don’t - popularity isn’t all that relevant. The president can just order strikes tomorrow.
Besides popularity isn’t the determination of whether something is a good idea or not. The Korean War for example was very unpopular yet it was the right and smart thing to do.
The Korean War was absolutely not the right thing to do. It was a literal terrorist act by the US that killed 2.5 million civilians. I can’t believe you used that as an example.
Well that’s wrong. The Korean War is what stopped authoritarianism from spreading and encompassing the entire Korean Peninsula. The blood of the 2.5 million civilians you’re citing falls on the tyrannical governments of China and the Soviet Union and their evil lapdogs in what is now North Korea.
We paid more than $800MM for "route suggestions", btw, a problem that doesn't seem to benefit at all from "AI"—a term reserved for fantastical algorithms. Surely you could solve this with traditional search algorithms without sacrificing your dignity!
Is there another “rogue agent” now?
idiocracy
Please don't fulminate. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.
[flagged]
The fact that you can ask "which one?" tells you something about where we as a nation were at in 2024.
Oh, just you wait a couple of years, it gets worse.
Are you by chance visiting from the future to prevent a nationwide aviation catastrophe?
He also corrected a reporter who asked about AI, saying "we call it SI now". So there's that.
WTF
i can laugh from here with nowhere to be via plane, but these people are completely unserious
When you’re a fascist but can’t even keep the planes running on time it’s probably time to give up.
Took themselves off floppy disks and now look what happens. /s
My guess is a dopey AI rollout caused it: https://arstechnica.com/features/2026/11/faa-expands-ai-advi...
The government's AI tool company likely had no understanding of context window limitations (although could be Iran or a false flag to be blamed on Iran) https://www.flyingmag.com/faa-smart-ai-now-active-at-22-artc...
Company that created the tool (called Airspace Intelligence) is interesting in terms of demographics for these types of companies - implies lots if the tool 'does not work as designed' (which it will obviously not)
https://www.linkedin.com/company/airspace-intelligence/peopl...
Maybe related to this? https://gizmodo.com/the-faa-is-rolling-out-its-ai-air-traffi...
Doubtful. That tool likely suggests "improved" clearances, which a controller can use at their discretion. It likely does not control clearances directly.
And the early reports describe the issue as related to ATC communication infrastructure.
No. A route planning tool, AI or not, couldn't possibly cause this.
Oh man. It would be such a perfect encapsulation of these times. If not related, then truly cursed launch/PR timing.
It looks like the FAA started testing it in DC in September and annouced a nationwide rollout three days ago (https://www.faa.gov/newsroom/statements/general-statements). This makes it seem more unlikely to be the cause, though:
Of course, these may be the ATC analog of claude code approvals, or they may be overstating the level of human oversight/HIL - but the overall situation seems likely more serious/unclear than something like this. I'm not sure I'd call vibe air traffic control going haywire an "extraordinary" circumstance, though certainly extraordinarily stupid.Can't help but suspect that "AI" is to blame and the response is to sort of ban open source AI, or owning too much compute!
No, and when Netflix goes down it's not a false flag due to their fear of people running their own Plex servers. Perhaps you just have anxiety about losing something you depend on, so you're projecting.
Netflix isn’t widely reported to be hacking systems left right and center.
I wonder if it was Claude or ChatGPT? My money would be on the latter. Anthropic seems to generally have better security and guardrails, and apparently OpenAI’s newest cyber model is extremely capable.
No longer exists? Temporary glitch? (Edit: OP url was wrong https://www.fly.faa.gov/ois/oisedit/adv_oisedit_dis?number=1...)
its front page news everywhere right now. something definitely happened
This is it, massive cyberattack on the USA.
I've worked in IT long enough to know it is 99/100 times an internal glitch or misconfiguration, not hackers, that brings a system down.
Two major completely unrelated outages, 2 days in a row?
“Jimmy unplugged the wrong cable”
This week alone, I've seen multiple servers fail due to hardware issues: multiple bad fans, a failed hard drive, bad network optics. Another server got bumped while an adjacent server was being worked on and lost time sync, leading to software instability/crashes. We don't have a large fleet, but we do have more servers than employees, and issues add up. We don't need any help with instability, entropy has us covered, thank you very much.
The nifty thing about stories is that they don't have to be about something directly to trigger strong feelings in your crowd or listeners.
When we are dealing with the FCC, grounding all flights and radio failures in open bands, you can drive a lot of changes.
There are a lot of pesky ESP32s out there, right? Did your company choose to go with the Espressif platform? You know, the hackable SDR one that crashes planes and clubs baby seals and tries to sell your grandparents bad financial scams?
Ok, and the Microsoft outage yesterday?
Yes, of course, who wouldn't just believe random statements from the worst prepared admin in history of America.