Networks & InfrastructureFaa OutageAir Traffic ControlNetwork ResilienceFiber Migration
FAA ground stop: HPE rejects router blame as Lumen flags legacy copper rut
A November 13 FAA network failure grounded U.S. flights for hours. HPE says its Juniper device worked as configured; Lumen says copper is no longer fit for purpose.
Networks & InfrastructureWhy it matters
- The FAA ordered a nationwide ground stop on November 13, the longest since 9/11, affecting tens of thousands of flights.
- The FAA is replacing roughly 2,000 Cisco 6000-Series routers with Juniper Networks devices.
- AT&T received an initial $74.3 million award to lead the FAA's replacement network, targeted for 2028-2029.
- Analyst Roy Chua says two edge sites 'shouldn't be able to destabilize 17 centers', pointing to a blast radius problem.
- Lumen says its network was not involved and is accelerating copper retirement.
The story
The Federal Aviation Administration ordered a nationwide ground stop on November 13 after router-related traffic exceeded 1,000 connection attempts every few seconds, disrupting aircraft radio communications and radar data feeds in the longest U.S. flight grounding since the September 2001 attacks. Tens of thousands of flights were affected before the system stabilized and the stop lifted at 8:15 p.m.
The fallout has pulled in an unusual cast: defense contractor L3Harris Technologies, Hewlett Packard Enterprise and its Juniper Networks business, and three U.S. carriers building the FAA's replacement network. The incident appears to have started during a routine telecom upgrade by L3Harris.
How did the outage unfold?
The FAA's telecommunications network became unstable on the Friday afternoon. Rumors initially pointed to Microsoft's global cloud failover the same day, an AI-powered FAA system called SMART deployed days earlier, and even a possible cyberattack.
Transportation Secretary Sean Duffy instead identified L3Harris's copper-to-fiber migration work as a potential trigger. L3Harris confirmed it was "modernizing the communications infrastructure supporting the National Airspace System in close coordination with the FAA" and cooperating with the investigation, without accepting responsibility.
The contractor said it had completed more than 70% of the planned airspace communications conversion from legacy copper wires to fiber.
Was a Juniper router to blame?
At a press briefing, FAA Administrator Bryan Bedford revealed the agency is replacing roughly 2,000 Cisco 6000-Series routers with Juniper Networks devices. The Wall Street Journal reported that a Juniper device in southern Arkansas triggered the entire outage. HPE strongly refuted that account.
"Based on what we know today, the device functioned as it was configured to by the installer. It did not fail or act improperly," an HPE spokesperson told ABC News.
Duffy said the suspect router is being examined at an L3Harris lab in Florida, without confirming whether it is a Cisco or Juniper unit. Investigators have also identified a second rogue device in Amarillo, Texas.
Analyst Roy Chua, founder and principal at AvidThink, cautioned against pinning the outage on one vendor. He drew a parallel to the November 2009 FAA Telecommunications Infrastructure outage, blamed on Harris Corporation after a replacement router was misconfigured during planned maintenance.
"To me, both events point at change management as much as equipment age," Chua said.
He argued the two suspect edge sites "shouldn't be able to destabilize 17 centers." He cited known mitigation practices: segmentation, control-plane policing and rate limits, storm control, disabling unused features, and out-of-band management. He noted the FCC drew similar lessons from CenturyLink's 37-hour outage in 2018.
Who builds the replacement network?
L3Harris inherited the FTI contract when Harris Corporation merged with L3 Technologies in June 2019. The FAA has since turned to the major carriers for a successor.
Verizon won the $2.4 billion FAA Enterprise Network Services contract in 2023. AT&T then took over as lead contractor, receiving an initial $74.3 million planning and development award. The FAA wants the new network ready by 2028.
Lumen's role only became public when Duffy named it as the third partner. "I have AT&T, Verizon, and Lumen all ready to go," Duffy said. He complained about missing funding and repeated remarks from September, when a severed backup fiber cable caused major East Coast delays.
"One of the biggest problems we continue to see is our network, and the architecture of our network is from 1980," Duffy said. "I'll be going tomorrow to build it and this telecom system would be ready by the end of 2028, the first part of 2029."
What did Lumen say about copper?
A Lumen spokesperson said Lumen's network was not involved in the FAA disruption and is operating normally. The company pointed to its accelerated copper decommissioning plans disclosed in recent earnings.
"These events made it clear we urgently need to modernize the U.S.' air traffic control system," the spokesperson said. "Decades-old copper networks were never built for today's demands. Lumen is working to retire its legacy copper and invest in modern network capabilities that mission-critical systems need."
Would programmable networks fix this?
Chua said carrier diversity removes single-provider risk but not common-mode risk. "The risk moves upwards to the shared overlay, and to the integrator that's maintaining the layer above," he explained.
He noted past Mplify efforts to standardize fault reporting via APIs, not universally supported by carriers. Lumen and EY promote programmable, network-as-a-service models as outage mitigation.
Chua was measured: pre-change validation, staged activation, streaming telemetry and automated isolation would have helped detection and containment, but it took about 45 minutes to trace the fault, partly attributed by the FAA to paper maintenance logs. The Rogers outage in 2022 and Optus in 2023 showed change windows and automation are often where failures begin.
The FAA's investigation continues as AT&T, Verizon and Lumen line up the replacement build, pending the funding Duffy says would put the new network in service by late 2028 or early 2029.
Also reported
Source: sdxcentral.com
More from Daniel Okafor
Show full bio
Senior reporter covering regulation, policy and their impact on Carrier business, 5G and 6G, fiber and broadband at Telecom Gazette.
546 articles