Microsoft Blames 365 Outage on a Route-Pulling Maintenance Bug

Microsoft has published a preliminary post incident review for the July 23 outage that pulled Microsoft 365 and a long list of Azure services offline, and the cause was not a failed server, a bad deploy, or an attack. A software defect in the automation that turns a maintenance request into actual network changes swept extra devices into the isolation perimeter, and their IP routes were withdrawn along with the one device the job was supposed to touch. Connectivity between the West US datacenter and Microsoft's wide-area network stayed broken for five hours, from 14:44 to 19:41 UTC.

The shape of the failure is the part worth sitting with. Traffic that stayed inside West US kept working normally. Anything entering or leaving the facilities did not. A service could be healthy, its dependencies could be healthy, and the two could still fail to reach each other, because the routes that carried them out of the building had been retracted by a maintenance job that believed it was isolating a single device.

RelatedMicrosoft 365 Down: SharePoint Is 78% of the Complaints

The confirmed details, as Microsoft states them:

• Incident MO1437424 ran 14:44 to 19:41 UTC on July 23, 2026, roughly five hours from first impact to full recovery.
• The trigger was a defect in Microsoft's automated maintenance change process, which incorrectly identified additional devices as part of the maintenance activity and removed more IP routes than intended.
• Only traffic crossing the West US region boundary broke. Workloads that ran entirely inside the region were unaffected.
• The blast radius crossed product lines: OneDrive, SharePoint Online, Teams, the Microsoft 365 admin center, Power Automate, Copilot Chat and Loop, plus Azure App Service, Application Gateway, Azure AD B2C, Kubernetes Service and Virtual Desktop, plus Defender and Sentinel.

What exactly did the automation get wrong?

Working on a live network device means isolating it first. You stop advertising the routes that point traffic at it, wait for the traffic to drain, then do the work. Microsoft runs that through an automated change process: a maintenance request goes in, and a concrete set of device operations comes out. The conversion step is where it broke. In Microsoft's wording, "automated systems included some additional devices in the perimeter to be isolated" beyond what the initial assessment covered, so "a set of IP routes was removed in error while isolating a device for routine maintenance."

Nothing about the individual actions was malformed. Every route withdrawal the system performed was a route withdrawal it intended to perform. The defect was upstream of all of it, in the translation from "isolate this device" to "here is the list of devices in the isolation perimeter." Once that list was wrong, the rest of the pipeline executed a correct plan for the wrong scope, and the West US perimeter lost the routes that carried traffic to and from the wide-area network.

Why did the redundancy check not stop it?

There was a safety check, and it passed. Microsoft's maintenance process verifies that at least one of the two redundant paths into a facility stays operational before work begins, and that verification ran. It was evaluated against the device named in the request. The isolation that followed was wider than the request, so the check answered an accurate question about a smaller blast radius than the one that actually executed.

That is a specific and uncomfortable class of bug, and it is not unique to Microsoft. Any team that has automated infrastructure maintenance has probably built the same seam: a guardrail that validates the plan, and an executor that recomputes what the plan means. As long as those two derive their scope independently, the guardrail is checking a different change than the one that ships. The fix is not more validation on the conversion step. It is making the safety check consume the same computed device list the action will use, so a scope error fails the check instead of sailing past it.

Was our read of the complaint data wrong?

Yes, partly, and it is worth saying so plainly. Our coverage during the outage leaned on the Downdetector complaint mix: SharePoint at 78 percent, Excel at 11, the admin center at 6, and argued the fault lived in the content layer beneath Teams and Office rather than in Teams itself. The dependency reasoning holds up. The location does not. The break was in routing at the region perimeter, a layer below anything SharePoint owns.

What complaint mix actually measures is where users notice a fault, weighted by how chatty a service is and how much of its traffic crosses a region boundary. SharePoint sits behind Teams files, Office for the web and the admin center, so a single network fault surfaced as three separate app failures and landed in the reports under one name. The signal that made SharePoint look like the culprit is the same property that made it a bad place to point. Complaint telemetry is good for detecting an incident quickly. It is close to useless for locating one.

1. 14:44 UTCRoutes withdrawn, traffic in and out of West US starts failing 10:44 a.m. ET
2. 15:24 UTCMicrosoft acknowledges the incident publicly, tracking it as MO1437424 Downdetector reports peaked at 2,403 minutes earlier
3. ~15:44 UTCEngineers identify the cause within the first hour and begin reconnecting services
4. 17:45 UTCMitigation underway routes restored to the devices pulled in error
5. 18:26 UTCResolution completed
6. 19:41 UTCFull recovery, five hours after first impact
7. By Aug 6Final post incident review due Microsoft committed to publishing findings within 14 days

Who actually felt this, and who did not?

The region boundary is the line that decided who had an outage. This was not West US going dark. It was West US becoming unreachable from outside while continuing to work internally, which is a meaningfully different failure and one that a lot of resilience planning does not model. If your app and its database both sat in West US and served users from there, you likely saw very little. If your users, your identity provider, or your storage were anywhere else, you saw everything.

RelatedThe Teen Suing Meta Over Addiction Just Dropped the Case

The detail worth flagging for security teams is that Microsoft Defender and Sentinel were both in the blast radius. When the tooling you would use to determine whether an event is a network problem or an attack terminates behind the same regional perimeter as the workloads it watches, you lose the instruments and the patient at the same moment. For the several hours before Microsoft's acknowledgement propagated, a lot of teams were staring at unreachable services with no telemetry and no way to rule out something worse.

What should teams change after this?

Microsoft's own advice is to consider a multi-region approach for mission critical data, which is correct and also the kind of guidance that has been true since before this outage. Three narrower items come out of the specifics:

• Check where your observability exits. If your monitoring, alerting and security analytics leave through the same regional perimeter as the workload, plan for them to go dark together and keep one out-of-band signal that does not.
• Audit your own maintenance automation for the same seam. If a pre-flight safety check reads the request and the executor computes its own target list, they can disagree. Bind both to one computed scope.
• Stop diagnosing from complaint dashboards. Downdetector tells you something is wrong and roughly when. It does not tell you where, and the service with the most complaints is usually just the most visible dependent.

Our take

The interesting failures at hyperscale have quietly stopped being about components and started being about the software that decides which components to touch. No device failed here. No capacity ran out. A conversion routine drew a slightly larger circle than a human had approved, and five hours of a major cloud region's external connectivity went with it. The safety check that existed to prevent exactly this outcome was not skipped, disabled, or broken. It just answered a question about a different change.

That should be a familiar shape to anyone who runs infrastructure automation, and it is a better takeaway than another round of cloud concentration commentary. Microsoft owes a final review in two weeks. The thing to read for is whether the guardrail and the action have been taught to share a definition of scope, because everything else in this incident followed from the moment they did not.

Original analysis by GenZTech. Root cause details from Microsoft's preliminary post incident review for MO1437424, as reported by BleepingComputer.