The company said the disruption, which began on Thursday, 12 November, lasted almost 24 hours before active customer impact ended on Friday. Its preliminary post-incident report, dated 15 November, traced the failure to three interacting problems: a dormant software bug, increased traffic following a Teams network switch and temporarily reduced server capacity.
At the worst point, more than 80% of servers supporting a critical networking gateway became unresponsive, preventing customers from connecting to Exchange Online, Microsoft’s cloud email and calendar platform. Failures spread internationally as automated systems redirected traffic towards remaining servers, placing those machines under additional pressure.
Microsoft said the underlying defect originated in an update first deployed in July to Windows servers dedicated to its Microsoft 365 networking gateways. The change operated worldwide for more than three months without an apparent problem before particular traffic conditions exposed an internal processing conflict that exhausted server resources.
The company acknowledged that its testing had failed to identify the vulnerability. Laboratory testing concentrated on machines with fewer processor cores, whereas the fault became more pronounced on newer, more powerful hardware used in production environments.
Approximately 15 hours before the outage, an automated failover moved Teams traffic from an alternative gateway onto the primary infrastructure also supporting Exchange Online. That increased demand and changed traffic patterns. Meanwhile, nearly 14% of the gateway’s server capacity was unavailable because of an unrelated issue from the preceding day.
Microsoft said neither the temporary capacity reduction nor the traffic switch alone would necessarily have caused the disruption. Combined with the software defect, however, they produced escalating request queues and widespread failures during peak usage in the United States.
The incident principally affected email access and delivery. Many customers could not open their mailboxes through Outlook, browser-based Outlook or other email clients, while messages accumulated in delivery queues. Calendar functions, including meeting invitations and scheduling, also encountered problems.
Features dependent on Exchange Online across Teams, Copilot, SharePoint and OneDrive were affected, although substantial portions of those services remained available. Teams chats and meetings continued functioning for much of the disruption after traffic was moved away from the affected infrastructure.
Other affected Microsoft 365 products included Defender, Purview, Intune, Planner, Power Apps, Power BI and Universal Print. The severity differed by region, access method and timing, with some users retaining intermittent connectivity while others experienced sustained failures.
Microsoft said the incident was technical and unrelated to cyberattacks or artificial intelligence agents. The defect concerned specialised gateway infrastructure, rather than ordinary Windows installations, according to its preliminary findings.
Recovery proved complicated because servers that engineers restarted could become overwhelmed again when waiting requests returned. Normal automated restoration mechanisms were not designed to recover from failures on this scale, while standard software rollback measures could not readily reverse a defect introduced months earlier.
Engineers eventually developed an update restoring the connection-handling process to its previous working configuration. They removed groups of servers from operation, applied the correction and restarted them before gradually allowing traffic to return.
The company identified an effective mitigation at 07:20 UTC on Friday. Service stability subsequently improved across affected regions, with European availability returning to healthy levels by 11:35 UTC and global webmail availability reaching expected levels at 14:17 UTC.
By 16:10 UTC on Friday, Microsoft determined that customers were no longer experiencing active disruption. Extended monitoring continued while delayed emails were processed. Final customer notifications followed on Saturday after the company confirmed sustained stability.
Microsoft apologised to customers and partners for the interruption. Charles Lamanna, its executive vice-president responsible for Copilot, Agents and Platform, acknowledged the effects on normal business operations.
The company has since added capacity, strengthened monitoring of connection handling and introduced safeguards intended to prevent automatic traffic redistribution from propagating failures. It also extended the email delivery window so delayed messages could be processed rather than returned as undeliverable.
A final post-incident report is scheduled for 20 November, when Microsoft expects to detail further corrective measures. The preliminary account identifies the affected service advisories as EX1550543 and MO1550821, covering Exchange Online and the wider Microsoft 365 platform respectively, for administrators tracking the service interruption and recovery.
Follow Arabian Post
Select Arabian Post as your preferred source on Google and MSN News for trusted business news and Arab politics and updates.