Skip to content
Theme
OnDevice Tools

WindowsMicrosoft 365 & Office

Microsoft 365 outage: 80% of servers failed, Microsoft says

Microsoft has explained a major Microsoft 365 email outage, saying a July change and a gateway failure left over 80% of servers unresponsive.

Illustration of Microsoft 365 servers failing during a network gateway outage

Key takeaways

  • Microsoft said the outage left over 80% of servers unresponsive.
  • The failure involved two network gateway systems and a hidden July change.
  • Microsoft said normal load testing missed the issue because lab hardware differed.
  • Microsoft said the outage was technical and not a cyber attack.
In this article
  1. What happened
  2. Two technical failures combined
  3. Why testing missed it
  4. Microsoft response and user reaction
  5. What it means for cloud services

What happened

Microsoft 365 suffered a major outage last week, according to 9to5 Mac. Many users could not send or receive email while the service was disrupted. The report said a large number of users were affected.

Microsoft has now said the problem was technical, not a cyber attack. At the worst point, unresponsive servers accounted for over 80% of the fleet. The company also said the repair took more time than expected.

Microsoft said engineers were working to restore normal service. It said the issue was not a security incident. It said the cause was a technical fault. The company has now published an explanation of what went wrong.

Two technical failures combined

Microsoft said two separate issues came together. The company runs two independent network gateway systems. One gateway failed, so the other had to carry all traffic.

Microsoft said each gateway has enough capacity to handle the full load on its own. That meant the failure of one gateway would not normally be enough to cause a major outage. However, a mistake made in July created a hidden fault.

The mistake was not detected at the time. The hidden fault caused a cascading failure when the remaining gateway was under pressure. Microsoft said an internal process suffered from lock contention, which could make a machine unavailable under some load conditions. The problem came from a fix intended for the Microsoft 365 gateway servers.

Why testing missed it

The problematic change began rolling out in July. Microsoft said it passed a safe deployment process and reached global coverage over several weeks. For more than three months, the company saw no impact.

Microsoft said normal load testing did not reveal the hidden issue. Its test labs used different traffic and hardware mixes from the live fleet. The contention problem was worse on newer machines with more processor cores.

The company said its load testing tended to focus on less powerful hardware models with fewer cores. This meant the update was not tested under the conditions where it was most likely to fail. The report highlights a gap between lab testing and live service conditions.

Microsoft response and user reaction

Charles Lamanna, a platform head at Microsoft, admitted the severity of the incident. He said the previously unknown defect spread across the fleet in less than two hours. He said the problem made servers unresponsive at a rapid pace.

Microsoft denied that the outage was connected to recent staff cuts. The company said the incident was not a cyber attack. It also said the problem was unrelated to staff reductions and the replacement of staff with AI systems.

Users were unhappy with the scale and duration of the disruption. The outage also produced a wave of online jokes. Some users posted memes about the failure. The episode shows why clear service updates matter during a major cloud failure.

What it means for cloud services

Illustration: cloud, Microsoft logo, outage, email and the labels Load Testing, Microsoft, Microsoft 365

The report describes a failure caused by a small undetected change meeting an unexpected load condition. Microsoft said the July update was stable for months before the outage. That makes the incident a reminder that cloud testing must cover live traffic and varied hardware.

The outage affected email, a core part of Microsoft 365 for many users. It also showed that backup capacity alone may not prevent failure if another hidden fault is present. The Microsoft explanation points to the need for testing that matches real-world conditions.

For organisations that rely on cloud email, the report shows how quickly a service problem can spread. It also shows why providers need to watch newer hardware as well as older systems. The facts released so far focus on the technical cause and the scale of the server failure. The account is based on the Microsoft explanation.

Sources

  1. Microsoft outage saw 80% of servers fail – here’s what happened9to5 Mac

Tags

Spotted a mistake? See our corrections policy. Read how News by OnDevice works in our editorial policy.