The 2026 Central-Outage Case Study on heain-core When Its Own Change Stops the Whole System Four Scenes from the Microsoft 365 Outage of 12–13 Nov 2026: a Wrong Change, Cascading Failure, Overwhelmed Recovery and Sites without a Centre
Year: 2026
Type: preprint
Abstract: On 12–13 November 2026 Microsoft 365 went down worldwide for at least about 18 hours. Five Thai banks sent customers to other channels; and Taiwanese brokers confirmed orders by fax [1]–[6]. Microsoft attributed it to "an issue during an earlier network capacity upgrade", not an attack. Lost servers pushed work onto the rest until they could not cope, and recovered servers failed again once they took requests [1][4]. On an HP Z840 the author built a 5-machine lab serving one organization's synthetic mail (a Master, three mail-sending Workers, a user machine) with no internet, and tested four scenes: M1 an honest admin's wrong change; M2 Workers dropping out one by one; M3 the Master returning to 300 and 1,000 clients at once; M4 no Master and no internet for 30 minutes. Core's code was checked first, and every scene prints what happened in the real incident (asis) beside heain's result. This paper does not simulate Microsoft's systems. On the Z840 (commit ae90f0f) all 38 checks passed with eight findings (O1–O8). What held: Workers never took more work than they asked for, no mail was sent twice even when a machine was stopped mid-job, the Master did not fall over when 1,000 clients reconnected at once, and sites worked on for 30 minutes without a centre. What did not: one setting at the Master stopped the whole service within about 6 seconds with no canary or automatic rollback; the Master accepted work without bound; and after the storm it dispatched more slowly as its queue grew, from 3.6 jobs per second at 9,300 queued to 0 at 56,046, closest to the real "recovered but overwhelmed". Every finding needs a core change and awaits the author's decision. Keywords: HEAIN Framework, heain-core, Outage, Cascading Failure, Configuration Rollout, Canary, Backpressure, Retry Storm, Thundering Herd, Load Shedding, Offline Operation, Microsoft 365
Author: BOONMA, JAKKRIT
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7229602943
