Engineers wrangle 55 TB restore and traffic replay as millions of messages queue up
September 3, 2025 By Richard Speed
A RAID failure has taken the Matrix.org homeserver offline, leaving users of the decentralized messaging service unable to send or receive messages while engineers attempt a 55 TB database restore.
To be clear, those with their own homeservers, such as government organizations, are unaffected, but anyone using Matrix.org as their homeserver will have been hearing the sound of silence from the platform while the team works to bring the service back online.
Problems began at 1117 UTC on September 2, when the secondary Matrix.org database lost its file system due to a RAID failure. The primary fell over at 1726 UTC, and a few minutes later, the organization admitted that things were indeed not very healthy.
The Matrix.org homeserver is backed by a large PostgreSQL database, which caused the organization grief in July when a long-gestating corruption of part of a table index caused issues with "rooms" in the system. The result was that attempts to join rooms would fail, messages wouldn't send, and occasional cryptic error messages would appear.
