Rackspace hosted Exchange suffered a catastrophic interruption starting December 2, 2022 and is still continuous since 12:37 AM December 4th. Initially referred to as connectivity and login issues, the guidance was ultimately upgraded to announce that they were handling a security occurrence.
Rackspace Hosted Exchange Issues
The Rackspace system went down in the early morning hours of December 2, 2022. At first there was no word from Rackspace about what the problem was, much less an ETA of when it would be fixed.
Customers on Buy Twitter Verified reported that Rackspace was not responding to support e-mails.
This has actually been rather the day with #Rackspace. Every hosted exchange client has actually been down for 14 hours or so. Assistance isn’t reading/responding to tickets. Updates are unhelpful.
I am concerned now that they succumbed to something bad like the ProxyNotShell PoC hack. https://t.co/jchKsAO3Z7
— Joe Sinkwitz (@CygnusSEO) December 2, 2022
A Rackspace consumer independently messaged me over social media on Friday to relate their experience:
“All hosted Exchange customers down over the previous 16 hours.
Uncertain how many companies that is, but it’s considerable.
They’re serving a 554 long hold-up bounce so individuals emailing in aren’t familiar with the bounce for numerous hours.”
The official Rackspace status page offered a running update of the failure but the initial posts had no details aside from there was a failure and it was being examined.
The first authorities upgrade was on December 2nd at 2:49 AM:
“We are examining a concern that is impacting our Hosted Exchange environments. More information will be posted as they appear.”
Thirteen minutes later on Rackspace began calling it a “connectivity concern.”
“We are investigating reports of connectivity problems to our Exchange environments.
Users may experience an error upon accessing the Outlook Web App (Webmail) and syncing their email customer(s).”
By 6:36 AM the Rackspace updates explained the continuous problem as “connection and login issues” then later that afternoon at 1:54 PM Rackspace revealed they were still in the “examination stage” of the interruption, still trying to find out what failed.
And they were still calling it “connectivity and login problems” in their Cloud Workplace environments at 4:51 PM that afternoon.
Rackspace Recommends Moving to Microsoft 365
Four hours later Rackspace referred to the circumstance as a “substantial failure”and started using their consumers complimentary Microsoft Exchange Plan 1 licenses on Microsoft 365 as a workaround till they comprehended the problem and could bring the system back online.
The main assistance mentioned:
“We experienced a considerable failure in our Hosted Exchange environment. We proactively closed down the environment to prevent any more concerns while we continue work to bring back service. As we continue to resolve the root cause of the concern, we have an alternate option that will re-activate your ability to send out and receive e-mails.
At no cost to you, we will be offering you access to Microsoft Exchange Plan 1 licenses on Microsoft 365 until more notice.”
Rackspace Hosted Exchange Security Incident
It was not until almost 24 hr later at 1:57 AM on December 3rd that Rackspace formally announced that their hosted Exchange service was experiencing a security occurrence.
The announcement further revealed that the Rackspace technicians had powered down and detached the Exchange environment.
“After additional analysis, we have identified that this is a security event.
The known effect is isolated to a portion of our Hosted Exchange platform. We are taking required actions to evaluate and secure our environments.”
Twelve hours later that afternoon they upgraded the status page with more details that their security team and outside specialists were still dealing with resolving the interruption.
Was Rackspace Service Impacted by a Vulnerability?
Rackspace has not launched information of the security occasion.
A security occasion generally includes a vulnerability and there are 2 extreme vulnerabilities presently in the wile that were patched in November 2022.
These are the 2 most existing vulnerabilities:
Microsoft Exchange Server Server-Side Demand Forgery (SSRF) Vulnerability
A Server Side Request Forgery (SSRF) attack allows a hacker to read and change data on the server.
Microsoft Exchange Server Remote Code Execution Vulnerability
A Remote Code Execution Vulnerability is one in which an attacker is able to run malicious code on a server.
An advisory published in October 2022 described the effect of the vulnerabilities:
“A confirmed remote attacker can carry out SSRF attacks to escalate advantages and perform arbtirary PowerShell code on susceptible Microsoft Exchange servers.
As the attack is targeted against Microsoft Exchange Mailbox server, the enemy can possibly access to other resources through lateral movement into Exchange and Active Directory site environments.”
The Rackspace failure updates have not indicated what the specific issue was, only that it was a security incident.
The most present status update since December fourth mentioned that the service is still down and clients are encouraged to migrate to the Microsoft 365 service.
Rackspace published the following on December 4, 2022 at 12:37 AM:
“We continue to make development in attending to the incident. The accessibility of your service and security of your information is of high significance.
We have actually committed substantial internal resources and engaged world-class external competence in our efforts to lessen negative impacts to consumers.”
It’s possible that the above kept in mind vulnerabilities belong to the security incident affecting the Rackspace Hosted Exchange service.
There has actually been no statement of whether customer information has been compromised. This occasion is still ongoing.
Featured image by Best SMM Panel/Orn Rin