Emergency Maintenance and Initial Network/Power Event Overview

Dear M5Hosting Customer,
As you may be aware, your server was rebooted today due to a momentary power loss at the data center facility (American Internet Services) from which M5Hosting provides your service. The root cause was a failed power system component during a scheduled generator test.

After the power was restored, M5Hosting also had some network issues due to some redundant network components behaving in an unexpected manner.

The failed power system components have been replaced and power will be taken off generator and back on the "street" power tonight during an emergency maintenance window declared by the data center facility. This maintenance window will begin at 8:30pm US/Pacific Time. While the window has been declared to be 2hrs, the actual cutover will be quite short. The window will allow for testing. This work is not expected to impact service at all.

I have pasted the most recent announcement from the data center facility below. We will also forward any subsequent announcements from them regarding this incident.

While the network and facility are fully functional and stable at this time, there is some work to do return all systems back to full redundancy and normal operating status. The maintenance discussed below will be the first step. This will return the power system. Our network work will take place during a late night maintenance window to be announced later and scheduled to mitigate any risks of an impact to service.

If you are still experiencing any issues with your server or other services which we provide for you at this time, please contact us at 877-344-4678 or submit a support ticket, so we take care of your needs immediately.

Sincerely,
Mike

The most recent Data Center facility announcement from AIS:

To Our Valued Customers,


Following this morning's facility issue at the 9725 Scranton Data Center, AIS Facilities and Engineering teams are announcing a 2 hour emergency maintenance window which will begin at 17:15 and which will conclude at or before 20:15.



During this window, main power for the facility will be returned from generator to normal operating mode. Throughout the day, engineering and mechanical teams have taken every precaution and performed considerable diagnostics to ensure that all systems are once again operating at 100% effectiveness. Even following these precautions, all maintenance windows involving critical power systems still carry the risk of causing of a power event during the maintenance operation, thus, we are bringing this important issue to your attention at this time. While maintaining a cautious posture, we at AIS do not anticipate any issues during this window.


During the maintenance window, the facility will remain fully staffed by 18 customer service engineers, 3 different mechanical engineering contractors, and AIS senior management personnel.


We are in the process of producing a detailed post incident report which will be made available during business hours tomorrow, Thursday March 19th.


A follow up communication will be sent upon completion of the maintenance window. We apologize for any inconvenience this issue may have caused.


AIS Support Services