Urgent "Failover" to redundant core switch @ 2am

A probable bug in the Cisco IOS running on the current active layer 3 core switch is preventing us from remotely managing the device. We have exhausted the options for restoring the functionality without interrupting the traffic flow through it.
Tonight, after 02:00 PDT we will force the redundant layer 3 core switch to take over while we address the issue with the current primary switch. The cut-over to the redundant switch should only take a few seconds. If we fail-back to the current primary within a short time, it too should only take a few seconds.

Date/Time: Thursday 8/21/08 @ 02:00 US/Pacific (GMT -7)
Duration: 5 to 30 seconds (1 minute "worst case")

As always, I invite your comments on this message, or the events it describes. Your feedback helps us deliver better services to you.

Thanks,
Mike

Hardware failure in shared hosting server "Witt"

The shared hosting server called witt.m5hosting.com suffered a disk failure. To replace the disk required a few minutes of downtime. This was performed shortly after midnight tonight. The server will be a little slow at disk intensive tasks for a few hours while the array is rebuilt and the disks are synchronized.

Update 8/14/08 17:00 PDT: Disk rebuild is complete. System IO tasks running at normal speed.

Intermittent Issues with Level 3 Communications Link

We have had a few reports, confirmed by the data center facility and our own monitoring systems, that routes through Level3 Communications were intermittently "flapping" from 13:31 to 13:58 PDT.

We have asked the data center to keep us updated with anything they may find, as they are the direct Level 3 customer and it is though the data center facility's network that we are connected to Level 3 Communications.

If you have a traceroute or MTR information which shows the issue, please send them to support at m5hosting.com

Thanks !

Network Maintenance - 02:00 to 06:00 Aug 10th, 2008

We will be making some nifty upgrades to our network this weekend. The maintenance was previously announced here.

We will be replacing our border routers, and completing some VLAN segmentation work that remained uncompleted from a previous maintenance window.