Tuesday, May 13, 2014

Issues at Manchester TCW



10:15 Our monitoring systems are currently alerting multiple failures at Manchester TCW.  Engineers are investigating and we will have more information shortly.  The nature of the failures do not indicate a single item failure.  We are waiting feedback from TCW remote hands before deciding our next course of action however engineers are prepping to go onsite.

10:27 Feedback from remote hands is our racks appear to be drenched in water.  Our Technical Manager is on his way to asses the damage and formulate workarounds where possible.

12:00 On-Site

12:50 Damaged equipment has been replaced.  A misconfiguration on a vendor side network has also been detected which will need rectifying - if the misconfiguration had not been present the impact on our clients would not have been so great.   Except for when we correct the configuration issue (which will not be today) we do not expect any further downtime.   Our sincere apologies for the downtime caused.


No comments: