During the course of this week we are increasing our switchport capacity at IFL2 and TCW. We are also taking this oppurtunity to relocate some equipment to increase our internal redundancy and resilience.
While the installations and relocations take place there will be points in time where some circuits and systems will be deemed at risk, however traffic at these points will be manually set to traverse through alternate pathways and routers.
----
Dec 3rd update.
IFL2 Complete.
----
Dec 8th update.
Due to delays in getting some fibre links provisioned at TCW this site will be delayed, new circuits should be in by Dec 18th.
Monday, November 30, 2009
Monday, October 19, 2009
Core Network
We currently experiencing problems across multiple links of the core network; our providers have been notified and we are waiting for an update.
Update 10:09; we've been told that the problems are down to a major incident in London which is affecting multiple parties, unfortunately it is of a scale which covers both our London based datacentres.
Update 10:18; THN now appears to be stable, all our connections to RBHX are now down (rather than flapping)
Update 10:47; we've now seen RBHX come back online, though no direct confirmation yet from our provider.
Update 13:45; we've just seen a blip on all our connections at RBHX, approx 1 minute.
For the near future services should still be considered at risk.
Update 10:09; we've been told that the problems are down to a major incident in London which is affecting multiple parties, unfortunately it is of a scale which covers both our London based datacentres.
Update 10:18; THN now appears to be stable, all our connections to RBHX are now down (rather than flapping)
Update 10:47; we've now seen RBHX come back online, though no direct confirmation yet from our provider.
Update 13:45; we've just seen a blip on all our connections at RBHX, approx 1 minute.
For the near future services should still be considered at risk.
Monday, September 28, 2009
Manchester Transit HSRP Problems
One of a pair of Cisco routers serving some transit customers is experiencing issues, currently it is not passing packets. This should have caused an automatic fail over to it's pair however for reasons currently unknown it did not fail over. The standby priority for all affected customers has been increased on the operational router so traffic is once again flowing.
During normal operation both routers would carry traffic, some customers having a higher priority on router 1 while the others have a higher priority on router 2. A number of scenarios are tested to ensure fail over does occur so this failure is a bit unusual.
An engineer is currently en-route to verify the status of the router experiencing problems.
During normal operation both routers would carry traffic, some customers having a higher priority on router 1 while the others have a higher priority on router 2. A number of scenarios are tested to ensure fail over does occur so this failure is a bit unusual.
An engineer is currently en-route to verify the status of the router experiencing problems.
Unexpected Reboot of rtr1.thn
At 17:03 rtr1.thn unexpectedly rebooted, customers served via this router would have noticed approx 7 minutes of downtime while the router reloaded.
The router appears to be stable after the reboot though reasons for why it may have rebooted are still being investigated. We will continue to monitor the router closely for the next few hours.
The router appears to be stable after the reboot though reasons for why it may have rebooted are still being investigated. We will continue to monitor the router closely for the next few hours.
Saturday, September 19, 2009
Transit router
At 17:33 one of our transit routers stopped passing packets on one of it's interfaces. The affected router was restarted at 18:00 correcting the problem.
Customers directly connected to this router would have seen an outage of 32 minutes. Others may have experienced an outage of approx 30 seconds while their sessions were redirected to another router.
C2 apologise for the inconvenience this outage may have caused.
Customers directly connected to this router would have seen an outage of 32 minutes. Others may have experienced an outage of approx 30 seconds while their sessions were redirected to another router.
C2 apologise for the inconvenience this outage may have caused.
Thursday, June 18, 2009
Scheduled maintenance 9th June 04:00 and 06:00.
Hi,
Following on from the Network incident report for 9th June, we are scheduling a network maintenance window for Sunday 21 June that will be open between 04:00 and 06:00. The purpose of this window will be to investigate and test the stability of the Manchester ring.
We are not expecting the network to experience any disruption in service, but the Manchester ring will obviously be more at risk of interruption while testing takes place.
Kind Regards
Stuart McKindley
Subscribe to:
Posts (Atom)