UPDATE: 5:44 pm EST 26th September 2011
Summary of issue and current status:
Visitors utilising the Telstra network for internet access to websites hosted on the Zeus Dynamic web cluster (websites resolving to 202.124.241.200 IP) are currently experiencing Timeout errors.
One of the IP addresses used for our shared hosting infrastructure (zeus-dynamic, 202.124.241.200) has been the subject of a DDOS attack. We have had to block this IP with one of our upstream providers (Telstra), in order to allow other client-facing services to be able to be kept online.
This means that some clients who are accessing their websites via Optus/PIPE connections will be able to access their services, whereas Telstra network traffic will not be able to get through to this webserver cluster. For example, TPG customers will be able to see the websites, while Bigpond customers will not.
Our engineers are working on solutions that will allow us to bring zeus-dynamic services back online, without affecting other client facing services. This work has been going on all day, and will continue overnight. At this point in time we cannot give an accurate estimate of when the Zeus Dynamic services will be brought back online via the Telstra internet service provider connections.
UPDATE: 4:18 pm EST 26th September 2011
Telstra link to Zeus Dynamic (202.124.241.200) has been shut down again temporarily. This means customers using Telstra internet connections will not be able to access websites hosted on Zeus Dynamic shared hosting infrastructure.
Further update will be posted when confirmed as resolved.
Unfortunately we cannot supply an ETA on a DDOS attack mitigation process.
UPDATE: 2:40 pm EST 26th September 2011
Engineers have confirmed the network traffic to have stabilised, and all services should operate as normal. Please note, mail sent from web applications during these attacks will be delayed in arrival to the recipients. No data was lost.
UPDATE: 2:20 pm EST 26th September 2011
There appears to be a secondary attack taking place at the moment. Services affected appear to be only the Shared infrastructure Zeus Dynamic and Apache sites, currently with no impact on shared mail, nor IIS or Zeus Static services.
TheConsole admin panel access has been reported as slow.
Further updates will be posted as information becomes available.
UPDATE: 1:23 pm EST 26th September 2011
Telstra have confirmed traffic has stabilised and HTTP access has been re-enabled on Zeus Dynamic websites as of 1:20pm. All services should now be back to normal and fully functional.
UPDATE: 12:17pm EST 26th September 2011
HTTP access to Zeus Dynamic (202.124.241.200) cluster was completely blocked temporarily to stop offending traffic. Access to the network will be re-enabled shortly following confirmation from Telstra that traffic has stabilised. Currently no ETA is available due to the nature of the event.
As mentioned previously, DDOS attacks only affect the speed/performance of service, no data will be lost.
UPDATE: 11:37am EST 26th September 2011
Please note, one service is still intermittently unavailable. Websites hosted on the shared Zeus Dynamic cluster (202.124.241.200) are still experiencing intermittent outages due to our operations surrounding blocking of the attacker. Update will be posted when fully resolved.
UPDATE: 11:10am EST 26th September 2011
Network connectivity problems have now been resolved. Connectivity is being restored across all services, and no data was lost. We apologise again for this issue, the damage from this attack was mitigated within a record time by our engineers.
UPDATE: 10:50am EST 26th September 2011
It has been confirmed by the engineers that we are currently experiencing a distributed denial of service attack (DDOS) on our network.
Unfortunately given the nature of the attack, it is difficult to confirm ETA on completion of work associated with re-diverting network bandwidth abuse. Further details will be posted as soon as available.
Please note, DDOS attacks only affect the speed/performance of service, no data will be lost.
Original notice: 10:30am EST GMT +1000
TPP Internet Technical Services team would like to acknowledge and advise of a current network connectivity problem within our client facing data centre facilities.
Date and time: connectivity problems commenced at 10:15AM EST (GMT +1000)
Impact: Customers on shared and VPS hosting will experience intermittent connectivity problems to services. Websites and email connections will throw Timeout errors.
Services affected: All services reliant on network connectivity. No data will be lost.
We apologise for inconvenience caused and will post an update as soon as permanently resolved.
{ 0 comments }
