Spectacular DDoS on one of my hosts systems
Posted: Sun Dec 23, 2007 11:07 pm
I just go this email a few hours ago. Was this a strategic and powerful attack or what?
Re: Network Outage - December 20, 2007 - Incident Update
Dear Customer,
We experienced a major network attack starting 08:30 PM on December 20, 2007, and resulted in a network outage that lasted for 3 hours.
The outage originated with a massive DDOS (Distributed Denial of Service) attack against one of our web servers.
At this point, we were seeing 50-80% packet loss to our network, and the origin/destination of attack was being investigated.
Quite soon, the attack went up to 800,000 pps (packet per second) and 500 megabit per second steady almost exhausting our channels to our upstreams.
The attack died by itself for about 4 minutes and returned with almost double the capacity at 1 million packets per second and sustained 900mbps on inbound data. This even went up intermittently to 1.5 million packets per second and 1.3 Gigabit per second of traffic.
Due to this huge inflow of traffic, our local blocks to mitigate this attack weren't successful and we had to seek help from our ISPs. This added to the resolution time since we had to liaison with multiple external units to get destination identified and an appropriate block instated. At approx 10:10PM, we were able to get the destination identified and block was immediately placed at our ISPs. However, despite being physically redundant, our core fiber link didn't come up even when the inbound traffic had settled to our normal rate. This added another hour of outage while we coordinated with the fiber team to get the fault located and corrected. This fault was related to the massive inflow on traffic during the DDOS.
For business continuity, we plan to add an alternative redundant fiber link in addition to this our current (redundant) fiber link before end of first quarter of 2008. Other suitable measures that can help in such cases will also be considered and implemented.
As per the last update, the DDOS attack was still on at sustained 400mbps of traffic, however being blocked far from our network, we continue to run safe and fine.
The server being attacked has also been successfully migrated to a new IP Address, allowing our customers to run their website without being affected anymore with the issue.
Thank you for your patience and cooperation throughout the issue.
We would also like to take this moment to wish you and yours a very Merry Christmas, a great New Year, and safe travel during this holiday season.
Regards,
Customer Support