Page 1 of 1

Spectacular DDoS on one of my hosts systems

Posted: Sun Dec 23, 2007 11:07 pm
by RobertGonzalez
I just go this email a few hours ago. Was this a strategic and powerful attack or what?
Re: Network Outage - December 20, 2007 - Incident Update

Dear Customer,

We experienced a major network attack starting 08:30 PM on December 20, 2007, and resulted in a network outage that lasted for 3 hours.
The outage originated with a massive DDOS (Distributed Denial of Service) attack against one of our web servers.
At this point, we were seeing 50-80% packet loss to our network, and the origin/destination of attack was being investigated.
Quite soon, the attack went up to 800,000 pps (packet per second) and 500 megabit per second steady almost exhausting our channels to our upstreams.
The attack died by itself for about 4 minutes and returned with almost double the capacity at 1 million packets per second and sustained 900mbps on inbound data. This even went up intermittently to 1.5 million packets per second and 1.3 Gigabit per second of traffic.
Due to this huge inflow of traffic, our local blocks to mitigate this attack weren't successful and we had to seek help from our ISPs. This added to the resolution time since we had to liaison with multiple external units to get destination identified and an appropriate block instated. At approx 10:10PM, we were able to get the destination identified and block was immediately placed at our ISPs. However, despite being physically redundant, our core fiber link didn't come up even when the inbound traffic had settled to our normal rate. This added another hour of outage while we coordinated with the fiber team to get the fault located and corrected. This fault was related to the massive inflow on traffic during the DDOS.

For business continuity, we plan to add an alternative redundant fiber link in addition to this our current (redundant) fiber link before end of first quarter of 2008. Other suitable measures that can help in such cases will also be considered and implemented.

As per the last update, the DDOS attack was still on at sustained 400mbps of traffic, however being blocked far from our network, we continue to run safe and fine.
The server being attacked has also been successfully migrated to a new IP Address, allowing our customers to run their website without being affected anymore with the issue.

Thank you for your patience and cooperation throughout the issue.
We would also like to take this moment to wish you and yours a very Merry Christmas, a great New Year, and safe travel during this holiday season.

Regards,
Customer Support

Posted: Mon Dec 24, 2007 12:45 am
by matthijs
1.5 million packets per second and 1.3 Gigabit per second of traffic
Wow, that data center must have been smoking 8O

Posted: Mon Dec 24, 2007 12:48 am
by RobertGonzalez
Yeah, the numbers seemed really frickin' high to me.

Posted: Mon Dec 24, 2007 2:33 am
by alex.barylski
Everah wrote:Yeah, the numbers seemed really frickin' high to me.
For a hosting company...I'm not sure thats high...the last shared hosting company I was with claimed to serve a 15 tera-bytes a month...and they piggy backed on someone elses servers.

I'm thinking most big data centers are probably into the peta-bytes and beyond...what that averages out to in seconds...who knows. :P

It would be interesting to see their traffic charts in a line graph...just to see what the norm is...cause without telling you that...any numbers shown to you could be just smoke and mirrors.

Posted: Mon Dec 24, 2007 4:22 am
by Benjamin
And who was on that IP?

Posted: Mon Dec 24, 2007 4:54 am
by Jenk
The problem with gaging size of DDoS attacks if relevance of size, so if your host is someone the size of GoDaddy then that attack wouldn't be considered that big, but if it were a small independent hosting whom only have one rack of rented datacenter floorspace, then it would be friggin huge :)

Posted: Mon Dec 24, 2007 5:31 am
by s.dot
Must've took a lot of machines to pull that one off.

Posted: Mon Dec 24, 2007 8:58 am
by John Cartwright
Atleast you have some good support guys! Which host you using Everah?

Posted: Mon Dec 24, 2007 9:26 am
by RobertGonzalez
This was my JodoHost account.