GoDaddy’s services started to drop off of the internet
The outage lasted approximately 6 hours, from 10:00 PDT (17:00 UTC) and being fully restored about 16:00 PDT (23:00 UTC)
A twitter account, claiming to represent part of Anonymous, took responsibility, claiming to have launched a massive DDoS attack against GoDaddy
Some news outlets and blogs misunderstand what a DDoS attack is, and report that Anonymous has hacked GoDaddy
“We have determined the service outage was due to a series of internal network events that corrupted router data tables.” – Interim Godaddy CEO Scott Wagner
The issue was compounded because the downtime affected not only GoDaddy hosting customers, but also customers that only used GoDaddy for DNS
GoDaddy hosts 5 million web sites and manages a total of 52 million domain names
For example, the DNS for jupiterbroadcasting.com is hosted at GoDaddy, while the actual site resides at ScaleEngine, but because the DNS was down, viewers were unable to lookup the IP address of jupiterbroadcasting.com in order to connect to ScaleEngine
DNS caching will have helped reduce the effect of this downtime somewhat, especially for more popular sites, and for users coming from larger ISPs, the DNS records for JB have a TTL of 1 day, so users would only have issues reaching the site if the records had not yet been cached, or once the cache expired. At the time of this writing, the records for JB still had 28461 seconds left in my local Google Public DNS cache, but we not cached at my local OpenDNS
This event ruined GoDaddy’s previous 99.999% uptime record for DNS (99.999%, or 5 nines as it is called in the industry, allows for only 6 minutes of cumulative downtime in an entire year, compared to 4 nines, which allows about 53 minutes of downtime per year, or 99.9% which is nearly 9 hours)
GoDaddy uses Anycast for the DNS servers, this means that while it looks like each domain is only assigned to 2 DNS servers, each of those two IP addresses actually exists in multiple data centers around the world. Traffic is routed to the closest server, and if that servers route fails, after a few minutes the BGP routers at your ISP or an intervening transit provider route the traffic to the next closest server
However, due to what I assume was some human error after the failure of one or more network components, the routes that GoDaddy broadcasted to their upstream providers were in some way incorrect, and caused traffic to no longer reach the GoDaddy servers
Anycast is commonly used for DNS but is not very often used for TCP based services due to the fact that the routes can change at any time, and suddenly the same IP address points to a different server, and your connection is dropped. There are some cases where people have successfully used Anycast for short lived TCP connections
Security researcher David Schuetz was analyzing the the data posted online, and found an unusually large number of devices that mentioned Blue Toad, 19 out of the 1 million records analyzed
Schuetz then contacted Blue Toad to report what he had found
Schuetz also said he couldn’t say conclusively if Anonymous’ claims about the FBI were false or true
Blue Toad makes apps for publishing companies, long known for collecting extensive data about their readers for market research and marketing purposes
Paul DeHart, CEO of Blue Toad said his firm would not be contacting individual consumers to notify them that their information had been compromised, instead leaving it up to individual publishers to contact readers as they see fit
The company’s forensic analysis claims to show the data had been stolen “in the past two weeks”
This is contrary to the original claim that the data was stolen from an FBI computer months ago
Are you a PHP programmer with time to spare for an open source security project? I am considering building an open source web service to handle sensitive information disclosure, and I am looking for some help building the site and API. email allan@jupiterbroadcasting.com