Packets Over 90,000 | TechSNAP 102

chris

9 years ago

A CloudFlare outage takes down three quarter of a million sites, we’ll tell you what went wrong.

Some old school malware gets the job done, Allan’s cool toys from Japan, a big batch of your questions our answers, and much more on this week’s TechSNAP.

Thanks to:

GoDaddy.com

Use our code hostdeal4 to score economy hosting for $1 a month, for one year.

35% off your ENTIRE order just use our code go35off4 until the end of the month!

Ting.com

Visit techsnap.ting.com to save $25 off your device or service credits.

Direct Download:

RSS Feeds:

Support the Show:

Purchase anything at Amazon.com
Purchase anything at Think Geek using this link
Purchase anything at Newegg using this link
Contribute directly on our donation page or check out our advertising options and reach millions of amazing people!
Grab our Chrome Extension and auto-tag your shopping session for us!

Show Notes:

Get TechSNAP on your Android:

Browser Affiliate Extension:

Jupiter Broadcasting Affiliate Extensions for Chrome and Firefox

CloudFlare outage takes down reported 750,000 sites

CloudFlare is an online WAF (Web Application Firewall) and CDN
We have talked about outages they have had in the past, but this one is different
Sites went down on March 3rd at 09:47 UTC (4:47am EST, 1:47am PST), the sites, DNS and other services all returned a ‘no route to host’ error
Service was restored at 10:49 – a total of just over an hour
Unlike some previous outages, this took down all 23 of their data centers
According to CloudFlare, they originally detected a DDoS attack against one or more customers’ DNS servers, and moved to filter that attack
Their attack profiling software reported that the attack packets were between 99,971 and 99,985 bytes
This is actually impossible, largest possible IP (Layer 3) packet is 65535 bytes, due to the size of the field that holds the length of the packet in the header (16 bits)
Also, most layer 2 protocols (like Ethernet) also impose a smaller frame size, the default for Ethernet is 1500 bytes. The CloudFlare network is configured to use ‘jumbo frames’ with a maximum packet size of 4470 bytes
When they used the Juniper Flowspec protocol to push a rule to all of their routers to block packets of the impossible size, the routers accepted the rules, but then started exhausting all of their ram, probably due to a bug related to the fact that the rule was invalid
This caused many of the routers to reboot, but then they would load the same rules and just reboot again, in a continuous loop
Some of the routers also stopped responding on their management interfaces, requiring that the devices be manually power cycled (in the 23 different data centers in 14 countries)
When the routers were down, the routes would be withdrawn/expired, and it ended up that there were no routes left to allow anyone to reach the CloudFlare network
CloudFlare is working with Juniper to see if this is a bug in Flowspec, or if it was specific to the rules and traffic of CloudFlare
CloudFlare is also attempting to find a way to be able to use flowspec to push rules only to a specific subset of their routers (those under attack, or those in a specific region), so that next time it doesn’t take down their entire network at once
Official Post Mortem

Malware doesn’t have to be advanced to be effective

Researchers have discovered a malware attack that has been ongoing for 10 years using off-the-shelf remote administration software TeamViewer along with proprietary malware
The attackers managed to exfiltrate encryption keys and ‘secret’ documents from a number of sensitive places, including an unidentified NATO/EU member country’s embassy and an industrial manufacturer in Russia, multiple research and educational institutions in France and Belgium, and a high-profile Hungarian governmental victim
Kaspersky’s research indicated the threat actors were most likely Russian, because the malware scripts changed the character set to CP1251 and used the Russian localized version of TimeViewer. They also found some possible links to the Red October malware
CrySyS.hu Report
Kaspersky research paper

BBC twitter accounts hacked by Pro-Assad Syrians

The twitter accounts for BBC Weather, BBC Arabic and BBC Ulster Radio were compromised and posted unauthorized tweets
“The attacks began in the early afternoon on Thursday. At the same time, BBC staff were alerted to a phishing email that had been sent to some BBC email accounts. It is not yet clear if the two are related”
The attack was apparently the work of the “Syrian Electronic Army”, a group that has previously shown strong support for Syrian President Bashar-al-Assad
Humorous tweets were posted, such as:
- “Saudi weather station down due to head-on collision with camel.”
- “Chaotic weather forecast for Lebanon as the government decides to distance itself from the Milky Way.”
The accounts are now back under the control of the BBC
It is not clear how the accounts were compromised

Feedback:

[asa]B0070I17LQ[/asa]