phones – Jupiter Broadcasting https://www.jupiterbroadcasting.com Open Source Entertainment, on Demand. Fri, 28 Sep 2018 17:15:49 +0000 en-US hourly 1 https://wordpress.org/?v=5.5.3 https://original.jupiterbroadcasting.net/wp-content/uploads/2019/04/cropped-favicon-32x32.png phones – Jupiter Broadcasting https://www.jupiterbroadcasting.com 32 32 Not Dead Yet | User Error 49 https://original.jupiterbroadcasting.net/127346/not-dead-yet-user-error-49/ Fri, 28 Sep 2018 09:15:49 +0000 https://original.jupiterbroadcasting.net/?p=127346 Show Notes: error.show/49

The post Not Dead Yet | User Error 49 first appeared on Jupiter Broadcasting.

]]>

Show Notes: error.show/49

The post Not Dead Yet | User Error 49 first appeared on Jupiter Broadcasting.

]]>
Oracle Kiss and Tell | TTT 230 https://original.jupiterbroadcasting.net/93056/oracle-kiss-and-tell-ttt-230/ Tue, 26 Jan 2016 11:46:40 +0000 https://original.jupiterbroadcasting.net/?p=93056 Microsoft is releases an iOS keyboard, Amazon is looking for a bigger piece of the Android pie & we finally find out Google’s Android revenue. Plus Uber tracks it’s drivers to see if they’re speeding, our time-saving kickstarter of the week & more! Direct Download: MP3 Audio | OGG Audio | Video | HD Video […]

The post Oracle Kiss and Tell | TTT 230 first appeared on Jupiter Broadcasting.

]]>

post thumbnail

Microsoft is releases an iOS keyboard, Amazon is looking for a bigger piece of the Android pie & we finally find out Google’s Android revenue. Plus Uber tracks it’s drivers to see if they’re speeding, our time-saving kickstarter of the week & more!

Direct Download:

MP3 Audio | OGG Audio | Video | HD Video | Torrent | YouTube

RSS Feeds:

MP3 Feed | OGG Feed | iTunes Feed | Video Feed | Video Feed | Torrent Feed

Become a supporter on Patreon

Patreon

Show Notes:

The post Oracle Kiss and Tell | TTT 230 first appeared on Jupiter Broadcasting.

]]>
Is that a server in your pocket? | LINUX Unplugged 128 https://original.jupiterbroadcasting.net/92786/is-that-a-server-in-your-pocket-lup-128/ Tue, 19 Jan 2016 21:14:18 +0000 https://original.jupiterbroadcasting.net/?p=92786 This week we dive into what the community thinks about putting a server in their pocket, show you some smart tricks with Gimp & some Windows nightmares. Plus some router chat & more! Thanks to: Get Paid to Write for DigitalOcean Direct Download: MP3 Audio | OGG Audio | Video | HD Video | Torrent […]

The post Is that a server in your pocket? | LINUX Unplugged 128 first appeared on Jupiter Broadcasting.

]]>

post thumbnail

This week we dive into what the community thinks about putting a server in their pocket, show you some smart tricks with Gimp & some Windows nightmares. Plus some router chat & more!

Thanks to:

Ting


DigitalOcean


Linux Academy

Direct Download:

MP3 Audio | OGG Audio | Video | HD Video | Torrent | YouTube

RSS Feeds:

MP3 Feed | OGG Feed | iTunes Feed | Video Feed | Torrent Feed | WebM Torrent Feed

Become a supporter on Patreon:

Patreon

Show Notes:

Pre-Show:

Smart Gimp Tricks

The Quickmask for adjusting selections: I’m a born-again Quickmask evangelist, because I went so long without realizing that it was there, and it makes selections so much easier.

Decomposing an image and using the components as a mask to select part of an image—that’s an easy way to select skies and get rid of a drab overcast sky, or to change or enhance the color of the sky.

Using the Dodge/Burn tool to make a background really white, for when you want an object to stand out and make the background go away. It can be a lot easier than selecting or erasing.

Feedback:

Noah Hit Something on the Head

“I wish Ubuntu would just use GNOME, and go back to trying to contribute useful bits.”

ScreenShot

Now, I have a few extensions installed, but the way I use GNOME isn’t much different from how someone would use Unity; I keep the Dash up at all times, I have those pesky tray apps up in the corner, and so on. This is so close to how I used to work in Unity, OS X, etc., that I’m puzzled at why GNOME gets treated like the red-headed stepchild sometimes.

Now, I know that once in a while the GNOME dev team decides to go off the deep end (Let’s make the filemanager work like old-school Finder and do that by default! Hell, let’s override users’ pre-existing settings, that’s how good it is!) but overall GNOME just keeps getting better.

Windows Secureboot Causes a Mess

I just made a potentially costly mistake: we nuked and repaved my friend’s brand new Lenovo Yoga 500 with Linux… without booting into Windows to disable SecureBoot explicitly. He didn’t want to accept the EULA; and we successfully booted into USB key (Ubuntu MATE 15.10, Ubuntu w/Unity 15.04) so we thought we would just go ahead.

I thought Ubuntu would have been candidate, but apparently not. Is this the correct way? Which distros would work? We tried installing and booting into Ubuntu Unity 15.04 (which is supposed to have the appropriate signature) but after install and reboot, we get the above. Given that Windows has been obliterated at this point, what options do I have?

Unfortunately, he’s leaving in just over a week, so if the conclusion ends up being “install Windows to do this” I’ll take it….. but I’d rather not!

Ever Heard of FriendOS?

I heard yesterday about Friendos. It looks like this: https://youtu.be/Y5n0f5DSbSM?t=16m14s , so it’s like Amiga Workbench in the browser powered by a Linux backend. They releasing public beta as open source this week.It will be able to run both thml5 and native applications. Their website is a little enigmatic right now:

Rover Log – Live Tracker

Live map of the adventures of Jupiter Broadcasting’s Rover Studio.

TING

A Server in Your Pocket

Ocean is a mobile server, a device that combines the portability of a mobile phone with the flexibility of a Linux web server.

Want a portable Linux-powered web server that will fit into your pocket? Look no further than Ocean.

Ocean has been designed from the ground up for portability, and features an integrated battery that allows you to run web and Bluetooth applications in places where direct power is limited.

The device is approximately the size of an iPhone 6, and can easily fit in your pocket.

This bundle costs $149 and ships in February. Higher capacity versions are slated to ship later in the year.

DigitalOcean

Numbers don’t lie—it’s time to build your own router

I’ve noticed a trend lately. Rather than replacing a router when it literally stops working, I’ve needed to act earlier—swapping in new gear because an old router could no longer keep up with increasing Internet speeds available in the area. (Note, I am duly thankful for this problem.) As the latest example, a whole bunch of Netgear ProSafe 318G routers failed me for the last time as small businesses have upgraded from 1.5-9mbps traditional T1 connections to 50mbps coax (cable).

A lot of you are probably muttering, “right, pfSense, sure.” Some of you might even be thinking about smoothwall or untangle NG. I played with most of the firewall distros out there, but I decided to go more basic, more old school: a plain, CLI-only install of Ubuntu Server and a few iptables rules.

Admittedly, this likely isn’t the most practical approach for every reader, but it made sense for me. I have quite a bit of experience finessing iptables and the Linux kernel itself for high throughput at Internet scale, and the fewer shiny features and graphics and clicky things that are put between me and the firewall table, the less fluff I have to get out of the way and the fewer new not-applicable-in-the-rest-of-my-work things I have to learn. Any rule I already know how to create in iptables to manage access to my servers, I also know how to apply to my firewall—if my firewall’s running the same distro as my servers are.

Cumulus Networks is a system software company founded with the principle of enabling high capacity networks that are easy to deploy and affordable. Led by networking experts and innovators from Cisco and VMware, we provide great networking for layer 2, layer 3 and overlay architectures supported by improved economics and a robust ecosystem — a modern alternative to proprietary vendor-locked stacks that constrain IT innovation.

Linux Academy

Gnome Core Apps

It would appear that the GNOME developers are currently in the process of revisiting the desktop environment’s moduleset and defining a clear set of core apps, which should form the default user experience in upcoming releases of the GNOME desktop (most probably starting with GNOME 3.20, which should be available in spring 2016).

At the moment of writing this article, the GNOME developers have only managed to announce that the Cheese webcam viewer app has been integrated as a core GNOME app as it is required by the GNOME Control Center, GNOME Initial Setup and GNOME Contacts components. They are also in talks with the developers of the Gedit text editor to make it a core app too.

Furthermore, the GNOME Color Manager component will also be pushed to the core apps moduleset, as the GNOME Control Center software requires it. However, the GNOME developers will also define a set of non-core apps, which they don’t recommend GNU/Linux OS vendors to include in their distributions when using the GNOME desktop environment by default.

Support Jupiter Broadcasting on Patreon

Post Show:

Phoenix OS

Google Android may have been developed as a smartphone operating system (and later ported to tablets, TVs, watches, and other platforms), but over the past few years we’ve seen a number of attempts to turn it into a desktop operating system.

One of the most successful has been Remix OS, which gives Android a taskbar, start menu, and an excellent window management system. The Remix OS team has also generated a lot of buzz over the past year, and this week the operating system gained a lot of new alpha testers thanks to a downloadable version of Remix OS that you can run on many recent desktop or notebook computers.

But Remix OS isn’t the only game in town. Phoenix OS is another Android-as-desktop operating system, and while it’s still pretty rough around the edges, there are a few features that could make it a better option for some testers.

The post Is that a server in your pocket? | LINUX Unplugged 128 first appeared on Jupiter Broadcasting.

]]>
Amazon’s Strongarm | Tech Talk Today 41 https://original.jupiterbroadcasting.net/64342/amazons-strongarm-tech-talk-today-41/ Tue, 12 Aug 2014 09:23:18 +0000 https://original.jupiterbroadcasting.net/?p=64342 Amazon has gone to war on multiple fronts, and is asking for you to enlist. But we’ll cut through the crap and discuss what’s really at play. Plus Xiaomi gets caught red handed spying on their users, the Bitcoin hijack that’s super impressive & more! Direct Download: MP3 Audio | OGG Audio | Video | […]

The post Amazon's Strongarm | Tech Talk Today 41 first appeared on Jupiter Broadcasting.

]]>

post thumbnail

Amazon has gone to war on multiple fronts, and is asking for you to enlist. But we’ll cut through the crap and discuss what’s really at play. Plus Xiaomi gets caught red handed spying on their users, the Bitcoin hijack that’s super impressive & more!

Direct Download:

MP3 Audio | OGG Audio | Video | HD Video | Torrent | YouTube

RSS Feeds:

MP3 Feed | OGG Feed | iTunes Feed | Video Feed | Torrent Feed

Become a supporter on Patreon:

Foo

Show Notes:

Xiaomi phones send user data to remote servers: F-Secure

At first, F-Secure did not configure an Mi Cloud (Xiaomi’s equivalent of Apple’s iCloud that stores user data) account and simply inserted a sim card, connected the phone to Wi-Fi, turned on GPS, added a contact and made and received a call and exchanged messages. The company found that the phone number of contacts added to the phone book and from SMS messages received were also forwarded. The phone follows a similar pattern even when one configures an Mi Cloud account.

“Next we connected to and logged into Mi Cloud, the iCloud-like service from Xiaomi. Then we repeated the same test steps as before. This time, the IMSI details were sent to api.account.xiaomi.com, as well as the IMEI and phone number,” writes F-Secure in its blog.

Xiaomi Makes its iMessage-Like Service Optional

Xiaomi is making the cloud messaging service that is automatically activated on its devices optional for user

These concerns refer to the MIUI Cloud Messaging service described above. As we believe it is our top priority to protect user data and privacy, we have decided to make MIUI Cloud Messaging an opt-in service and no longer automatically activate users. We have scheduled an OTA system update for today (Aug 10th) to implement this change.

After the upgrade, new users or users who factory reset their devices can enable the service by visiting “Settings > Mi Cloud > Cloud Messaging” from their home screen or “Settings > Cloud Messaging” inside the Messaging app — these are also the places where users can turn off Cloud Messaging.

Amazon wants you to ask Hachette’s CEO for lower e-book prices

Amazon Gets Increasingly Nervous

John Scalzi on Amazon/Hachette

John Scalzi:

Amazon is not your friend. Neither is any other corporation. It and they do what they do for their own interest and are more than willing to try to make you try believe that what they do for their own benefit is in fact for yours. It’s not. In this particular case, this is not about readers or authors or anyone else but Amazon wanting eBooks capped at $9.99 for its own purposes. It should stop pretending that this is about anything other than that. Readers, authors, and everyone else should stop pretending it’s about anything other than that, too.

Disney Disc Preorders Disappear From Amazon

Hacker Redirects Traffic From 19 Internet Providers to Steal Bitcoins

Researchers at Dell’s SecureWorks security division say they’ve uncovered a series of incidents in which a bitcoin thief redirected a portion of online traffic from no less than 19 Internet service providers, including data from the networks of Amazon and other hosting services like DigitalOcean and OVH, with the goal of stealing cryptocurrency from a group of bitcoin users.

Though each redirection lasted just 30 second or so, the thief was able to perform the attack 22 times, each time hijacking and gaining control of the processing power of a group of bitcoin miners, the users who expend processing power to add new coins to the currency’s network.


The attacker specifically targeted a collection of bitcoin mining “pools”


The redirection technique tricked the pools’ participants into continuing to devote their processors to bitcoin mining while allowing the hacker to keep the proceeds. At its peak, according to the researchers’ measurements, the hacker’s scam was pocketing a flow of bitcoins and other digital currencies including dogecoin and worldcoin worth close to $9,000 a day.


The Dell researchers believe the bitcoin thief used a technique called BGP hijacking, which exploits the so-called border gateway protocol, the routing instructions that direct traffic at the connection points between the Internet’s largest networks. The hacker took advantage of a staff user account at a Canadian internet service provider to periodically broadcast a spoofed command that redirected traffic from other ISPs, starting in February and continuing through May of of this year.


In fact, the BGP bitcoin-stealing exploits represent less of a new vulnerability in bitcoin than the persistent fragility of the internet itself, Dell’s researchers say. If one Canadian ISP can be used to redirect large flows of the Internet to steal a pile of cryptocurrency, other attackers could just as easily steal massive drifts of Internet data for espionage or pure disruption. The Dell researchers suggest that companies set up monitoring through a service like BGPmon, which can detect BGP hijacking attacks. B

Dutch government funds safe Dorpbox alternative Localbox

submitted by clementl

This links to a page where you can download the server. It’s written in PHP with Symfony.

The downside is that there are only clients for Windows, Android and iOS. They are planning to release the source of those in this fall.

The post Amazon's Strongarm | Tech Talk Today 41 first appeared on Jupiter Broadcasting.

]]>
Server Room Fire | TechSNAP 44 https://original.jupiterbroadcasting.net/16812/server-room-fire-techsnap-44/ Thu, 09 Feb 2012 19:08:08 +0000 https://original.jupiterbroadcasting.net/?p=16812 It’s a worst case scenario, when a server room catches fire in this week’s war story! Plus: The secrets to reliable SQL replication.

The post Server Room Fire | TechSNAP 44 first appeared on Jupiter Broadcasting.

]]>

post thumbnail

It’s a worst case scenario, when a server room catches fire in this week’s war story!

Plus: We’ll share a story that might make you re-think taking advantage of your hard drive warranty, the secrets to reliable SQL replication.

All that and more, in this episode of TechSNAP!

Thanks to:

GoDaddy.com Use our codes TechSNAP10 to save 10% at checkout, or TechSNAP20 to save 20% on hosting!

Super special savings for TechSNAP viewers only. Get a .co domain for only $7.99 (regular $29.99, previously $17.99). Use the GoDaddy Promo Code cofeb8 before February 29, 2012 to secure your own .co domain name for the same price as a .com.

Pick your code and save:
cofeb8: .co domain for $7.99
techsnap7: $7.99 .com
techsnap10: 10% off
techsnap20: 20% off 1, 2, 3 year hosting plans
techsnap40: $10 off $40
techsnap25: 25% off new Virtual DataCenter plans
Deluxe Hosting for the Price of Economy (12+ mo plans)
Code:  hostfeb8
Dates: Feb 1-29

   

Direct Download Links:

HD Video | Large Video | Mobile Video | MP3 Audio | OGG Audio | YouTube

 

Subscribe via RSS and iTunes:

   

Show Notes:

Crypto crack makes satellite phones vulnerable to eavesdropping

  • Researchers at the Ruhr University Bochum in Germany have reverse engineered the GMR–1 and GMR–2 encryption systems used by satellite phones and found serious weaknesses
  • Both algorithms rely on security by obscurity, but by downloading and disassembling the firmware, researchers were able to isolate the cryptographic algorithms
  • “Unlike standard algorithms such as AES and Blowfish—which have been subjected to decades of scrutiny from some of the world’s foremost cryptographers—these secret encryption schemes often rely more on obscurity than mathematical soundness and peer review to rebuff attacks”
  • The GMR–1 encryption system uses an algorithm that closely resembles the proprietary A5/2 encryption system that former protected GSM phone networks, before it was phased out in 2006 due to weaknesses that allowed attackers to decrypt calls in real time
  • The attack against GMR–1 allows anyone with a modest PC and some open source software to decrypt a call in less than an hour. With a cluster of more powerful machines, it is possible to decrypt a call in real time
  • GMR–2 phones are also vulnerable to cracking when there is known plaintext. This is a particularly glaring issue because the datagrams contains predictable headers and other content that can be known by the attacker, making such attacks possible
  • Researchers have not yet reverse engineered the audio codec that is used for voice calls, so a call can be decrypted, but not played back (yet). However other data types that do not use the audio codec (fax, SMS, data), have successfully been intercepted
  • Researchers are only able to intercept communications between the satellite and the user, not communications in the other direction, so only one side of the call can be eavesdropped. This is likely a limitation of the way satellite signals work, to intercept the signal from the phone to the satellite, you would need line of sight, usually requiring an EL-INT aircraft or satellite.

Customer buys refurbished drive from newegg, finds existing partitions and data

  • This story raises a number of questions about used and refurbished drives
  • Everyone knows that they should securely erase their drive before they resell it, we covered some of the techniques on TechSNAP 31 – How Malware Makes Money
  • However, how do you securely erase a drive when it has failed in some way?
  • You send the drive back to the store or the manufacturer in order to receive a replacement drive, however, you must trust to them to securely erase your data, since the drive was not usable when it left you
  • In this case it would seem that the drives we repaired, turned around and sold to another customer, without the data being properly erased
  • It would seem the only option that customers have is to not return the failed drive, which means not taking advantage of their warranty and having to pay full price for the replacement drive

Feedback:

Q: chocamo from the chatroom asks about MySQL Replication

A: MySQL has a few different replication modes built in, the main one being asynchronous replication, where a slave server constantly reads from the binary log of all changes made to the database. So you start with your two servers in a converged state (meaning they have exactly the same data), then then each time an UPDATE or INSERT command is run on the master, the slave runs the same commands, in the same order, and should continue to have the same data.

However, the slave is read only. If you want to do load balancing of more than just reads, you need to do what is called ‘multi-master replication’, In this setup, you have 2 or more servers that are all masters, and each is also the slave of the server in front of it. Something like: A -> B -> C -> A. So when an INSERT is done on server B, server C then executes that same INSERT statement, and then A, and when the query gets back to B, B notices that the query originated at B, and so skips it, preventing a loop. If you attempt an approach such as this, you will also need to adjust the auto_increment settings in MySQL, you will want the auto_increment_increment to be at least as many servers as you have, and then each server should have a different auto_increment_offset. This is to prevent primary key collisions, so that if an INSERT is done on each of the three servers at the same time, each row ends up with a unique key, otherwise replications stops until you solve the primary key collision. In the ScaleEngine setup, we also have 2 real-only slaves, one from server A and one from server C, the first offers read-only access to customers, to be used by applications that support using a read-only slave, and the other is used for taking backups (we pause replication to get a perfectly consistent copy of the entire database, then resume replication to catch back up to real-time data)

MySQL 5.5 also introduces ‘semi-synchronous replication’. In this mode, the MySQL client does not return from the query until the data has been written to not only the master, but at least X of the N slaves. This allows you to ensure that the data has actually been replicated and is safe from the failure of the master server. Normal replication in MySQL is asynchronous, meaning that when you make a change, the client returns a successful result as soon as the data has been written to the server you are connected to, and then replications happens later, this is normally the desired behaviour because it provides the greatest speed, however if the server you wrote to fails before any other servers replicate the change, that change could be lost. Semi-Synchronous replication attempts to solve this issue by allowing you to wait until there is at least 1 or more additional replicas of the data before returning a successful write. Fully synchronous replication is normally undesirable due to the performance impact.

If a table is too large, you can use ‘partitioning’ to break it in to smaller tables. You can also use the MySQL ‘Federation’ feature, to make databases from more than one server appear to be local to a single server, allowing you to move different databases to different physical machines.

War Story:

This week’s features another war story from our good friend Irish_Darkshadow (the other other Alan)


Setting:
IBM has essentially two “faces”, one is the commercial side that deals with all of the clients and the other is a completely internal organisation called the IGA (IBM Global Account) that provides IT infrastructure and support to all parts of IBM engaged with commercial business.

The IBM email system uses Lotus Domino as the server component and Lotus Notes as the client side application. The Domino servers handle the email for the company but also serve as database hosts and applications hosts. At the point in time when this war story took place, each country had their own server farm for these email / database / application servers. Each individual EMEA (Europe / Middle East / Africa) country then routed email from their in-country servers to the two “hubs”, those being Portsmouth (North Harbour) in the UK and Ehningen in Germany.

The events described below took place in the summer of 2004.

War Story:

Well, there I was once more with the 24×7 on-call phone and bouncing through my weekend without a care in the world. Well, sort of I suppose, if you don’t count a German girlfriend with shopping addiction and two kids with the inability to be quiet and give daddy some quality time with his computers. It was a Sunday afternoon and we were at the cinema which I figured was a safer option than what I chose to do for my last was story (getting very drunk).

The on-call phone started to ring almost immediately after we got out of the movie and it was the duty manager telling me that she had been “summoned” to the office to some of the higher ups for the EMEA geography. My first instinct was “and this is my problem, why?” but I resisted the urge to expose my inner bastard and played nice instead. I suspected that she had simply guessed that being called in to the office without any details was likely not a good sign and it might be useful to have some insurance (or a scape goat) beside her for the upcoming call. Apparently as I was the Crit Sit Manager for that week, I was the aforementioned insurance.

Being the devious little git that I am, I decided to bring the kids with me to the office. That would then allow me to counter any requirements on my time there with a need to get the kids home to feed them / wash them / imprison them…whatever fitted best. Essentially they would be my passport to get out of the office and buy myself some time if I needed it.

The Duty Manager that day was one of those people who had graduated to the position despite having absolutely no technical skill or capability but had an uncanny knack of lunching with the right people and “networking” with the right higher ups. Upon arrival in the office I sat in her office with her to chat about any details she had left out during her call to me. I had the kids running up and down the aisles of the call centre with one of the agents I trusted keeping an eye on them.

Nothing new was divulged prior to the big conference call kicking off and even when they started to expain the purpose of the call, details were being kept very very vague. The driver on the call was a guy from Italian Service Management which completely threw me as I had never seen a high level call originate from that part of the organisation.

The key part of the call went something like this :

Italian Guy: We are, eh, here today to eh, discuss a situation in the Vimercate (vim err kaa tay) site. Eh, perhaps we should proceed on that basis.

Duty Manager: Hello there, xxx here. I’m the duty manager for the EMEA CSC this weekend. I’m not sure what the Vimercate site is. Could you please explain ?

Me : *presses mute on the phone
Vimercate is the server farm location for Italy, all of the email and Lotus Notes database / applications for the country are run from there. If that site is down then IBM Italy will be unable to do ANY business for the duration of the outage.
*
unmutes the phone

Italian Guy: It is one of our locations here in Italy that is responsible for some servers.

Duty Manager: Ah ok, thanks for the explanation.

Italian Guy: Well about two hours ago eh….we a, received a call from the cleaning contractors that there was a, some cigarette coming out of the server room. We immediately alerted the rest of Service Management and started dealing with the crisis as a critical situation.

Me: ** rolls about laughing then thinks to telnet to some email servers in that site and nothing was connecting…….the urgency of the call started to dawn on me at this point.

Duty Manager: I’m sorry but I don’t understand what you mean when you say that there was a cigarette coming out of the server room. Did I mishear you?

Italian Guy: Sorry, not cigarette, I mean to say smoke. There was smoke coming out of the server room.

Duty Manager: Oh lord, has anyone been hurt? Is there any emergency service personnel at the site?

Italian Guy: Yes, the fire service were alerted almost immediately and nobody other than the cleaning staff was in the site when the alarm was raised. The fire has spread to other parts of the building and the firemen have been unable to get to the server room yet.

Me: Hi, I’m the crit sit manager here today. Could you please give me a current status on the server room itself? If those servers are not recoverable then we will need to activate the business continuity location and get the backup tapes couriered there. We could be up and running within 12 hours that way.

Italian Guy: Yes, yes, we know all of that. We are service management. We have already started to deal with those things. We invited you onto this call so that you are aware of the issue and can place voice messages on your incoming call lines and have your agents prepared to explain things to our users if they call your help desk. Nothing more.

Me: I have no doubt that you are on top of the situation but in such circumstances the in-country Service Management report in to the EMEA Critical Situations team who then coordinate all actions until there is a satisfactory resolution as per the EOP (Enterprise Operating Procedures). I will be taking point on this for you and liaising with EMEA Service Management for the duration of this situation.

**lots of back and forward, territorial pissing contest arguing took place until it was agree to have a followup call every hour. The second call went something like this :

Me: Good evening folks, how are things progressing on the site now?

Italian Guy: The emergency services are having difficulty due to the age of the building and they have not been able to get to the server room yet. There is nothing else new to say.

Duty Manager: So does that mean the servers are destroyed now or is there still some chance?

Italian Guy: The fire suppression system in the server room activated, that is all that we know right now.

** we adjourned the call and the next two were more of the same until the fifth call :

Italian Guy: The firemen have made it to the server room and have reported that the fire suppression system has not worked correctly. The servers themselves have been fire damaged.

Duty Manager: That’s very unfortunate, how are your efforts to get the backup tapes to the secondary site going?

Italian Guy: Eh, there is a problem with that too. The tape libraries are in the same room as the servers in an enclosure. The firemen have not retrieved them for us yet.

Me: Whoa, hold on a minute. The tapes that we’ve been trying to get into play for the last four hours are actually in the same room with the fire? Why didn’t you tell us that earlier ? If both the servers AND the backup tapes are destroyed then IBM Italy will be offline for days while a secondary site is configured. This completely changes the severity of this situation.

Italian Guy: yes, we believe that both the servers and the tapes have been damaged at this time.

**at this point I resisted the urge to reach my arm through the phone line and throttle this guy.

Duty Manager: So what can we do at this point?

Me: We need to get EMEA Service Management to start prepping a completely fresh site to take over for the ruined server farm. The problem is now that we’ve lost four hours waiting for tapes that were never going to arrive, we could have had the new servers being readied all that time.

So this all continued for a few more calls, I had my girlfriend pick up the kids between the calls and take them home and I just dived in and tried to maintain some momentum in the resolution efforts. Rather than drag it out and bore you to tears, here were the remaining revelations :
Servers were burnt to a crisp.
Backup tapes (which were in the same room) were partially burned but all were smoke damaged.
The fire suppression system simply failed to work
The firemen had to use water due to the composition of the building…WATER…on a room full of electronics.
It took 2 full days to build the new server environment which essentially meant that IBM Italy were unable to do business electronically for that duration.
Nobody ever explained why the tapes were in the server room other than to say – it was an oversight by the IT Manager. Really? an oversight?!?!!
The only bright spot in the entire debacle was that some of the data on the tapes was salvaged and shortened the duration of the outage significantly for some people.

I’m not sure there is a moral to the story or a catchy tag line like “patch your shit” but I suppose that my overriding memory of the whole situation was when I wondered how anyone thought it would be a good idea to put backup tapes in the same physical location as the servers and then neglected to do regular maintenance on an old building that was clearly a fire trap.


Round-Up:

The post Server Room Fire | TechSNAP 44 first appeared on Jupiter Broadcasting.

]]>