Jupiter Broadcasting

Preventing a btrfs Nightmare | LAS 320

chris

What happens when btrfs goes bad? After rescuing our system from a massive crash, we’ll share what went wrong, how you can prevent problems, reclaim gigabytes of space, and optimize your Linux box for an SSD drive.

Plus a quick look at Ubuntu MATE Remix, the NSA targets Linux users, solving the distro hopping bug….

AND SO MUCH MORE!

All this week on, The Linux Action Show!

Thanks to:




Download:

HD Video | Mobile Video | WebM Torrent | MP3 Audio | Ogg Audio | YouTube | HD Torrent

RSS Feeds:

HD Video Feed | Large Video Feed | Mobile Video Feed | MP3 Feed | Ogg Feed | iTunes Feeds | Torrent Feed

— Show Notes: —

How I saved myself from a btrfs nightmare:


Brought to you by: System76

We asked: Do you trust btrfs?

It really started about three weeks ago, when I was doing a big package upgrade. The upgrade installs started to fail reporting out of space.

df reported more than 30GBs free on / partition. Despite df’s optimized summarization of my situation I cleared my package cache, deleted some unneeded VMs , and about 20 Steam games that I had stashed in /opt.

Fast forward to this Friday and my system won’t boot. I knew it was a risk, but I was desperate. I decided to compress all the things. Based on my quick reading of a few wiki pages, I could compress files on demand but necessarily compress the entire file system all the time. IE compress a lot of crap small libraries, reduce the amount of blocks they take up, and free up some space.

btrfs has built in support for doing this. I have ran compression on my /home file system since day one of this install. I opt for LZO compression. The ration of compression is lower than other options, but the performance is fantastic. I set off compressing the messy areas of my / (like /var /usr/lib).

But after a sanity check reboot, and many errors about failing to find a file, it was clear my understanding was wrong, and the mount options in my /etc/fstab needed to be updated.

I dd’ed the latest Antergos ISO onto a USB 3.0 thumb drive and booted the Bonobo into the live environment, fired up gparted to remind my self which physical device my rootfs was on (hey it’s been over a year!) and then promptly created a mount point for it under /mnt/fix.

I jumped into my /mnt/fix folder and immediately was back at home on my Bonobo’s root file system. A quick nano (YES) of my /mnt/fix/etc/fstab file and I added the compress flag to my root file system’s mount options.

Unmounted and rebooted the Bonobo, and the 1+ year Arch install fired right up.

I then set off to figure out how to better use btrfs on my system, and specifically tune the file system for my SSD drives.

Why btrfs?

“a new copy on write (CoW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration.”

The filesystem disk format is no longer unstable, and it’s not expected to change unless there are strong reasons to do so. If there is a format change, file systems with a unchanged format will continue to be mountable and usable by newer kernels.

It kept telling me my device was full, but I had 30GB free. btrfs balance to the rescue.

Why the different DF?

btrfs filesystem df /

btrfs filesystem show /dev/sda3

So, in general, it is impossible to give an accurate estimate of the amount of free space on any btrfs filesystem. Yes, this sucks. If you have a really good idea for how to make it simple for users to understand how much space they’ve got left, please do let us know, but also please be aware that the finest minds in btrfs development have been thinking about this problem for at least a couple of years, and we haven’t found a simple solution yet.

Step 1: Rebalance, it’s not just for RAID arrays anymore!

btrfs balance start / -v

In general usage, no. A full unfiltered balance typically takes a long time, and will rewrite huge amounts of data unnecessarily. You may wish to run a balance on metadata only (see Balance_Filters) if you find you have very large amounts of metadata space allocated but unused, but this should be a last resort. At some point, this kind of clean-up will be made an automatic background process.

Step 2: Defragment that disk

btrfs filesystem defragment -r -v /

find / -xdev -type d -print -exec btrfs filesystem defragment ‘{}’ \;

Step 3: Compress that disk’s file system.

What are the differences between compression methods?

There’s a speed/ratio trade-off:

The differences depend on the actual data set and cannot be expressed by a single number or recommendation. Do your own benchmarks. LZO seems to give satisfying results for general use.

btrfs filesystem defragment -r -v -clzo /

LABEL=rootfs / btrfs defaults,compress=lzo 0 1

Step 4: Optimize for SSD

noatime,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache

Example:

LABEL=homefs /home btrfs defaults,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache 0 1

btrfs gotchas?

This page lists problems one might face when trying btrfs, some of these are not really bugs, but rather inconveniences about things not yet implemented, or yet undocumented design decisions.

— Picks —

Runs Linux

This GSM Base Station Powered by the BeagleBone Black, Runs Linux

Desktop App Pick

New Linux Podcast App ‘Vocal’

The project’s developer, Nathan Dyer, has made beta builds — still unstable and not feature complete — available for testing through a dedicated PPA for Ubuntu 14.04 LTS and 14.10.

Weekly Spotlight

Tails – Privacy for anyone anywhere

Tails is a live operating system, that you can start on almost any computer from a DVD, USB stick, or SD card. It aims at preserving your privacy and anonymity

HTTPS Everywhere is a Firefox, Chrome, and Opera extension that encrypts your communications with many major websites, making your browsing more secure. Encrypt the web: Install HTTPS Everywhere today.

— NEWS —

NSA targets the privacy-conscious | Das Erste – Panorama – Meldungen

The investigation discloses the following:

  • Two servers in Germany – in Berlin and Nuremberg – are under surveillance by the NSA.
  • Merely searching the web for the privacy-enhancing software tools outlined in the XKeyscore rules causes the NSA to mark and track the IP address of the person doing the search. Not only are German privacy software users tracked, but the source code shows that privacy software users worldwide are tracked by the NSA.
  • Among the NSA’s targets is the Tor network funded primarily by the US government to aid democracy advocates in authoritarian states.
  • The XKeyscore rules reveal that the NSA tracks all connections to a server that hosts part of an anonymous email service at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) in Cambridge, Massachusetts. It also records details about visits to a popular internet journal for Linux operating system users called “the Linux Journal – the Original Magazine of the Linux Community”, and calls it an “extremist forum”.
If you read Boing Boing, the NSA considers you a target for deep surveillance

Tor and Tails have been part of the mainstream discussion of online security, surveillance and privacy for years. It’s nothing short of bizarre to place people under suspicion for searching for these terms.

One expert suggested that the NSA’s intention here was to separate the sheep from the goats — to split the entire population of the Internet into “people who have the technical know-how to be private” and “people who don’t” and then capture all the communications from the first group.

Another expert said that s/he believed that this leak may come from a second source, not Edward Snowden, as s/he had not seen this in the original Snowden docs; and had seen other revelations that also appeared independent of the Snowden materials.

Schneier on Security: NSA Targets the Privacy-Conscious for Surveillance

Jake Appelbaum et. al, are reporting on XKEYSCORE selection rules that target users — and people who just visit the websites of — Tor, Tails, and other sites. This isn’t just metadata; this is “full take” content that’s stored forever.

Dear NSA, Privacy is a Fundamental Right, Not Reasonable Suspicion | Electronic Frontier Foundation

Even the U.S. Foreign Intelligence Surveillance Court recognizes this, as the FISA prohibits targeting people or conducting investigations based solely on activities protected by the First Amendment. Regardless of whether the NSA is relying on FISA to authorize this activity or conducting the spying overseas, it is deeply problematic.

NSA: Linux Journal is an “extremist forum” and its readers get flagged for extra surveillance | Linux Journal

The Ultra-Simple App That Lets Anyone Encrypt Anything | Threat Level | WIRED

Wired reports that Nadim Kobeissi will release a bet aversion of an all-purpose file encyrption browser plugin called miniLock at the HOPE hacker conference in New York. The free and open source plugin is meant to make it easy to drag and drop files to encrypt so that no one but the intended recipient can unscramble them.

Wayland in Fedora Update

So the summary is that while we expect to have a version of Wayland in Fedora Workstation 21 that will be able to run a fully functional desktop, there are some missing pieces we now know that will not make it. Which means that since we want to ship at least one Fedora release with a feature complete Wayland as an option before making it default, that means that Fedora Workstation 23 is the earliest Wayland can be the default.

The KDE Improv Project Has Announced Its End

Carl Symons on the behalf of Aaron Seigo and the rest of the Improv crew have sent out an email to the backers saying the project is over, they will issue partial refunds, etc. It’s only a partial refund right now as they had already invested some money into buying long lead times with their Chinese manufacturer.

There were also credit card processing fees, etc, but they’re working out a path for full reimbursement. It’s also said Aaron invested $200k USD into the project.

There was simply not enough support to make the project work, despite having fully functional, production ready devices and a strong commitment to succeed. The Free software community does not seem ready at this point to make a concerted stand on the pressing issue of hardware freedom

The group of folks behind the KDE desktop environment have been trying for a few years to deliver a tablet with wouldn’t rely on proprietary software. More recently they unveiled the Improv Board, a small, cheap computer module designed to ship with Mer Linux.

At this point, all the team is promising are partial refunds since part of the money has already been spent. But the goal is to eventually provide full refunds to folks who have put their faith (and money) into the project.

In a statement, the team suggests “the Free software community does not seem ready at this point to make a concerted stand on the pressing issue of hardware freedom,”
the Improv board was expected to sell for around $75

Ubuntu MATE Remix

Objectives

The Ubuntu MATE Remix has the following primary goal:

Ubuntu MATE Remix has some secondary objectives:

Feedback:

— Chris’ Stash —

Hang in our chat room:

irc.geekshed.net #jupiterbroadcasting

— What’s Matt Doin? —

— Find us on Google+ —

— Find us on Twitter —

— Follow the network on Facebook: —

— Catch the show LIVE Sunday 10am Pacific / 1pm Eastern / 6pm UTC: —