Arcing ZFS | BSD Now 116

chris

7 years ago

This week on BSDNow, we are going to be talking to George Wilson AKA ‘zfsdude’ about some exciting changes coming to ZFS’s ARC. Also, the latest news & a whole slew of your questions!

Thanks to:

Get Paid to Write for DigitalOcean

Direct Download:

RSS Feeds:

– Show Notes: –

Headlines

How to create new binary packages in the Ports system on OpenBSD

Creating a port is often a great first step you can take to get involved in your favorite BSD of choice, and (often) doesn’t require any actual programming to do so.
In this article we have a great walkthrough for users on creating a new ported application, and eventually binary package, on OpenBSD
As mentioned in the tutorial, a good starting place is always an existing port, which can you use as a template for your new creation. Tip: Try to pick something similar, I.E. python for a python app, Qt for Qt, etc.
This tutorial will first walk you through the process of creating your Makefile and related description about the new port.
Once you’ve created the initial Makefile, there are a bunch of new “make” targets you can begin to run to try building your port, everything from “make fetch” to “make makesum” and “make package”. Using these tests you can verify that your port is correct and results in the installable package/app you wanted.

Status update on pledge(2)

OpenBSD has been working very aggressively to convert much of their base system applications to using pledge(2) “Formerly Tame(2))
Theo has provided a great status update on where that stands as of right now and the numbers look like the following:
Out of 600 ELF binaries, 368 of them have been updated to utilize pledge(2) in some manner
This is quite a few, and includes everything from openssl, ping, sftp, grep, gzip and much more
There are still a number of “pledge-able” commands waiting for conversion, such as login, sysctl, nfsd, ssh and others.
He also mentions that there does exist some subset of commands which aren’t viable pledge(2) candidates, such as simple things like “true”, or commands like reboot/mount or even perl itself.

FreeBSD booting on the Onion Omega

Tiny $19 MIPS SoC ($25 with dock that provides built in mini-USB Serial interface, power supply, LED lights, GPIO expansion, USB port, etc)
A number of pluggable ‘expansions’ are available, including:
- Arduino Dock (connect the Omega device to your existing Arduino components)
- Blue Tooth Lower Energy
- 10/100 Ethernet Port
- Relay expansion (2 relays each, can stack up to 8 expansions to control 16 relays)
- Servo expansion (control up to 16 PWM servos, like robotic arms or camera mounts)
- OLED expansion (1″ monochrome 128×64 OLED display)
- Thermal Printer Kit (includes all wiring and other components)
The device is the product of a successful Kick Starter campaign from March of this year
Specs:
Atheros AR9330 rev1 400MHZ MIPS 24K
64MB DDR2 400MHz
16MB Flash
802.11b/g/n 150Mbps Atheros Wifi + 100mbps Atheros Wired Ethernet
18 GPIO Pins
USB Controller
Using the freebsd-wifi-build tool, I was able to build a new firmware for the device based on a profile for a similar device based on the same Atheros chip. I hope to have time to validate some of the settings and get them posted up into the wiki and get the kernel configuration committed to FreeBSD in the next week or two
It is an interesting device compared to the TP-Link WDR3600’s we did at BSDCan, as it has twice as much flash, leaving more room for the system image, but only half as much ram, and a slower CPU

SSH Performance testing

There has been a discussion about the value of upkeeping the HPN (High Performance Networking) patch to OpenSSH in the base system of FreeBSD
As part of this, I did some fresh benchmarks on my pair of new high end servers
The remaining part to be done is testing different levels of latency
By tweaking the socket buffer sizes, I was able to saturate the full 10 gigabit with netcat, iperf, etc
From the tests that have been done so far, it doesn’t look like even the NONE cipher can reach that level of performance because of the MAC (Message Authentication Code)
It does appear that some of the auto-tuning in HPN is not worked as expected
Explicitly setting -oTcpRcvBuf=7168 (KB) is enough to saturate a gigabit with 50ms RTT (round trip time)

iXsystems

Interview – George Wilson – wilzun@gmail.com / @zfsdude

OpenZFS and Delphix

News Roundup

Nicholas Marriott has replaced the aging version of less(1) in OpenBSD

Sometimes less isn’t more, it’s just less
In this story, we have news that the old version of less(1) in OpenBSD has now been ripped out in favor of the more modern fork from illumos founder Garrett D’Amore.
In addition to being a “more” modern version, it also includes far “less” of the portability code, uses terminfo, replacing termcap and is more POSIX compliant.

FreeBSD gets initial support for advanced SMR drives

Kenneth D. Merry ken@freebsd.org has developed initial support for Host Managed, and Host Aware Shingled Magnetic Recording drives in FreeBSD, available as a patch against both -current and 10-stable
“This includes support for Host Managed, Host Aware and Drive Managed SMR
drives that are either SCSI (ZBC) or ATA (ZAC) attached via a SAS
controller. This does not include support for SMR ATA drives attached via
an ATA controller. Also, I have not yet figured out how to properly detect
a Host Managed ATA drive, so this code won’t do that.”
SMR drives have overlapping tracks, because the read head can be much smaller than the write head
The drawback to this approach is that writes to the disk must take place in 256 MB “zones” that must be written from the beginning
New features in the patch:
A new ‘camcontrol zone’ command that allows displaying and managing drive zones via SCSI/ATA passthrough.
- A new zonectl(8) utility that uses the new DIOCZONECMD ioctl to display and manage zones via the da(4) (and later ada(4)) driver.
- Changes to diskinfo -v to display the zone mode of a drive.
- A new disk zone API, sys/sys/disk_zone.h.
- A new bio type, BIO_ZONE, and modifications to GEOM to support it. This new bio will allow filesystems to query zone support in a drive and manage zoned drives.
Extensive modifications to the da(4) driver to handle probing SCSI and SATA behind SAS SMR drives.
- Additional CAM CDB building functions for zone commands.
“We (Spectra Logic) are working on ZFS changes that will use this CAM and GEOM infrastructure to make ZFS play well with SMR drives. Those changes aren’t yet done.”
It is good to see active development in this area, especially from experts in archival storage
A second patch is also offered, that improves the pass(4) passthrough interface for disks, and introduces a new camdd(8) command, a version of dd that uses the pass(4) interface, kqueue, and separate reader/writer threads for improved performance
He also presents a feature wishlist that includes some interesting benchmarking features, including a ‘sink’ mode, where reads from the device are just thrown away, rather than having to write then to /dev/null

Initial implemtnation of 802.11n now in iwm(4)

OpenBSD laptop users rejoice! 802.11n has landed!
Initially only for the iwm(4) driver, support is planned for other devices in the future
Includes support for all the required (non-optional) bits to make 802.11N functional
Adds a new 11n mode to ifmedia, and MCS (modulation coding scheme) that sits alongside the ieee80211_rateset structure.
No support for MIMO / SGI (Short Guard Interval) or 40 MHz wide-channels, but perhaps we will see those in a future update.
They are asking users for testing against a wide variety of any/all APs!

Freebsd adds support for Bluetooth LE Security Management

FreeBSD + BlueTooth, not something we discuss a lot about, but it is still under active development.
The most recently added features come from Takanori Watanabe, and adds new LE Security Management.
Specifically, it enables support for BLE Security Manager Protocol(SMP), and enables a userland tool to wait for the underlying HCI connection to be encrypted.

Building OpnSense on HardenedBSD

Looking for a way to further Harden your router? We have a tutorial from the HardenedBSD developer, Shawn Webb, about how to build OpnSense on HBSD 10-STABLE.
You’ll need to first be running HBSD 10-STABLE somewhere, in this article he is using bhyve for the builder VM.
The build process itself is mostly pretty straight-forward, but there are a number of different repos that all have to be checked out, so pay attention to which goes where.
+In this example he does a targeted build for a Netgate RCC-VE-4860, but you can pick your particular build.

Beastie Bits

1 BTC bounty for chromium bug!

DesktopBSD 2.0 M1 released

By implementing asynchronous pru_attach for UDP, Sepherosa Ziehau has increased connect rate by around 15K connections per second

Stephen Bourne, known for the Bourne Shell, will be giving a talk at NYCBUG this week

Tor Browser 5.0.3 for OpenBSD released
– The Tor BSD Diversity Project aim to
– Increase the number of Tor relays running BSDs. We envision this happening by increasing the total number of relays, with the addition of more BSD users running relays;
– Make the Tor Browser available under BSD operating systems using native packaging mechanisms. Our first target is OpenBSD;
– Engage the broader BSD community about the Tor anonymity network and the place that BSD Unix should occupy in the privacy community at large.

Screenshots from Unix People circa 2002

Feedback/Questions

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv