openbgpd – Jupiter Broadcasting https://www.jupiterbroadcasting.com Open Source Entertainment, on Demand. Thu, 13 Dec 2018 18:13:28 +0000 en-US hourly 1 https://wordpress.org/?v=5.5.3 https://original.jupiterbroadcasting.net/wp-content/uploads/2019/04/cropped-favicon-32x32.png openbgpd – Jupiter Broadcasting https://www.jupiterbroadcasting.com 32 32 Ho, Ho, Ho – 12.0 | BSD Now 276 https://original.jupiterbroadcasting.net/128421/ho-ho-ho-12-0-bsd-now-276/ Thu, 13 Dec 2018 10:12:22 +0000 https://original.jupiterbroadcasting.net/?p=128421 ##Headlines ###FreeBSD 12.0 is available After a long release cycle, the wait is over: FreeBSD 12.0 is now officially available. We’ve picked a few interesting things to cover in the show, make sure to read the full Release Notes Userland: Group permissions on /dev/acpi have been changed to allow users in the operator GID to […]

The post Ho, Ho, Ho - 12.0 | BSD Now 276 first appeared on Jupiter Broadcasting.

]]>

##Headlines
###FreeBSD 12.0 is available

  • After a long release cycle, the wait is over: FreeBSD 12.0 is now officially available.
  • We’ve picked a few interesting things to cover in the show, make sure to read the full Release Notes

Userland:
Group permissions on /dev/acpi have been changed to allow users in the operator GID to invoke acpiconf(8) to suspend the system.
The default devfs.rules(5) configuration has been updated to allow mount_fusefs(8) with jail(8).
The default PAGER now defaults to less(1) for most commands.
The newsyslog(8) utility has been updated to reject configuration entries that specify setuid(2) or executable log files.
The WITH_REPRODUCIBLE_BUILD src.conf(5) knob has been enabled by default.
A new src.conf(5) knob, WITH_RETPOLINE, has been added to enable the retpoline mitigation for userland builds.
Userland applications:
The dtrace(1) utility has been updated to support if and else statements.
The legacy gdb(1) utility included in the base system is now installed to /usr/libexec for use with crashinfo(8). The gdbserver and gdbtui utilities are no longer installed. For interactive debugging, lldb(1) or a modern version of gdb(1) from devel/gdb should be used. A new src.conf(5) knob, WITHOUT_GDB_LIBEXEC has been added to disable building gdb(1). The gdb(1) utility is still installed in /usr/bin on sparc64.
The setfacl(1) utility has been updated to include a new flag, -R, used to operate recursively on directories.
The geli(8) utility has been updated to provide support for initializing multiple providers at once when they use the same passphrase and/or key.
The dd(1) utility has been updated to add the status=progress option, which prints the status of its operation on a single line once per second, similar to GNU dd(1).
The date(1) utility has been updated to include a new flag, -I, which prints its output in ISO 8601 formatting.
The bectl(8) utility has been added, providing an administrative interface for managing ZFS boot environments, similar to sysutils/beadm.
The bhyve(8) utility has been updated to add a new subcommand to the -l and -s flags, help, which when used, prints a list of supported LPC and PCI devices, respectively.
The tftp(1) utility has been updated to change the default transfer mode from ASCII to binary.
The chown(8) utility has been updated to prevent overflow of UID or GID arguments where the argument exceeded UID_MAX or GID_MAX, respectively.
Kernel:
The ACPI subsystem has been updated to implement Device object types for ACPI 6.0 support, required for some Dell, Inc. Poweredge™ AMD® Epyc™ systems.
The amdsmn(4) and amdtemp(4) drivers have been updated to attach to AMD® Ryzen 2™ host bridges.
The amdtemp(4) driver has been updated to fix temperature reporting for AMD® 2990WX CPUs.
Kernel Configuration:
The VIMAGE kernel configuration option has been enabled by default.
The dumpon(8) utility has been updated to add support for compressed kernel crash dumps when the kernel configuration file includes the GZIO option. See rc.conf(5) and dumpon(8) for additional information.
The NUMA option has been enabled by default in the amd64 GENERIC and MINIMAL kernel configurations.
Device Drivers:
The random(4) driver has been updated to remove the Yarrow algorithm. The Fortuna algorithm remains the default, and now only, available algorithm.
The vt(4) driver has been updated with performance improvements, drawing text at rates ranging from 2- to 6-times faster.
Deprecated Drivers:
The lmc(4) driver has been removed.
The ixgb(4) driver has been removed.
The nxge(4) driver has been removed.
The vxge(4) driver has been removed.
The jedec_ts(4) driver has been removed in 12.0-RELEASE, and its functionality replaced by jedec_dimm(4).
The DRM driver for modern graphics chipsets has been marked deprecated and marked for removal in FreeBSD 13. The DRM kernel modules are available from graphics/drm-stable-kmod or graphics/drm-legacy-kmod in the Ports Collection as well as via pkg(8). Additionally, the kernel modules have been added to the lua loader.conf(5) module_blacklist, as installation from the Ports Collection or pkg(8) is strongly recommended.
The following drivers have been deprecated in FreeBSD 12.0, and not present in FreeBSD 13.0: ae(4), de(4), ed(4), ep(4), ex(4), fe(4), pcn(4), sf(4), sn(4), tl(4), tx(4), txp(4), vx(4), wb(4), xe(4)
Storage:
The UFS/FFS filesystem has been updated to support check hashes to cylinder-group maps. Support for check hashes is available only for UFS2.
The UFS/FFS filesystem has been updated to consolidate TRIM/BIO_DELETE commands, reducing read/write requests due to fewer TRIM messages being sent simultaneously.
TRIM consolidation support has been enabled by default in the UFS/FFS filesystem. TRIM consolidation can be disabled by setting the vfs.ffs.dotrimcons sysctl(8) to 0, or adding vfs.ffs.dotrimcons=0 to sysctl.conf(5).
NFS:
The NFS version 4.1 server has been updated to include pNFS server support.
ZFS:
ZFS has been updated to include new sysctl(8)s, vfs.zfs.arc_min_prefetch_ms and vfs.zfs.arc_min_prescient_prefetch_ms, which improve performance of the zpool(8) scrub subcommand.
The new spacemap_v2 zpool feature has been added. This provides more efficient encoding of spacemaps, especially for full vdev spacemaps.
The large_dnode zpool feature been imported, allowing better compatibility with pools created under ZFS-on-Linux 0.7.x
Many bug fixes have been applied to the device removal feature. This feature allows you to remove a non-redundant or mirror vdev from a pool by relocating its data to other vdevs.
Includes the fix for PR 229614 that could cause processes to hang in zil_commit()
Boot Loader Changes:
The lua loader(8) has been updated to detect a list of installed kernels to boot.
The loader(8) has been updated to support geli(8) for all architectures and all disk-like devices.
The loader(8) has been updated to add support for loading Intel® microcode updates early during the boot process.

Networking:
The pf(4) packet filter is now usable within a jail(8) using vnet(9).
The pf(4) packet filter has been updated to use rmlock(9) instead of rwlock(9), resulting in significant performance improvements.
The SO_REUSEPORT_LB option has been added to the network stack, allowing multiple programs or threads to bind to the same port, and incoming connections load balanced using a hash function.

  • Again, read the release notes for a full list, check out the errata notices. A big THANKS to the entire release engineering team and all developers involved in the release, much appreciated!

###Abandon Linux. Move to FreeBSD or Illumos

If you use GNU/Linux and you are only on opensource, you may be doing it wrong. Here’s why.
Is your company based on opensource based software only? Do you have a bunch of developers hitting some kind of server you have installed for them to “do their thing”? Being it for economical reasons (remember to donate), being it for philosophycal ones, you may have skipped good alternatives. The BSD’s and Illumos.
I bet you are running some sort of Debian, openSuSE or CentOS. It’s very discouraging having entered into the IT field recently and discover many of the people you meet do not even recognise the name BSD. Naming Solaris seems like naming the evil itself. The problem being many do not know why. They can’t point anything specific other than it’s fading out. This has recently shown strong when Oracle officials have stated development for new features has ceased and almost 90 % of developers for Solaris have been layed off. AIX seems alien to almost everybody unless you have a white beard. And all this is silly.
And here’s why. You are certainly missing two important features that FreeBSD and Illumos derivatives are enjoying. A full virtualization technology, much better and fully developed compared to the LXC containers in the Linux world, such as Jails on BSD, Zones in Solaris/Illumos, and the great ZFS file system which both share.
You have probably heard of a new Linux filesystem named Btrfs, which by the way, development has been dropped from the Red Hat side. Trying to emulate ZFS, Oracle started developing Btrfs file system before they acquired Sun (the original developer of ZFS), and SuSE joined the effort as well as Red Hat. It is not as well developed as ZFS and it hasn’t been tested in production environments as extensively as the former has. That leaves some uncertainty on using it or not. Red Hat leaving it aside does add some more. Although some organizations have used it with various grades of success.
But why is this anyhow interesting for a sysadmin or any organization? Well… FreeBSD (descendant of Berkeley UNIX) and SmartOS (based on Illumos) aglutinate some features that make administration easier, safer, faster and more reliable. The dream of any systems administrator.
To start, the ZFS filesystem combines the typical filesystem with a volume manager. It includes protection against corruption, snapshots and copy-on-write clones, as well as volume manager.
Jails is another interesting piece of technology. Linux folks usually associate this as a sort of chroot. It isn’t. It is somehow inspired by it but as you may know you can escape from a chroot environment with a blink of an eye. Jails are not called jails casually. The name has a purpose. Contain processes and programs within a defined and totally controlled environment. Jails appeared first in FreeBSD in the year 2000. Solaris Zones debuted on 2005 (now called containers) are the now proprietary version of those.
There are some other technologies on Linux such as Btrfs or Docker. But they have some caveats. Btrfs hasn’t been fully developed yet and it’s hasn’t been proved as much in production environments as ZFS has. And some problems have arisen recently although the developers are pushing the envelope. At some time they will match ZFS capabilities for sure. Docker is growing exponentially and it’s one of the cool technologies of modern times. The caveat is, as before, the development of this technology hasn’t been fully developed. Unlike other virtualization technologies this is not a kernel playing on top of another kernel. This is virtualization at the OS level, meaning differentiated environments can coexist on a single host, “hitting” the same unique kernel which controls and shares the resources. The problem comes when you put Docker on top of any other virtualization technology such as KVM or Xen. It breaks the purpose of it and has a performance penalty.
I have arrived into the IT field with very little knowledge, that is true. But what I see strikes me. Working in a bank has allowed me to see a big production environment that needs the highest of the availability and reliability. This is, sometimes, achieved by bruteforce. And it’s legitime and adequate. Redundancy has a reason and a purpose for example. But some other times it looks, it feels, like killing flies with cannons. More hardware, more virtual machines, more people, more of this, more of that. They can afford it, so they try to maintain the cost low but at the end of the day there is a chunky budget to back operations.
But here comes reality. You’re not a bank and you need to squeeze your investment as much as possible. By using FreeBSD jails you can avoid the performance penalty of KVM or Xen virtualization. Do you use VMWare or Hyper-V? You can avoid both and gain in performance. Not only that, control and manageability are equal as before, and sometimes easier to administer. There are four ways to operate them which can be divided in two categories. Hardcore and Human Being. For the Hardcore use the FreeBSD handbook and investigate as much as you can. For the Human Being way there are three options to use. Ezjail, Iocage and CBSD which are frameworks or programs as you may call to manage jails. I personally use Iocage but I have also used Ezjail.
How can you use jails on your benefit? Ever tried to configure some new software and failed miserably? You can have three different jails running at the same time with different configurations. Want to try a new configuration in a production piece of hardware without applying it on the final users? You can do that with a small jail while the production environment is on in another bigger, chunkier jail.
Want to divide the hardware as a replica of the division of the team/s you are working with? Want to sell virtual machines with bare metal performance? Do you want to isolate some piece of critical software or even data in a more controlled environment? Do you have different clients and you want to use the same hardware but you want to avoid them seeing each other at the same time you maintain performance and reliability?
Are you a developer and you have to have reliable and portable snapshots of your work? Do you want to try new options-designs without breaking your previous work, in a timeless fashion? You can work on something, clone the jail and apply the new ideas on the project in a matter of seconds. You can stop there, export the filesystem snapshot containing all the environment and all your work and place it on a thumbdrive to later import it on a big production system. Want to change that image properties such as the network stack interface and ip? This is just one command away from you.
But what properties can you assign to a jail and how can I manage them you may be wondering. Hostname, disk quota, i/o, memory, cpu limits, network isolation, network virtualization, snapshots and the manage of those, migration and root privilege isolation to name a few. You can also clone them and import and export them between different systems. Some of these things because of ZFS. Iocage is a python program to manage jails and it takes profit from ZFS advantages.
But FreeBSD is not Linux you may say. No it is not. There are no run levels. The systemd factor is out of this equation. This is so since the begginning. Ever wondered where did vi come from? The TCP/IP stack? Your beloved macOS from Apple? All this is coming from the FreeBSD project. If you are used to Linux your adaptation period with any BSD will be short, very short. You will almost feel at home. Used to packaged software using yum or apt-get? No worries. With pkgng, the package management tool used in FreeBSD has almost 27.000 compiled packages for you to use. Almost all software found on any of the important GNU/Linux distros can be found here. Java, Python, C, C++, Clang, GCC, Javascript frameworks, Ruby, PHP, MySQL and the major forks, etc. All this opensource software, and much more, is available at your fingertips.
I am a developer and… frankly my time is money and I appreciate both much more than dealing with systems configuration, etc. You can set a VM using VMWare or VirtualBox and play with barebones FreeBSD or you can use TrueOS (a derivative) which comes in a server version and a desktop oriented one. The latter will be easier for you to play with. You may be doing this already with Linux. There is a third and very sensible option. FreeNAS, developed by iXSystems. It is FreeBSD based and offers all these technologies with a GUI. VMWare, Hyper-V? Nowadays you can get your hands off the CLI and get a decent, usable, nice GUI.
You say you play on the cloud. The major players already include FreeBSD in their offerings. You can find it in Amazon AWS or Azure (with official Microsoft support contracts too!). You can also find it in DigitalOcean and other hosting providers. There is no excuse. You can use it at home, at the office, with old or new hardware and in the cloud as well. You can even pay for a support contract to use it. Joyent, the developers of SmartOS have their own cloud with different locations around the globe. Have a look on them too.
If you want the original of ZFS and zones you may think of Solaris. But it’s fading away. But it really isn’t. When Oracle bouth Sun many people ran away in an stampide fashion. Some of the good folks working at Sun founded new projects. One of these is Illumos. Joyent is a company formed by people who developed these technologies. They are a cloud operator, have been recently bought by Samsung and have a very competent team of people providing great tech solutions. They have developed an OS, called SmartOS (based on Illumos) with all these features. The source from this goes back to the early days of UNIX. Do you remember the days of OpenSolaris when Sun opensourced the crown jewels? There you have it. A modern opensource UNIX operating system with the roots in their original place and the head planted on today’s needs.
In conclusion. If you are on GNU/Linux and you only use opensource software you may be doing it wrong. And missing goodies you may need and like. Once you put your hands on them, trust me, you won’t look back. And if you have some “old fashioned” admins who know Solaris, you can bring them to a new profitable and exciting life with both systems.
Still not convinced? Would you have ever imagined Microsoft supporting Linux? Even loving it? They do love now FreeBSD. And not only that, they provide their own image in the Azure Cloud and you can get Microsoft support, payed support if you want to use the platform on Azure. Ain’t it… surprising? Convincing at all?
PS: I haven’t mentioned both softwares, FreeBSD and SmartOS do have a Linux translation layer. This means you can run Linux binaries on them and the program won’t cough at all. Since the ABI stays stable the only thing you need to run a Linux binary is a translation between the different system calls and the libraries. Remember POSIX? Choose your poison and enjoy it.

###A partly-cloudy IPsec VPN

  • Audience

I’m assuming that readers have at least a basic knowledge of TCP/IP networking and some UNIX or UNIX-like systems, but not necessarily OpenBSD or FreeBSD. This post will therefore be light on details that aren’t OS specific and are likely to be encountered in normal use (e.g., how to use vi or another text editor.) For more information on these topics, read Absolute FreeBSD (3ed.) by Michael W. Lucas.

  • Overview

I’m redoing my DigitalOcean virtual machines (which they call droplets). My requirements are:

  • VPN
  • Road-warrior access, so I can use private network resources from anywhere.
  • A site-to-site VPN, extending my home network to my VPSes.
  • Hosting for public and private network services.
  • A proxy service to provide a public IP address to services hosted at home.

The last item is on the list because I don’t actually have a public IP address at home; my firewall’s external address is in the RFC 1918 space, and the entire apartment building shares a single public IPv4 address.1 (IPv6? Don’t I wish.) The end-state network will include one OpenBSD droplet providing firewall, router, and VPN services; and one FreeBSD droplet hosting multiple jailed services.
I’ll be providing access via these droplets to a NextCloud instance at home. A simple NAT on the DO router droplet isn’t going to work, because packets going from home to the internet would exit through the apartment building’s connection and not through the VPN. It’s possible that I could do work around this issue with packet tagging using the pf firewall, but HAProxy is simple to configure and unlikely to result in hard-to-debug problems. relayd is also an option, but doesn’t have the TLS parsing abilities of HAProxy, which I’ll be using later on.
Since this system includes jails running on a VPS, and they’ve got RFC 1918 addresses, I want them reachable from my home network. Once that’s done, I can access the private address space from anywhere through a VPN connection to the cloudy router.
The VPN itself will be of the IPsec variety. IPsec is the traditional enterprise VPN standard, and is even used for classified applications, but has a (somewhat-deserved) reputation for complexity, but recent versions of OpenBSD turn down the difficulty by quite a bit.

This VPN both separates internal network traffic from public traffic and uses encryption to prevent interception or tampering.
Once traffic has been encrypted, decrypting it without the key would, as Bruce Schneier once put it, require a computer built from something other than matter that occupies something other than space. Dyson spheres and a frakton of causality violation would possibly work, as would mathemagical technology that alters the local calendar such that P=NP.2 Black-bag jobs and/or suborning cloud provider employees doesn’t quite have that guarantee of impossibility, however. If you have serious security requirements, you’ll need to do better than a random blog entry.

##News Roundup
###KLEAK: Practical Kernel Memory Disclosure Detection

Modern operating systems such as NetBSD, macOS, and Windows isolate their kernel from userspace programs to increase fault tolerance and to protect against malicious manipulations [10]. User space programs have to call into the kernel to request resources, via system calls or ioctls. This communication between user space and kernel space crosses a security boundary. Kernel memory disclosures – also known as kernel information leaks – denote the inadvertent copying of uninitialized bytes from kernel space to user space. Such disclosed memory may contain cryptographic keys, information about the kernel memory layout, or other forms of secret data. Even though kernel memory disclosures do not allow direct exploitation of a system, they lay the ground for it.
We introduce KLEAK, a simple approach to dynamically detect kernel information leaks. Simply said, KLEAK utilizes a rudimentary form of taint tracking: it taints kernel memory with marker values, lets the data travel through the kernel and scans the buffers exchanged between the kernel and the user space for these marker values. By using compiler instrumentation and rotating the markers at regular intervals, KLEAK significantly reduces the number of false positives, and is able to yield relevant results with little effort.
Our approach is practically feasible as we prove with an implementation for the NetBSD kernel. A small performance penalty is introduced, but the system remains usable. In addition to implementing KLEAK in the NetBSD kernel, we applied our approach to FreeBSD 11.2. In total, we detected 21 previously unknown kernel memory disclosures in NetBSD-current and FreeBSD 11.2, which were fixed subsequently. As a follow-up, the projects’ developers manually audited related kernel areas and identified dozens of other kernel memory disclosures.
The remainder of this paper is structured as follows. Section II discusses the bug class of kernel memory disclosures. Section III presents KLEAK to dynamically detect instances of this bug class. Section IV discusses the results of applying KLEAK to NetBSD-current and FreeBSD 11.2. Section V reviews prior research. Finally, Section VI concludes this paper.

###How To Create Official Synth Repo

  • System Environment

  • Make sure /usr/dports is updated and that it contains no cruft (git pull; git status). Remove any cruft.

  • Make sure your ‘synth’ is up-to-date ‘pkg upgrade synth’. If you already updated your system you may have to build synth from scratch, from /usr/dports/ports-mgmt/synth.

  • Make sure /etc/make.conf is clean.

  • Update /usr/src to the current master, make sure there is no cruft in it

  • Do a full buildworld, buildkernel, installkernel and installworld

  • Reboot

  • After the reboot, before proceeding, run ‘uname -a’ and make sure you are now on the desired release or development kernel.

  • Synth Environment

  • /usr/local/etc/synth/ contains the synth configuration. It should contain a synth.ini file (you may have to rename the template), and you will have to create or edit a LiveSystem-make.conf file.

  • System requirements are hefty. Just linking chromium alone eats at least 30GB, for example. Concurrent c++ compiles can eat up to 2GB per process. We recommend at least 100GB of SSD based swap space and 300GB of free space on the filesystem.

  • synth.ini should contain this. Plus modify the builders and jobs to suit your system. With 128G of ram, 30/30 or 40/25 works well. If you have 32G of ram, maybe 8/8 or less.

; Take care when hand editing!

[Global Configuration]
profile_selected= LiveSystem

[LiveSystem]
Operating_system= DragonFly
Directory_packages= /build/synth/live_packages
Directory_repository= /build/synth/live_packages/All
Directory_portsdir= /build/synth/dports
Directory_options= /build/synth/options
Directory_distfiles= /usr/distfiles
Directory_buildbase= /build/synth/build
Directory_logs= /build/synth/logs
Directory_ccache= disabled
Directory_system= /
Number_of_builders= 30
Max_jobs_per_builder= 30
Tmpfs_workdir= true
Tmpfs_localbase= true
Display_with_ncurses= true
leverage_prebuilt= false

  • LiveSystem-make.conf should contain one line to restrict licensing to only what is allowed to be built as a binary package:

LICENSES_ACCEPTED= NONE

  • Make sure there is no other cruft in /usr/local/etc/synth/

  • In the example above, the synth working dirs are in “/build/synth”. Make sure the base directories exist. Clean out any cruft for a fresh build from-scratch:

rm -rf /build/synth/live_packages/*
rm -rf /build/synth/logs
mkdir /build/synth/logs

  • Run synth everything. I recommend doing this in a ‘screen’ session in case you lose your ssh session (assuming you are ssh’d into the build machine).

(optionally start a screen session)
synth everything

  • A full synth build takes over 24 hours to run on a 48-core box, around 12 hours to run on a 64-core box. On a 4-core/8-thread box it will take at least 3 days. There will be times when swap space is heavily used. If you have not run synth before, monitor your memory and swap loads to make sure you have configured the jobs properly. If you are overloading the system, you may have to ^C the synth run, reduce the jobs, and start it again. It will pick up where it left off.
  • When synth finishes, let it rebuild the database. You then have a working binary repo.
  • It is usually a good idea to run synth several times to pick up any stuff it couldn’t build the first time. Each of these incremental runs may take a few hours, depending on what it tries to build.

###Interview with founder and maintainer of GhostBSD, Eric Turgeon

  • Thanks you Eric for taking part. To start off, could you tell us a little about yourself, just a bit of background?
  • How did you become interested in open source?
  • When and how did you get interested in the BSD operating systems?
  • On your Twitter profile, you state that you are an automation engineer at iXsystems. Can you share what you do in your day-to-day job?
  • You are the founder and project lead of GhostBSD. Could you describe GhostBSD to those who have never used it or never heard of it?
  • Developing an operating system is not a small thing. What made you decide to start the GhostBSD project and not join another “desktop FreeBSD” related project, such as PC-BSD and DesktopBSD at the time?
  • How did you get to the name GhostBSD? Did you consider any other names?
  • You recently released GhostBSD 18.10? What’s new in that version and what are the key features? What has changed since GhostBSD 11.1?
  • The current version is 18.10. Will the next version be 19.04 (like Ubuntu’s version numbering), or is a new version released after the next stable TrueOS release
  • Can you tell us something about the development team? Is it yourself, or are there other core team members? I think I saw two other developers on your Github project page.
  • How about the relationship with the community? Is it possible for a community member to contribute, and how are those contributions handled?
  • What was the biggest challenge during development?
  • If you had to pick one feature readers should check out in GhostBSD, what is it and why?
  • What is the relationship between iXsystems and the GhostBSD project? Or is GhostBSD a hobby project that you run separately from your work at iXsystems?
  • What is the relationship between GhostBSD and TrueOS? Is GhostBSD TrueOS with the MATE desktop on top, or are there other modifications, additions, and differences?
  • Where does GhostBSD go from here? What are your plans for 2019?
  • Is there anything else that wasn’t asked or that you want to share?

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

The post Ho, Ho, Ho - 12.0 | BSD Now 276 first appeared on Jupiter Broadcasting.

]]> OpenBSD in Stereo | BSD Now 275 https://original.jupiterbroadcasting.net/128321/openbsd-in-stereo-bsd-now-275/ Sun, 09 Dec 2018 11:02:53 +0000 https://original.jupiterbroadcasting.net/?p=128321 ##Headlines ###DragonflyBSD 5.4 released DragonFly version 5.4 brings a new system compiler in GCC 8, improved NUMA support, a large of number network and virtual machine driver updates, and updates to video support. This release is 64-bit only, as with previous releases. The details of all commits between the 5.2 and 5.4 branches are available […]

The post OpenBSD in Stereo | BSD Now 275 first appeared on Jupiter Broadcasting.

]]>

##Headlines
###DragonflyBSD 5.4 released

DragonFly version 5.4 brings a new system compiler in GCC 8, improved NUMA support, a large of number network and virtual machine driver updates, and updates to video support. This release is 64-bit only, as with previous releases.
The details of all commits between the 5.2 and 5.4 branches are available in the associated commit messages for 5.4.0rc and 5.4.0.

  • Big-ticket items
  • Much better support for asymmetric NUMA (Non-Uniform Memory Access) configurations. In particular, both the memory subsystem and the scheduler now understand the Threadripper 2990WX’s architecture. The scheduler will prioritize CPU nodes with direct-attached memory and the memory subsystem will normalize memory queues for CPU nodes without direct-attached memory (which improves cache locality on those CPUs).
  • Incremental performance work. DragonFly as a whole is very SMP friendly. The type of performance work we are doing now mostly revolves around improving fairness for shared-vs-exclusive lock clashes, reducing cache ping-ponging due to non-contending SMP locks (i.e. massive use of shared locks on shared resources), and so forth.
  • Major updates to dports brings us to within a week or two of FreeBSD’s ports as of this writing, in particular major updates to chromium, and making the whole mess work with gcc-8.
  • Major rewriting of the tty clist code and the tty locking code, significantly improving concurrency across multiple ttys and ptys.
  • GCC 8
  • DragonFly now ships with GCC 8.0, and runs as the default compiler. It is also now used for building dports.
  • GCC 4.7.4 and GCC 5.4.1 are still installed. 4.7.4 is our backup compiler, and 5.4.1 is still there to ensure a smooth transition, but should generally not be used. buildworld builds all three by default to ensure maximum compatibility.
  • Many passes through world sources were made to address various warnings and errors the new GCC brought with it.
  • HAMMER2
  • HAMMER2 is recommended as the default root filesystem in non-clustered mode.
  • Clustered support is not yet available.
  • Increased bulkfree cache to reduce the number of iterations required.
  • Fixed numerous bugs.
  • Improved support on low-memory machines.
  • Significant pre-work on the XOP API to help support future networked operations.
  • Details
  • Checksums
    MD5 (dfly-x86_64-5.4.0_REL.img) = 7277d7cffc92837c7d1c5dd11a11b98f
    MD5 (dfly-x86_64-5.4.0_REL.iso) = 6da7abf036fe9267479837b3c3078408
    MD5 (dfly-x86_64-5.4.0_REL.img.bz2) = a77a072c864f4b72fd56b4250c983ff1
    MD5 (dfly-x86_64-5.4.0_REL.iso.bz2) = 4dbfec6ccfc1d59c5049455db914d499
  • Downloads Links

DragonFly BSD is 64-bit only, as announced during the 3.8 release.

  • USB: dfly-x86_64-5.4.0_REL.img as bzip2 file
  • ISO: dfly-x86_64-5.4.0_REL.iso as bzip2 file
  • Uncompressed ISO: dfly-x86_64-5.4.0_REL.iso (For use with VPS providers as an install image.)

###Down the Gopher hole with OpenBSD, Gophernicus, and TLS

In the early 2000s I thought I had seen the worst of the web – Java applets, Macromedia (>Adobe) Flash, animated GIFs, javascript snow that kept you warm in the winter by burning out your CPU, and so on. For a time we learned from these mistakes, and started putting the burden on the server-side – then with improvements in javascript engines we started abusing it again with JSON/AJAX and it all went down hill from there.

Like cloud computing, blockchains, machine learning and a tonne of other a la mode technologies around today – most users and service providers don’t need websites that consume 1GB of memory processing JS and downloading 50MB of compressed data just to read Alice’s one-page travel blog or Bob’s notes on porting NetBSD to his blood-pressure monitor.

Before the HTTP web we relied on Prestel/Minitel style systems, BBS systems, and arguably the most accessible of all – Gopher! Gopher was similar to the locally accessed AmigaGuide format, in that it allowed users to search and retrieve documents interactively, with links and cross-references. Its efficiency and distraction-free nature make it attractive to those who are tired of the invasive, clickbait, ad-filled, javascript-laden web2/3.x. But enough complaining and evangelism – here’s how to get your own Gopher Hole!

Gophernicus is a modern gopher daemon which aims to be secure (although it still uses inetd -_-); it’s even in OpenBSD ports so at least we can rely on it to be reasonably audited.

If you need a starting point with Gopher, SDF-EU’s wiki has a good article here.

Finally, if you don’t like gopher(1) – there’s always lynx(1) or NCSA Mosaic!

I’ve added TLS support to Gophernicus so you don’t need to use stunnel anymore. The code is ugly and unpolished though so I wouldn’t recommend for production use.

##News Roundup
###OpenBSD in Stereo with Linux VFIO

I use a Huawei Matebook X as my primary OpenBSD laptop and one aspect of its hardware support has always been lacking: audio never played out of the right-side speaker. The speaker did actually work, but only in Windows and only after the Realtek Dolby Atmos audio driver from Huawei was installed. Under OpenBSD and Linux, and even Windows with the default Intel sound driver, audio only ever played out of the left speaker.
Now, after some extensive reverse engineering and debugging with the help of VFIO on Linux, I finally have audio playing out of both speakers on OpenBSD.

  • VFIO

The Linux kernel has functionality called VFIO which enables direct access to a physical device (like a PCI card) from userspace, usually passing it to an emulator like QEMU.
To my surprise, these days, it seems to be primarily by gamers who boot Linux, then use QEMU to run a game in Windows and use VFIO to pass the computer’s GPU device through to Windows.
By using Linux and VFIO, I was able to boot Windows 10 inside of QEMU and pass my laptop’s PCI audio device through to Windows, allowing the Realtek audio drivers to natively control the audio device. Combined with QEMU’s tracing functionality, I was able to get a log of all PCI I/O between Windows and the PCI audio device.

  • Using VFIO

To use VFIO to pass-through a PCI device, it first needs to be stubbed out so the Linux kernel’s default drivers don’t attach to it. GRUB can be configured to instruct the kernel to ignore the PCI audio device (8086:9d71) and explicitly enable the Intel IOMMU driver by adding the following to /etc/default/grub and running update-grub
With the audio device stubbed out, a new VFIO device can be created from it
Then the VFIO device (00:1f.3) can be passed to QEMU
I was using my own build of QEMU for this, due to some custom logging I needed (more on that later), but the default QEMU package should work fine. The events.txt was a file of all VFIO events I wanted logged (which was all of them).
Since I was frequently killing QEMU and restarting it, Windows 10 wanted to go through its unexpected shutdown routine each time (and would sometimes just fail to boot again). To avoid this and to get a consistent set of logs each time, I used qemu-img to take a snapshot of a base image first, then boot QEMU with that snapshot. The snapshot just gets thrown away the next time qemu-img is run and Windows always starts from a consistent state.
QEMU will now log each VFIO event which gets saved to a debug-output file.
With a full log of all PCI I/O activity from Windows, I compared it to the output from OpenBSD and tried to find the magic register writes that enabled the second speaker. After days of combing through the logs and annotating them by looking up hex values in the documentation, diffing runtime register values, and even brute-forcing it by mechanically duplicating all PCI I/O activity in the OpenBSD driver, nothing would activate the right speaker.
One strange thing that I noticed was if I booted Windows 10 in QEMU and it activated the speaker, then booted OpenBSD in QEMU without resetting the PCI device’s power in-between (as a normal system reboot would do), both speakers worked in OpenBSD and the configuration that the HDA controller presented was different, even without any changes in OpenBSD.

A Primer on Intel HDA
Most modern computers with integrated sound chips use an Intel High Definition Audio (HDA) Controller device, with one or more codecs (like the Realtek ALC269) hanging off of it. These codecs do the actual audio processing and communicate with DACs and ADCs to send digital audio to the connected speakers, or read analog audio from a microphone and convert it to a digital input stream. In my Huawei Matebook X, this is done through a Realtek ALC298 codec.
On OpenBSD, these HDA controllers are supported by the azalia(4) driver, with all of the per-codec details in the lengthy azalia_codec.c file. This file has grown quite large with lots of codec- and machine-specific quirks to route things properly, toggle various GPIO pins, and unmute speakers that are for some reason muted by default.
The azalia driver talks to the HDA controller and sets up various buffers and then walks the list of codecs. Each codec supports a number of widget nodes which can be interconnected in various ways. Some of these nodes can be reconfigured on the fly to do things like turning a microphone port into a headphone port.
The newer Huawei Matebook X Pro released a few months ago is also plagued with this speaker problem, although it has four speakers and only two work by default. A fix is being proposed for the Linux kernel which just reconfigures those widget pins in the Intel HDA driver. Unfortunately no pin reconfiguration is enough to fix my Matebook X with its two speakers.
While reading more documentation on the HDA, I realized there was a lot more activity going on than I was able to see through the PCI tracing.
For speed and efficiency, HDA controllers use a DMA engine to transfer audio streams as well as the commands from the OS driver to the codecs. In the output above, the CORBWP=0; size=256 and RIRBRP=0, size=256 indicate the setup of the CORB (Command Output Ring Buffer) and RIRB (Response Input Ring Buffer) each with 256 entries. The HDA driver allocates a DMA address and then writes it to the two CORBLBASE and CORBUBASE registers, and again for the RIRB.
When the driver wants to send a command to a codec, such as CORB_GET_PARAMETER with a parameter of COP_VOLUME_KNOB_CAPABILITIES, it encodes the codec address, the node index, the command verb, and the parameter, and then writes that value to the CORB ring at the address it set up with the controller at initialization time (CORBLBASE/CORBUBASE) plus the offset of the ring index. Once the command is on the ring, it does a PCI write to the CORBWP register, advancing it by one. This lets the controller know a new command is queued, which it then acts on and writes the response value on the RIRB ring at the same position as the command (but at the RIRB’s DMA address). It then generates an interrupt, telling the driver to read the new RIRBWP value and process the new results.
Since the actual command contents and responses are handled through DMA writes and reads, these important values weren’t showing up in the VFIO PCI trace output that I had gathered. Time to hack QEMU.

  • Logging DMA Memory Values in QEMU

Since DMA activity wouldn’t show up through QEMU’s VFIO tracing and I obviously couldn’t get Windows to dump these values like I could in OpenBSD, I could make QEMU recognize the PCI write to the CORBWP register as an indication that a command has just been written to the CORB ring.
My custom hack in QEMU adds some HDA awareness to remember the CORB and RIRB DMA addresses as they get programmed in the controller. Then any time a PCI write to the CORBWP register is done, QEMU fetches the new CORB command from DMA memory, decodes it into the codec address, node address, command, and parameter, and prints it out. When a PCI read of the RIRBWP register is requested, QEMU reads the response and prints the corresponding CORB command that it stored earlier.
With this hack in place, I now had a full log of all CORB commands and RIRB responses sent to and read from the codec:
An early version of this patch left me stumped for a few days because, even after submitting all of the same CORB commands in OpenBSD, the second speaker still didn’t work. It wasn’t until re-reading the HDA spec that I realized the Windows driver was submitting more than one command at a time, writing multiple CORB entries and writing a CORBWP value that was advanced by two. This required turning my CORB/RIRB reading into a for loop, reading each new command and response between the new CORBWP/RIRBWP value and the one previously seen.
Sure enough, the magic commands to enable the second speaker were sent in these periods where it submitted more than one command at a time.

  • Minimizing the Magic

The full log of VFIO PCI activity from the Windows driver was over 65,000 lines and contained 3,150 CORB commands, which is a lot to sort through. It took me a couple more days to reduce that down to a small subset that was actually required to activate the second speaker, and that could only be done through trial and error:

  • Boot OpenBSD with the full list of CORB commands in the azalia driver
  • Comment out a group of them
  • Compile kernel and install it, halt the QEMU guest
  • Suspend and wake the laptop, resetting PCI power to the audio device to reset the speaker/Dolby initialization and ensure the previous run isn’t influencing the current test (I’m guessing there is an easier to way to reset PCI power than suspending the laptop, but oh well)
  • Start QEMU, boot OpenBSD with the new kernel
  • Play an MP3 with mpg123 which has alternating left- and right-channel audio and listen for both channels to play

This required a dozen or so iterations because sometimes I’d comment out too many commands and the right speaker would stop working. Other times the combination of commands would hang the controller and it wouldn’t process any further commands. At one point the combination of commands actually flipped the channels around so the right channel audio was playing through the left speaker.

  • The Result

After about a week of this routine, I ended up with a list of 662 CORB commands that are needed to get the second speaker working. Based on the number of repeated-but-slightly-different values written with the 0x500 and 0x400 commands, I’m guessing this is some kind of training data and that this is doing the full Dolby/Atmos system initialization, not just turning on the second speaker, but I could be completely wrong.
In any case, the stereo sound from OpenBSD is wonderful now and I can finally stop downmixing everything to mono to play from the left speaker. In case you ever need to do this, sndiod can be run with -c 0:0 to reduce the channels to one.
Due to the massive size of the code needed for this quirk, I’m not sure if I’ll be committing it upstream in OpenBSD or just saving it for my own tree. But at least now the hardware support chart for my Matebook is all yeses for the things I care about.
I’ve also updated the Linux bug report that I opened before venturing down this path, hoping one of the maintainers of that HDA code that works at Intel or Realtek knew of a solution I could just port to OpenBSD. I’m curious to see what they’ll do with it.

###Why BSD/OS is the best candidate for being the only tested legally open UNIX

  • Introduction

The UNIX® system is an old operating system, possibly older than many of the readers of this post. However, despite its age, it still has not been open sourced completely. In this post, I will try to detail which parts of which UNIX systems have not yet been open sourced. I will focus on the legal situation in Germany in particular, taking it representative of European law in general – albeit that is a stretch, knowing the diversity of European jurisdictions. Please note that familiarity with basic terms of copyright law is assumed.

  • Ancient UNIX

The term “Ancient UNIX” refers to the versions of UNIX up to and including Seventh Edition UNIX (1979) including the 32V port to the VAX. Ancient UNIX was created at Bell Laboratories, a subsidiary of AT&T at the time. It was later transferred of the AT&T UNIX Support Group, then AT&T Information Systems and finally the AT&T subsidiary UNIX System Laboratories, Inc. (USL). The legal situation differs between the United States of America and Germany.
In a ruling as part of the UNIX System Laboratories, Inc. v. Berkeley Software Design, Inc. (USL v. BSDi) case, a U.S. court found that USL had no copyright to the Seventh Edition UNIX system and 32V – arguably, by extension, all earlier versions of Ancient UNIX as well – because USL/AT&T had failed to affix copyright notices and could not demonstrate a trade secret. Due to the obsessive tendency of U.S. courts to consider themselves bound to precedents (cf. the infamous Pierson v. Post case), it can be reasonably expected that this ruling would be honored and applied in subsequent cases. Thus under U.S. law, Ancient UNIX can be safely assumed to belong in the public domain.
The situation differs in Germany. Unlike the U.S., copyright never needed registration in order to exist. Computer programs are works in the sense of the German 1965 Act on Copyright and Related Rights (Copyright Act, henceforth CopyA) as per CopyA § 2(1) no. 1. Even prior to the amendment of CopyA § 2(1) to include computer programs, computer programs have been recognized as copyrightable works by the German Supreme Court (BGHZ 112, 264 Betriebssystem, no. 19); CopyA § 137d(1) rightly clarifies that. The copyright holder at 1979 would still have been USL via Bell Labs and AT&T. Copyright of computer programs is transferred to the employer upon creation under CopyA § 69(1).
Note that this does not affect expiry (Daniel Kaboth/Benjamin Spies, commentary on CopyA §§ 69a‒69g, in: Hartwig Ahlberg/Horst-Peter Götting (eds.), Urheberrecht: UrhG, KUG, VerlG, VGG, Kommentar, 4th ed., C. H. Beck, 2018, no. 16 ad CopyA § 69b; cf. Bundestag-Drucksache [BT-Drs.] 12/4022, p. 10). Expiry occurs 70 years after the death of the (co-)author that died most recently as per CopyA § 65(1) and 64; this has been the case since at least the 1960s, meaning there is no way for copyright to have expired already (old version, as per Bundesgesetzblatt Part I No. 51 of September 16, 1965, pp. 1273‒1294).
In Germany, private international law applies the so-called “Territorialitätsprinzip” for intellectual property rights. This means that the effect of an intellectual property right is limited to the territory of a state (Anne Lauber-Rönsberg, KollisionsR, in: Hartwig Ahlberg/Horst-Peter Götting (eds.), ibid., pp. 2241 et seqq., no. 4). Additionally, the “Schutzlandprinzip” applies; this means that protection of intellectual property follows the lex loci protectionis, i.e. the law of the country for which protection is sought (BGH GRUR 2015, 264 HiHotel II, no. 25; BGH GRUR 2003, 328 Sender Felsberg, no. 24), albeit this is criticized in parts of doctrine (Lauber-Rönsberg, ibid., no. 10). The “Schutzlandprinzip” requires that the existence of an intellectual property right be verified as well (BGH ZUM 2016, 522 Wagenfeld-Leuchte II, no. 19).
Thus, in Germany, copyright on Ancient UNIX is still alive and well. Who has it, though? A ruling by the U.S. Court of Appeals, Tenth Circuit, in the case of The SCO Group, Inc. v. Novell, Inc. (SCO v. Novell) in the U.S. made clear that Novell owns the rights to System V – thus presumably UNIX System III as well – and Ancient UNIX, though SCO acquired enough rights to develop UnixWare/OpenServer (Ruling 10-4122 [D.C. No. 2:04-CV-00139-TS], pp. 19 et seq.). Novell itself was purchased by the Attachmate Group, which was in turn acquired by the COBOL vendor Micro Focus. Therefore, the rights to SVRX and – outside the U.S. – are with Micro Focus right now. If all you care about is the U.S., you can stop reading about Ancient UNIX here.
So how does the Caldera license factor into all of this? For some context, the license was issued January 23, 2002 and covers Ancient UNIX (V1 through V7 including 32V), specifically excluding System III and System V. Caldera, Inc. was founded in 1994. The Santa Cruz Operation, Inc. sold its rights to UNIX to Caldera in 2001, renamed itself to Tarantella Inc. and Caldera renamed itself The SCO Group. Nemo plus iuris ad alium transferre potest quam ipse habet; no one can transfer more rights than he has. The question now becomes whether Caldera had the rights to issue the Caldera license.
I’ve noted it above but it needs restating: Foreign decisions are not necessarily accepted in Germany due to the “Territorialitätsprinzip” and “Schutzlandprinzip” – however, I will be citing a U.S. ruling for its assessment of the facts for the sake of simplicity. As per ruling 10-4122, “The district court found the parties intended for SCO to serve as Novell’s agent with respect to the old SVRX licenses and the only portion of the UNIX business transferred outright under the APA [asset purchase agreement] was the ability to exploit and further develop the newer UnixWare system. SCO was able to protect that business because it was able to copyright its own improvements to the system. The only reason to protect the earlier UNIX code would be to protect the existing SVRX licenses, and the court concluded Novell retained ultimate control over that portion of the business under the APA.” The relevant agreements consist of multiple pieces:
the base Asset Purchase Agreement “APA” (Part I)
the base Asset Purchase Agreement “APA” (Part II)
the Operating Agremeent and Amendment 1 to the APA
the Amendment 2 to the APA
The APA dates September 19, 1995, from before the Caldera license. Caldera cannot possibly have acquired rights that The Santa Cruz Operation, Inc. itself never had. Furthermore, I’ve failed to find any mention of Ancient UNIX; all that is transferred is rights to SVRX. Overall, I believe that the U.S. courts’ assesment of the facts represents the situation accurately. Thus for all intents and purposes, UNIX up to and including System V remained with Novell/Attachmate/Micro Focus. Caldera therefore never had any rights to Ancient UNIX, which means it never had the rights to issue the Caldera license. The Caldera license is null and void – in the U.S. because the copyright has been lost due to formalities, everywhere else because Caldera never had the rights to issue it.
The first step to truly freeing UNIX would this be to get Micro Focus to re-issue the Caldera license for Ancient UNIX, ideally it would now also include System III and System V.

  • BSD/OS

Another operating system near UNIX is of interest. The USL v. BSDi lawsuit includes two parties: USL, which we have seen above, and Berkeley Software Design, Inc. BSDi sold BSD/386 (later BSD/OS), which was a derivative of 4.4BSD. The software parts of the BSDi company were acquired by Wind River Systems, whereas the hardware parts went to iXsystems. Copyright is not disputed there, though Wind River Systems ceased selling BSD/OS products 15 years ago, in 2003. In addition, Wind River System let their trademark on BSD expire, though this is without consequence for copyright.
BSD/OS is notable in the sense that it powered much of early internet infrastructure. Traces of its legacy can still be found on Richard Stevens’ FAQ.
To truly make UNIX history free, BSD/OS would arguably also need to see a source code release. BSD/OS at least in its earliest releases under BSDi would ship with source code, though under a non-free license, far from BSD or even GPL licensing.

  • System V

The fate of System V as a whole is difficult to determine. Various licenses have been granted to a number of vendors (Dell UNIX comes to mind; HP for HP-UX, IBM for AIX, SGI UNIX, etc.). Sun released OpenSolaris – notoriously, Oracle closed the source to Solaris again after its release –, which is a System V Release 4 descendant. However, this means nothing for the copyright or licensing status of System V itself. Presumably, the rights with System V still remain with Novell (now Micro Focus): SCO managed to sublicense rights to develop and sell UnixWare/OpenServer, themselves System V/III descendants, to unXis, Inc. (now known as Xinuos, Inc.), which implies that Xinuos is not the copyright holder of System V.
Obviously, to free UNIX, System V and its entire family of descendants would also need to be open sourced. However, I expect tremendous resistance on part of all the companies mentioned. As noted in the “Ancient UNIX” section, Micro Focus alone would probably be sufficient to release System V, though this would mean nothing for the other commercial System V derivatives.

  • Newer Research UNIX

The fate of Bell Labs would be a different one; it would go on to be purchased by Lucent, now part of Nokia. After commercial UNIX got separated out to USL, Research UNIX would continue to exist inside of Bell Labs. Research UNIX V8, V9 and V10 were not quite released by Alcatel-Lucent USA Inc. and Nokia in 2017.
However, this is merely a notice that the companies involved will not assert their copyrights only with respect to any non-commercial usage of the code. It is still not possible, over 30 years later, to freely use the V8 code.

  • Conclusion
    In the U.S., Ancient UNIX is freely available. People located everywhere else, however, are unable to legally obtain UNIX code for any of the systems mentioned above. The exception being BSD/OS, assuming a purchase of a legitimate copy of the source code CD. This is deeply unsatisfying and I implore all involved companies to consider open sourcing (preferably under a BSD-style license) their code older than a decade, if nothing else, then at least for the sake of historical purposes. I would like to encourage everybody reading this to consider reaching out to Micro Focus and Wind River Systems about System V and BSD/OS, respectively. Perhaps the masses can change their minds.

A small note about patents: Some technologies used in newer iterations of the UNIX system (in particular the System V derivatives) may be encumbered with software patents. An open source license will not help against patent infringement claims. However, the patents on anything used in the historical operating systems will certainly have expired by now. In addition, European readers can ignore this entirely – software patents just aren’t a thing.

###OpenBGPD – Adding Diversity to the Route Server Landscape

  • Introduction

As of last year, there was effectively only a single solution in the Route Server vendor market: the BIRD Internet routing daemon. NIC.CZ (the organisation developing BIRD) has done fantastic work on maintaining their BGP-4 implementation, however, it’s not healthy to have virtually every Internet Exchange Point (IXP) in the RIPE NCC service region depend on a single open source project. The current situation can be compared to the state of the DNS root nameservers back in 2002 – their dependence on the BIND nameserver daemon and the resulting development of NSD as an alternative by NLnet, in cooperation with the RIPE NCC.
OpenBGPD used to be one of the most popular Route Server implementations until the early 2010s. OpenBGPD’s main problem was that its performance couldn’t keep up with the Internet’s growth, so it lost market share. An analysis by Job Snijders suggested that a modernised OpenBGPD distribution would be a most viable option to regain diversity on the Route Server level.

  • Missing features in OpenBGPD

The following main missing features were identified in OpenBGPD:

  • Performance

In previous versions of OpenBGPD, the filtering performance didn’t allow proper filtering of all EBGP sessions. Current best practice at IXP Route Servers is to carefully evaluate and validate of all routes learned from EBGP peers. The OpenBGPD ruleset required to do correct filtering (in many deployment scenarios) was simply too lengthy – and negatively impacted service performance during configuration reloads. While filtering performance is the biggest bottleneck, general improvements to the Routing Information Base were also made to improve scalability. IXP Route Servers with a few hundred peering sessions are commonplace and adding new sessions shouldn’t impact the Route Servers’ service to other peers. We found that performance was the most pressing issue that needed to be tackled.

  • Lack of RPKI Origin Validation

As we’ve seen, Internet operators are moving to adopt RPKI based BGP Origin Validation. While it was theoretically possible to emulate RFC 6811-style Origin Validation in previous versions of OpenBGPD, the required configuration wasn’t optimised for performance and wasn’t user friendly. We believe that BGP Origin Validation should be as easy as possible – this requires BGP-4 vendors to implement native, optimised routines for Origin Validation. Of course, enabling Origin Validation shouldn’t have an impact on performance either when processing BGP updates or when updating the Route Origin Authorisation (ROA) table itself.

  • Portability

OpenBGPD is an integral part of OpenBSD, but IXPs may prefer to run their services infrastructure on an operating system of their choice. Making sure that there’s a portable OpenBGPD version which follows the OpenBSD project release cycle will give IXPs this option.

  • Development steps

By addressing the issues mentioned above, we could bring back OpenBGPD as a viable Route Server implementation.
Since I was one of the core OpenBGPD developers, I was asked if I wanted to pick up this project again. Thanks to the funding from the RIPE NCC Project Fund, this was possible. Starting in June 2018, I worked full time on this important community project. Over the last few months, many of the problems are already addressed and are now part of the OpenBSD 6.4 release. So far, 154 commits were made to OpenBGPD during the 6.4 development cycle – around 8% of all commits ever to OpenBGPD! This shows that due to funding and dedicated resources, a lot of work could be pushed into the latest release of OpenBGPD.

  • OpenBGPD 6.4

The OpenBGPD version, as part of OpenBSD 6.4 release, demonstrates great progress. Even though there have been many changes to the core of OpenBGPD, the released version is as solid and reliable as previous releases and the many bug fixes and improvements make this the best OpenBGPD release so far. The changes in the filter language allow users to write more efficient rulesets while the introduction of RPKI origination validation fixes an important missing feature. For IXPs, OpenBGPD now is an alternative again. There are still open issues, but the gap is closing!

  • Feature highlights

The following changes should be highlighted:

  • Introduction of background soft-reconfiguration on config reload. Running the soft-reconfiguration task in the background allows for new updates and withdraws to be processed at the same time. This improves convergence time – one of the key metrics for Route Servers.
  • BGP Origin Validation when a roa-set is configured Every EBGP route announcement is validated against the locally configured VRP table entries. Depending on the validation process’s outcome, the validation state is set to valid, invalid or not found. The filter language has been extended to allow checking for the origin validation state, and thanks to this, it is possible to deny invalid prefixes or regard valid prefixes different to the ones that aren’t found. The roa-set table is read from the configuration file and updated during configuration reloads. On production systems reloading the roa-set and applying it to all prefixes is done in a couple of seconds.
  • Fast prefix-set lookups In OpenBSD 6.3 prefix-sets got introduced in OpenBGPD. A prefix-set combines many prefix lookups into a single filter rule. The original implementation wasn’t optimised but now a fast trie lookup is used. Thanks to this, large IRR DB prefix tables can now be implemented efficiently.
  • Introduction of as-sets Similar to prefix-sets, as-sets help group many AS numbers into a single lookup. Thanks to this, large IRR DB origin AS tables can be implemented efficiently.
    Introduction of origin-sets
  • Looking at the configurations of Route Servers doing full filtering, it was noticed that a common lookup was binding a prefix to an origin AS – similar to how a roa-set is used for RPKI. These origin-set tables are used to extend the IRR prefix lookup and generated from alternative sources.
  • Improving third party tools

Users can only benefit from the changes introduced in OpenBGPD 6.4 when the surrounding 3rd party tools are adjusted accordingly. Two opensource projects such as bgpq3 and arouteserver are frequently used by network operators and IXPs to generate BGP configurations. Thanks to our contributions to those projects, we were able to get them ready for all the new features in OpenBGPD.

  • bgpq3 was extended to create as-set and prefix-set tables based on IRR DB entries. This is replacing the old way of doing the same with a large amount of filter rules. Thanks to the quick response from the bgpq3 maintainer, it was possible to ship OpenBSD 6.4 with a bgpq3 package that includes all the new features.
  • arouteserver was adjusted to implement RPKI roa-set, as-set, prefix-set, and origin-set to generate a much better-performing configurations for the 6.4 version. With the v0.20.0 release of arouteserver, IXPs are able to generate an OpenBGPD configuration which is a ton faster but also implements the new functionalities. Looking at YYCIX (the resident IXP in Calgary, Canada) the ruleset generated by arouteserver was reduced from 370,000 rules to well under 6,000 rules. This resulted in the initial convergence time dropping from over 1 hour to less than 2 minutes, and subsequent configuration reloads are hitless and no longer noticeable.
  • What still needs to be done

A sizeable chunk of work still left on the table is the rework of the RIB data structures in OpenBGPD – these haven’t been changed since the initial design of OpenBGPD in 2003. There’s currently ongoing work (in small steps, to avoid jeopardising the stability of OpenBGPD) to modernise these data-structures. The goal is to provide better decoupling of the filter step from storing RIB database changes, to pave the way to multi-threaded operations at a later point.

  • Looking forward
  • Job Snijders oversaw this year’s fundraising and project management, he adds:

It’s been incredibly productive to create an environment where a core developer is allowed to work full time on the OpenBGPD code base. However, it’s important to note there still is room for a number of new features to help improve its operational capabilities (such as BMP, RFC 7313, ADD_PATH, etc). It’d be beneficial to the Internet community at large if we can extend Claudio Jeker’s involvement for another year. Open source software doesn’t grow on trees! Strategic investments are the only way to keep OpenBGPD’s roadmap aligned with Internet growth and operator requirements.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

The post OpenBSD in Stereo | BSD Now 275 first appeared on Jupiter Broadcasting.

]]> Firecracker Fundamentals | TechSNAP 391 https://original.jupiterbroadcasting.net/128256/firecracker-fundamentals-techsnap-391/ Fri, 30 Nov 2018 08:21:16 +0000 https://original.jupiterbroadcasting.net/?p=128256 Show Notes: techsnap.systems/391

The post Firecracker Fundamentals | TechSNAP 391 first appeared on Jupiter Broadcasting.

]]>

Show Notes: techsnap.systems/391

The post Firecracker Fundamentals | TechSNAP 391 first appeared on Jupiter Broadcasting.

]]> A Thoughtful Episode | BSD Now 273 https://original.jupiterbroadcasting.net/128146/a-thoughtful-episode-bsd-now-273/ Wed, 21 Nov 2018 08:40:23 +0000 https://original.jupiterbroadcasting.net/?p=128146 ##Headlines ###Some thoughts on NetBSD 8.0 NetBSD is a highly portable operating system which can be run on dozens of different hardware architectures. The operating system’s clean and minimal design allow it to be run in all sorts of environments, ranging from embedded devices, to servers, to workstations. While the base operating system is minimal, […]

The post A Thoughtful Episode | BSD Now 273 first appeared on Jupiter Broadcasting.

]]>

##Headlines
###Some thoughts on NetBSD 8.0

NetBSD is a highly portable operating system which can be run on dozens of different hardware architectures. The operating system’s clean and minimal design allow it to be run in all sorts of environments, ranging from embedded devices, to servers, to workstations. While the base operating system is minimal, NetBSD users have access to a large repository of binary packages and a ports tree which I will touch upon later.
I last tried NetBSD 7.0 about three years ago and decided it was time to test drive the operating system again. In the past three years NetBSD has introduced a few new features, many of them security enhancements. For example, NetBSD now supports write exclusive-or execute (W^X) protection and address space layout randomization (ASLR) to protect programs against common attacks. NetBSD 8.0 also includes USB3 support and the ability to work with ZFS storage volumes.

  • Early impressions

Since I had set up NetBSD with a Full install and enabled xdm during the setup process, the operating system booted to a graphical login screen. From here we can sign into our account. The login screen does not provide options to shut down or restart the computer. Logging into our account brings up the twm window manager and provides a virtual terminal, courtesy of xterm. There is a panel that provides a method for logging out of the window manager. The twm environment is sparse, fast and devoid of distractions.

  • Software management

NetBSD ships with a fairly standard collection of command line tools and manual pages, but otherwise it is a fairly minimal platform. If we want to run network services, have access to a web browser, or use a word processor we are going to need to install more software. There are two main approaches to installing new packages. The first, and easier approach, is to use the pkgin package manager. The pkgin utility works much the same way APT or DNF work in the Linux world, or as pkg works on FreeBSD. We can search for software by name, install or remove items. I found pkgin worked well, though its output can be terse. My only complaint with pkgin is that it does not handle “close enough” package names. For example, if I tried to run “pkgin install vlc” or “pkgin install firefox” I would quickly be told these items did not exist. But a more forgiving package manager will realize items like vlc2 or firefox45 are available and offer to install those.
The pkgin tool installs new programs in the /usr/pkg/bin directory. Depending on your configuration and shell, this location may not be in your user’s path, and it will be helpful to adjust your PATH variable accordingly.
The other common approach to acquiring new software is to use the pkgsrc framework. I have talked about using pkgsrc before and I will skip the details. Basically, we can download a collection of recipes for building popular open source software and run a command to download and install these items from their source code. Using pkgsrc basically gives us the same software as using pkgin would, but with some added flexibility on the options we use.
Once new software has been installed, it may need to be enabled and activated, particularly if it uses (or is) a background service. New items can be enabled in the /etc/rc.conf file and started or stopped using the service command. This works about the same as the service command on FreeBSD and most non-systemd Linux distributions.

  • Hardware

I found that, when logged into the twm environment, NetBSD used about 130MB of RAM. This included kernel memory and all active memory. A fresh, Full install used up 1.5GB of disk space. I generally found NetBSD ran well in both VirtualBox and on my desktop computer. The system was quick and stable. I did have trouble getting a higher screen resolution in both environments. NetBSD does not offer VirtualBox add-on modules. There are NetBSD patches for VirtualBox out there, but there is some manual work involved in getting them working. When running on my desktop computer I think the resolution issue was one of finding and dealing with the correct video driver. Screen resolution aside, NetBSD performed well and detected all my hardware.

  • Personal projects

Since NetBSD provides users with a small, core operating system without many utilities if we want to use NetBSD for something we need to have a project in mind. I had four mini projects in mind I wanted to try this week: install a desktop environment, enable file sharing for computers on the local network, test multimedia (video, audio and YouTube capabilities), and set up a ZFS volume for storage.
I began with the desktop. Specifically, I followed the same tutorial I used three years ago to try to set up the Xfce desktop. While Xfce and its supporting services installed, I was unable to get a working desktop out of the experience. I could get the Xfce window manager working, but not the entire session. This tutorial worked beautifully with NetBSD 7.0, but not with version 8.0. Undeterred, I switched gears and installed Fluxbox instead. This gave me a slightly more powerful graphical environment than what I had before with twm while maintaining performance. Fluxbox ran without any problems, though its application menu was automatically populated with many programs which were not actually installed.
Next, I tried installing a few multimedia applications to play audio and video files. Here I ran into a couple of interesting problems. I found the music players I installed would play audio files, but the audio was quite slow. It always sounded like a cassette tape dragging. When I tried to play a video, the entire graphical session would crash, taking me back to the login screen. When I installed Firefox, I found I could play YouTube videos, and the video played smoothly, but again the audio was unusually slow.
I set up two methods of sharing files on the local network: OpenSSH and FTP. NetBSD basically gives us OpenSSH for free at install time and I added an FTP server through the pkgin package manager which worked beautifully with its default configuration.
I experimented with ZFS support a little, just enough to confirm I could create and access ZFS volumes. ZFS seems to work on NetBSD just as well, and with the same basic features, as it does on FreeBSD and mainstream Linux distributions. I think this is a good feature for the portable operating system to have since it means we can stick NetBSD on nearly any networked computer and use it as a NAS.

  • Conclusions

NetBSD, like its close cousins (FreeBSD and OpenBSD) does not do a lot of hand holding or automation. It offers a foundation that will run on most CPUs and we can choose to build on that foundation. I mention this because, on its own, NetBSD does not do much. If we want to get something out of it, we need to be willing to build on its foundation – we need a project. This is important to keep in mind as I think going into NetBSD and thinking, “Oh I’ll just explore around and expand on this as I go,” will likely lead to disappointment. I recommend figuring out what you want to do before installing NetBSD and making sure the required tools are available in the operating system’s repositories.
Some of the projects I embarked on this week (using ZFS and setting up file sharing) worked well. Others, like getting multimedia support and a full-featured desktop, did not. Given more time, I’m sure I could find a suitable desktop to install (along with the required documentation to get it and its services running), or customize one based on one of the available window managers. However, any full featured desktop is going to require some manual work. Media support was not great. The right players and codecs were there, but I was not able to get audio to play smoothly.
My main complaint with NetBSD relates to my struggle to get some features working to my satisfaction: the documentation is scattered. There are four different sections of the project’s website for documentation (FAQs, The Guide, manual pages and the wiki). Whatever we are looking for is likely to be in one of those, but which one? Or, just as likely, the tutorial we want is not there, but is on a forum or blog somewhere. I found that the documentation provided was often thin, more of a quick reference to remind people how something works rather than a full explanation.
As an example, I found a couple of documents relating to setting up a firewall. One dealt with networking NetBSD on a LAN, another explored IPv6 support, but neither gave an overview on syntax or a basic guide to blocking all but one or two ports. It seemed like that information should already be known, or picked up elsewhere.
Newcomers are likely to be a bit confused by software management guides for the same reason. Some pages refer to using a tool called pkg_add, others use pkgsrc and its make utility, others mention pkgin. Ultimately, these tools each give approximately the same result, but work differently and yet are mentioned almost interchangeably. I have used NetBSD before a few times and could stumble through these guides, but new users are likely to come away confused.
One quirk of NetBSD, which may be a security feature or an inconvenience, depending on one’s point of view, is super user programs are not included in regular users’ paths. This means we need to change our path if we want to be able to run programs typically used by root. For example, shutdown and mount are not in regular users’ paths by default. This made checking some things tricky for me.
Ultimately though, NetBSD is not famous for its convenience or features so much as its flexibility. The operating system will run on virtually any processor and should work almost identically across multiple platforms. That gives NetBSD users a good deal of consistency across a range of hardware and the chance to experiment with a member of the Unix family on hardware that might not be compatible with Linux or the other BSDs.

###Showing a Gigabit OpenBSD Firewall Some Monitoring Love

I have a pretty long history of running my home servers or firewalls on “exotic” hardware. At first, it was Sun Microsystem hardware, then it moved to the excellent Soekris line, with some cool single board computers thrown in the mix. Recently I’ve been running OpenBSD Octeon on the Ubiquiti Edge Router Lite, an amazing little piece of kit at an amazing price point.

  • Upgrade Time!

This setup has served me for some time and I’ve been extremely happy with it. But, in the #firstworldproblems category, I recently upgraded the household to the amazing Gigabit fibre offering from Sonic. A great problem to have, but also too much of a problem for the little Edge Router Lite (ERL).
The way the OpenBSD PF firewall works, it’s only able to process packets on a single core. Not a problem for the dual-core 500 MHz ERL when you’re pushing under ~200 Mbps, but more of a problem when you’re trying to push 1000 Mbps.
I needed something that was faster on a per core basis but still satisfied my usual firewall requirements. Loosely:

  • small form factor
  • fan-less
  • multiple Intel Ethernet ports (good driver support)
  • low power consumption
  • not your regular off-the-shelf kit
  • relatively inexpensive

After evaluating a LOT of different options I settled on the Protectli Vault FW2B. With the specs required for the firewall (2 GB RAM and 8 GB drive) it comes in at a mere $239 USD! Installation of OpenBSD 6.4 was pretty straight forward, with the only problem I had was Etcher did not want to recognize the ‘.fs’ extension on the install image as bootable image. I quickly fixed this with good old Unix dd(1) on the Mac. Everything else was incredibly smooth.
After loading the same rulesets on my new install, the results were fantastic!

  • Monitoring

Now that the machine was up and running (and fast!), I wanted to know what it was doing. Over the years, I’ve always relied on the venerable pfstat software to give me an overview of my traffic, blocked packets, etc. It looks like this:
As you can see it’s based on RRDtool, which was simply incredible in its time. Having worked on monitoring almost continuously for almost the past decade, I wanted to see if we could re-implement the same functionality using more modern tools as RRDtool and pfstat definitely have their limitations. This might be an opportunity to learn some new things as well.
I came across pf-graphite which seemed to be a great start! He had everything I needed and I added a few more stats from the detailed interface statistics and the ability for the code to exit for running from cron(8), which is a bit more OpenBSD style. I added code for sending to some SaaS metrics platforms but ultimately stuck with straight Graphite. One important thing to note was to use the Graphite pickle port (2004) instead of the default plaintext port for submission. Also you will need to set a loginterface in your ‘pf.conf’.
A bit of tweaking with Graphite and Grafana, and I had a pretty darn good recreation of my original PF stats dashboard!
As you can see it’s based on RRDtool, which was simply incredible in its time. Having worked on monitoring almost continuously for almost the past decade, I wanted to see if we could re-implement the same functionality using more modern tools as RRDtool and pfstat definitely have their limitations. This might be an opportunity to learn some new things as well.
I came across pf-graphite which seemed to be a great start! He had everything I needed and I added a few more stats from the detailed interface statistics and the ability for the code to exit for running from cron(8), which is a bit more OpenBSD style. I added code for sending to some SaaS metrics platforms but ultimately stuck with straight Graphite. One important thing to note was to use the Graphite pickle port (2004) instead of the default plaintext port for submission. Also you will need to set a loginterface in your ‘pf.conf’.
A bit of tweaking with Graphite and Grafana, and I had a pretty darn good recreation of my original PF stats dashboard!

###The Source History of Cat

I once had a debate with members of my extended family about whether a computer science degree is a degree worth pursuing. I was in college at the time and trying to decide whether I should major in computer science. My aunt and a cousin of mine believed that I shouldn’t. They conceded that knowing how to program is of course a useful and lucrative thing, but they argued that the field of computer science advances so quickly that everything I learned would almost immediately be outdated. Better to pick up programming on the side and instead major in a field like economics or physics where the basic principles would be applicable throughout my lifetime.
I knew that my aunt and cousin were wrong and decided to major in computer science. (Sorry, aunt and cousin!) It is easy to see why the average person might believe that a field like computer science, or a profession like software engineering, completely reinvents itself every few years. We had personal computers, then the web, then phones, then machine learning… technology is always changing, so surely all the underlying principles and techniques change too. Of course, the amazing thing is how little actually changes. Most people, I’m sure, would be stunned to know just how old some of the important software on their computer really is. I’m not talking about flashy application software, admittedly—my copy of Firefox, the program I probably use the most on my computer, is not even two weeks old. But, if you pull up the manual page for something like grep, you will see that it has not been updated since 2010 (at least on MacOS). And the original version of grep was written in 1974, which in the computing world was back when dinosaurs roamed Silicon Valley. People (and programs) still depend on grep every day.
My aunt and cousin thought of computer technology as a series of increasingly elaborate sand castles supplanting one another after each high tide clears the beach. The reality, at least in many areas, is that we steadily accumulate programs that have solved problems. We might have to occasionally modify these programs to avoid software rot, but otherwise they can be left alone. grep is a simple program that solves a still-relevant problem, so it survives. Most application programming is done at a very high level, atop a pyramid of much older code solving much older problems. The ideas and concepts of 30 or 40 years ago, far from being obsolete today, have in many cases been embodied in software that you can still find installed on your laptop.
I thought it would be interesting to take a look at one such old program and see how much it had changed since it was first written. cat is maybe the simplest of all the Unix utilities, so I’m going to use it as my example. Ken Thompson wrote the original implementation of cat in 1969. If I were to tell somebody that I have a program on my computer from 1969, would that be accurate? How much has cat really evolved over the decades? How old is the software on our computers?
Thanks to repositories like this one, we can see exactly how cat has evolved since 1969. I’m going to focus on implementations of cat that are ancestors of the implementation I have on my Macbook. You will see, as we trace cat from the first versions of Unix down to the cat in MacOS today, that the program has been rewritten more times than you might expect—but it ultimately works more or less the same way it did fifty years ago.

  • Research Unix

Ken Thompson and Dennis Ritchie began writing Unix on a PDP 7. This was in 1969, before C, so all of the early Unix software was written in PDP 7 assembly. The exact flavor of assembly they used was unique to Unix, since Ken Thompson wrote his own assembler that added some features on top of the assembler provided by DEC, the PDP 7’s manufacturer. Thompson’s changes are all documented in the original Unix Programmer’s Manual under the entry for as, the assembler.
The first implementation of cat is thus in PDP 7 assembly. I’ve added comments that try to explain what each instruction is doing, but the program is still difficult to follow unless you understand some of the extensions Thompson made while writing his assembler. There are two important ones. First, the ; character can be used to separate multiple statements on the same line. It appears that this was used most often to put system call arguments on the same line as the sys instruction. Second, Thompson added support for “temporary labels” using the digits 0 through 9. These are labels that can be reused throughout a program, thus being, according to the Unix Programmer’s Manual, “less taxing both on the imagination of the programmer and on the symbol space of the assembler.” From any given instruction, you can refer to the next or most recent temporary label n using nf and nb respectively. For example, if you have some code in a block labeled 1:, you can jump back to that block from further down by using the instruction jmp 1b. (But you cannot jump forward to that block from above without using jmp 1f instead.)
The most interesting thing about this first version of cat is that it contains two names we should recognize. There is a block of instructions labeled getc and a block of instructions labeled putc, demonstrating that these names are older than the C standard library. The first version of cat actually contained implementations of both functions. The implementations buffered input so that reads and writes were not done a character at a time.
The first version of cat did not last long. Ken Thompson and Dennis Ritchie were able to persuade Bell Labs to buy them a PDP 11 so that they could continue to expand and improve Unix. The PDP 11 had a different instruction set, so cat had to be rewritten. I’ve marked up this second version of cat with comments as well. It uses new assembler mnemonics for the new instruction set and takes advantage of the PDP 11’s various addressing modes. (If you are confused by the parentheses and dollar signs in the source code, those are used to indicate different addressing modes.) But it also leverages the ; character and temporary labels just like the first version of cat, meaning that these features must have been retained when as was adapted for the PDP 11.
The second version of cat is significantly simpler than the first. It is also more “Unix-y” in that it doesn’t just expect a list of filename arguments—it will, when given no arguments, read from stdin, which is what cat still does today. You can also give this version of cat an argument of – to indicate that it should read from stdin.
In 1973, in preparation for the release of the Fourth Edition of Unix, much of Unix was rewritten in C. But cat does not seem to have been rewritten in C until a while after that. The first C implementation of cat only shows up in the Seventh Edition of Unix. This implementation is really fun to look through because it is so simple. Of all the implementations to follow, this one most resembles the idealized cat used as a pedagogic demonstration in K&R C. The heart of the program is the classic two-liner:

while ((c = getc(fi)) != EOF)
putchar(c);

There is of course quite a bit more code than that, but the extra code is mostly there to ensure that you aren’t reading and writing to the same file. The other interesting thing to note is that this implementation of cat only recognized one flag, -u. The -u flag could be used to avoid buffering input and output, which cat would otherwise do in blocks of 512 bytes.

  • BSD

After the Seventh Edition, Unix spawned all sorts of derivatives and offshoots. MacOS is built on top of Darwin, which in turn is derived from the Berkeley Software Distribution (BSD), so BSD is the Unix offshoot we are most interested in. BSD was originally just a collection of useful programs and add-ons for Unix, but it eventually became a complete operating system. BSD seems to have relied on the original cat implementation up until the fourth BSD release, known as 4BSD, when support was added for a whole slew of new flags. The 4BSD implementation of cat is clearly derived from the original implementation, though it adds a new function to implement the behavior triggered by the new flags. The naming conventions already used in the file were adhered to—the fflg variable, used to mark whether input was being read from stdin or a file, was joined by nflg, bflg, vflg, sflg, eflg, and tflg, all there to record whether or not each new flag was supplied in the invocation of the program. These were the last command-line flags added to cat; the man page for cat today lists these flags and no others, at least on Mac OS. 4BSD was released in 1980, so this set of flags is 38 years old.
cat would be entirely rewritten a final time for BSD Net/2, which was, among other things, an attempt to avoid licensing issues by replacing all AT&T Unix-derived code with new code. BSD Net/2 was released in 1991. This final rewrite of cat was done by Kevin Fall, who graduated from Berkeley in 1988 and spent the next year working as a staff member at the Computer Systems Research Group (CSRG). Fall told me that a list of Unix utilities still implemented using AT&T code was put up on a wall at CSRG and staff were told to pick the utilities they wanted to reimplement. Fall picked cat and mknod. The cat implementation bundled with MacOS today is built from a source file that still bears his name at the very top. His version of cat, even though it is a relatively trivial program, is today used by millions.
Fall’s original implementation of cat is much longer than anything we have seen so far. Other than support for a -? help flag, it adds nothing in the way of new functionality. Conceptually, it is very similar to the 4BSD implementation. It is only longer because Fall separates the implementation into a “raw” mode and a “cooked” mode. The “raw” mode is cat classic; it prints a file character for character. The “cooked” mode is cat with all the 4BSD command-line options. The distinction makes sense but it also pads out the implementation so that it seems more complex at first glance than it actually is. There is also a fancy error handling function at the end of the file that further adds to its length.

  • MacOS

The very first release of Mac OS X thus includes an implementation of cat pulled from the NetBSD project. So the first Mac OS X implementation of cat is Kevin Fall’s cat. The only thing that had changed over the intervening decade was that Fall’s error-handling function err() was removed and the err() function made available by err.h was used in its place. err.h is a BSD extension to the C standard library.
The NetBSD implementation of cat was later swapped out for FreeBSD’s implementation of cat. According to Wikipedia, Apple began using FreeBSD instead of NetBSD in Mac OS X 10.3 (Panther). But the Mac OS X implementation of cat, according to Apple’s own open source releases, was not replaced until Mac OS X 10.5 (Leopard) was released in 2007. The FreeBSD implementation that Apple swapped in for the Leopard release is the same implementation on Apple computers today. As of 2018, the implementation has not been updated or changed at all since 2007.
So the Mac OS cat is old. As it happens, it is actually two years older than its 2007 appearance in MacOS X would suggest. This 2005 change, which is visible in FreeBSD’s Github mirror, was the last change made to FreeBSD’s cat before Apple pulled it into Mac OS X. So the Mac OS X cat implementation, which has not been kept in sync with FreeBSD’s cat implementation, is officially 13 years old. There’s a larger debate to be had about how much software can change before it really counts as the same software; in this case, the source file has not changed at all since 2005.
The cat implementation used by Mac OS today is not that different from the implementation that Fall wrote for the 1991 BSD Net/2 release. The biggest difference is that a whole new function was added to provide Unix domain socket support. At some point, a FreeBSD developer also seems to have decided that Fall’s raw_args() function and cook_args() should be combined into a single function called scanfiles(). Otherwise, the heart of the program is still Fall’s code.
I asked Fall how he felt about having written the cat implementation now used by millions of Apple users, either directly or indirectly through some program that relies on cat being present. Fall, who is now a consultant and a co-author of the most recent editions of TCP/IP Illustrated, says that he is surprised when people get such a thrill out of learning about his work on cat. Fall has had a long career in computing and has worked on many high-profile projects, but it seems that many people still get most excited about the six months of work he put into rewriting cat in 1989.

  • The Hundred-Year-Old Program

In the grand scheme of things, computers are not an old invention. We’re used to hundred-year-old photographs or even hundred-year-old camera footage. But computer programs are in a different category—they’re high-tech and new. At least, they are now. As the computing industry matures, will we someday find ourselves using programs that approach the hundred-year-old mark?
Computer hardware will presumably change enough that we won’t be able to take an executable compiled today and run it on hardware a century from now. Perhaps advances in programming language design will also mean that nobody will understand C in the future and cat will have long since been rewritten in another language. (Though C has already been around for fifty years, and it doesn’t look like it is about to be replaced any time soon.) But barring all that, why not just keep using the cat we have forever?
I think the history of cat shows that some ideas in computer science are in fact very durable. Indeed, with cat, both the idea and the program itself are old. It may not be accurate to say that the cat on my computer is from 1969. But I could make a case for saying that the cat on my computer is from 1989, when Fall wrote his implementation of cat. Lots of other software is just as ancient. So maybe we shouldn’t think of computer science and software development primarily as fields that disrupt the status quo and invent new things. Our computer systems are built out of historical artifacts. At some point, we may all spend more time trying to understand and maintain those historical artifacts than we spend writing new code.

##News Roundup
###Trivial Bug in X.Org Gives Root Permission on Linux and BSD Systems

A vulnerability that is trivial to exploit allows privilege escalation to root level on Linux and BSD distributions using X.Org server, the open source implementation of the X Window System that offers the graphical environment.
The flaw is now identified as CVE-2018-14665 (credited to security researcher Narendra Shinde). It has been present in xorg-server for two years, since version 1.19.0 and is exploitable by a limited user as long as the X server runs with elevated permissions.

  • Privilege escalation and arbitrary file overwrite

An advisory on Thursday describes the problem as an “incorrect command-line parameter validation” that also allows an attacker to overwrite arbitrary files.
Privilege escalation can be accomplished via the -modulepath argument by setting an insecure path to modules loaded by the X.org server. Arbitrary file overwrite is possible through the -logfile argument, because of improper verification when parsing the option.

  • Bug could have been avoided in OpenBSD 6.4

OpenBSD, the free and open-source operating system with a strong focus on security, uses xorg. On October 18, the project released version 6.4 of the OS, affected by CVE-2018-14665. This could have been avoided, though.
Theo de Raadt, founder and leader of the OpenBSD project, says that X maintainer knew about the problem since at least October 11. For some reason, the OpenBSD developers received the message one hour before the public announcement this Thursday, a week after their new OS release.
“As yet we don’t have answers about why our X maintainer (on the X security team) and his team provided information to other projects (some who don’t even ship with this new X server) but chose to not give us a heads-up which could have saved all the new 6.4 users a lot of grief,” Raadt says.
Had OpenBSD developers known about the bug before the release, they could have taken steps to mitigate the problem or delay the launch for a week or two.
To remedy the problem, the OpenBSD project provides a source code patch, which requires compiling and rebuilding the X server.
As a temporary solution, users can disable the Xorg binary by running the following command:

chmod u-s /usr/X11R6/bin/Xorg

  • Trivial exploitation

CVE-2018-14665 does not help compromise systems, but it is useful in the following stages of an attack.
Leveraging it after gaining access to a vulnerable machine is fairly easy. Matthew Hickey, co-founder, and head of Hacker House security outfit created and published an exploit, saying that it can be triggered from a remote SSH session.
Three hours after the public announcement of the security gap, Daemon Security CEO Michael Shirk replied with one line that overwrote shadow files on the system. Hickey did one better and fit the entire local privilege escalation exploit in one line.
Apart from OpenBSD, other operating systems affected by the bug include Debian and Ubuntu, Fedora and its downstream distro Red Hat Enterprise Linux along with its community-supported counterpart CentOS.

###OpenBSD on the Desktop: some thoughts

I’ve been using OpenBSD on my ThinkPad X230 for some weeks now, and the experience has been peculiar in some ways.
The OS itself in my opinion is not ready for widespread desktop usage, and the development team is not trying to push it in the throat of anybody who wants a Windows or macOS alternative.
You need to understand a little bit of how *NIX systems work, because you’ll use CLI more than UI.
That’s not necessarily bad, and I’m sure I learned a trick or two that could translate easily to Linux or macOS.
Their development process is purely based on developers that love to contribute and hack around, just because it’s fun.
Even the mailing list is a cool place to hang on!
Code correctness and security are a must, nothing gets committed if it doesn’t get reviewed thoroughly first – nowadays the first two properties should be enforced in every major operating system.
I like the idea of a platform that continually evolves.
pledge(2) and unveil(2) are the proof that with a little effort, you can secure existing software better than ever.
I like the “sensible defaults” approach, having an OS ready to be used – UI included if you selected it during the setup process – is great.
Just install a browser and you’re ready to go.
Manual pages on OpenBSD are real manuals, not an extension of the “–help” command found in most CLI softwares.
They help you understand inner workings of the operating system, no internet connection needed.
There are some trade-offs, too.
Performance is not first-class, mostly because of all the security mitigations and checks done at runtime3.
I write Go code in neovim, and sometimes you can feel a slight slowdown when you’re compiling and editing multiple files at the same time, but usually I can’t notice any meaningful difference.
Browsers are a different matter though, you can definitely feel something differs from the experience you can have on mainstream operating systems.
But again, trade-offs.
To use OpenBSD on the desktop you must be ready to sacrifice some of the goodies of mainstream OSes, but if you’re searching for a zen place to do your computing stuff, it’s the best you can get right now.

###Review: NomadBSD 1.1

One of the most recent additions to the DistroWatch database is NomadBSD. According to the NomadBSD website: “NomadBSD is a 64-bit live system for USB flash drives, based on FreeBSD. Together with automatic hardware detection and setup, it is configured to be used as a desktop system that works out of the box, but can also be used for data recovery.”
The latest release of NomadBSD (or simply “Nomad”, as I will refer to the project in this review) is version 1.1. It is based on FreeBSD 11.2 and is offered in two builds, one for generic personal computers and one for Macbooks. The release announcement mentions version 1.1 offers improved video driver support for Intel and AMD cards. The operating system ships with Octopkg for graphical package management and the system should automatically detect, and work with, VirtualBox environments.
Nomad 1.1 is available as a 2GB download, which we then decompress to produce a 4GB file which can be written to a USB thumb drive. There is no optical media build of Nomad as it is designed to be run entirely from the USB drive, and write data persistently to the drive, rather than simply being installed from the USB media.

  • Initial setup

Booting from the USB drive brings up a series of text-based menus which ask us to configure key parts of the operating system. We are asked to select our time zone, keyboard layout, keyboard model, keyboard mapping and our preferred language. While we can select options from a list, the options tend to be short and cryptic. Rather than “English (US)”, for example, we might be given “en_US”. We are also asked to create a password for the root user account and another one for a regular user which is called “nomad”. We can then select which shell nomad will use. The default is zsh, but there are plenty of other options, including csh and bash. We have the option of encrypting our user’s home directory.
I feel it is important to point out that these settings, and nomad’s home directory, are stored on the USB drive. The options and settings we select will not be saved to our local hard drive and our configuration choices will not affect other operating systems already installed on our computer. At the end, the configuration wizard asks if we want to run the BSDstats service. This option is not explained at all, but it contacts BSDstats to provide some basic statistics on BSD users.
The system then takes a few minutes to apply its changes to the USB drive and automatically reboots the computer. While running the initial setup wizard, I had nearly identical experiences when running Nomad on a physical computer and running the operating system in a VirtualBox virtual machine. However, after the initial setup process was over, I had quite different experiences depending on the environment so I want to divide my experiences into two different sections.

  • Physical desktop computer

At first, Nomad failed to boot on my desktop computer. From the operating system’s boot loader, I enabled Safe Mode which allowed Nomad to boot. At that point, Nomad was able to start up, but would only display a text console. The desktop environment failed to start when running in Safe Mode.
Networking was also disabled by default and I had to enable a network interface and DHCP address assignment to connect to the Internet. Instructions for enabling networking can be found in FreeBSD’s Handbook. Once we are on-line we can use the pkg command line package manager to install and update software. Had the desktop environment worked then the Octopkg graphical package manager would also be available to make browsing and installing software a point-n-click experience.
Had I been able to run the desktop for prolonged amounts of time I could have made use of such pre-installed items as the Firefox web browser, the VLC media player, LibreOffice and Thunderbird. Nomad offers a fairly small collection of desktop applications, but what is there is mostly popular, capable software.
When running the operating system I noted that, with one user logged in, Nomad only runs 15 processes with the default configuration. These processes require less than 100MB of RAM, and the whole system fits comfortably on a 4GB USB drive.

  • Conclusions

Ultimately using Nomad was not a practical option for me. The operating system did not work well with my hardware, or the virtual environment. In the virtual machine, Nomad crashed consistently after just a few minutes of uptime. On the desktop computer, I could not get a desktop environment to run. The command line tools worked well, and the system performed tasks very quickly, but a command line only environment is not well suited to my workflow.
I like the idea of what NomadBSD is offering. There are not many live desktop flavours of FreeBSD, apart from GhostBSD. It was nice to see developers trying to make a FreeBSD-based, plug-and-go operating system that would offer a desktop and persistent storage. I suspect the system would work and perform its stated functions on different hardware, but in my case my experiment was necessarily short lived.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

The post A Thoughtful Episode | BSD Now 273 first appeared on Jupiter Broadcasting.

]]> Goes to 11.2 | BSD Now 252 https://original.jupiterbroadcasting.net/125771/goes-to-11-2-bsd-now-252/ Thu, 28 Jun 2018 08:28:10 +0000 https://original.jupiterbroadcasting.net/?p=125771 ##Headlines ###FreeBSD 11.2-RELEASE Available FreeBSD 11.2 was released today (June 27th) and is ready for download Highlights: OpenSSH has been updated to version 7.5p1. OpenSSL has been updated to version 1.0.2o. The clang, llvm, lldb and compiler-rt utilities have been updated to version 6.0.0. The libarchive(3) library has been updated to version 3.3.2. The libxo(3) […]

The post Goes to 11.2 | BSD Now 252 first appeared on Jupiter Broadcasting.

]]>

##Headlines
###FreeBSD 11.2-RELEASE Available

  • FreeBSD 11.2 was released today (June 27th) and is ready for download
  • Highlights:

OpenSSH has been updated to version 7.5p1.
OpenSSL has been updated to version 1.0.2o.
The clang, llvm, lldb and compiler-rt utilities have been updated to version 6.0.0.
The libarchive(3) library has been updated to version 3.3.2.
The libxo(3) library has been updated to version 0.9.0.
Major Device driver updates to:

  • cxgbe(4) – Chelsio 10/25/40/50/100 gigabit NICs – version 1.16.63.0 supports T4, T5 and T6
  • ixl(4) – Intel 10 and 40 gigabit NICs, updated to version 1.9.9-k
  • ng_pppoe(4) – driver has been updated to add support for user-supplied Host-Uniq tags

New drivers:
+ drm-next-kmod driver supporting integrated Intel graphics with the i915 driver.

  • mlx5io(4) – a new IOCTL interface for Mellanox ConnectX-4 and ConnectX-5 10/20/25/40/50/56/100 gigabit NICs
  • ocs_fc(4) – Emulex Fibre Channel 8/16/32 gigabit Host Adapters
  • smartpqi(4) – HP Gen10 Smart Array Controller Family

The newsyslog(8) utility has been updated to support RFC5424-compliant messages when rotating system logs
The diskinfo(8) utility has been updated to include two new flags, -s which displays the disk identity (usually the serial number), and -p which displays the physical path to the disk in a storage controller.
The top(1) utility has been updated to allow filtering on multiple user names when the -U flag is used
The umount(8) utility has been updated to include a new flag, -N, which is used to forcefully unmount an NFS mounted filesystem.
The ps(1) utility has been updated to display if a process is running with capsicum(4) capability mode, indicated by the flag ‘C’
The service(8) utility has been updated to include a new flag, -j, which is used to interact with services running within a jail(8). The argument to -j can be either the name or numeric jail ID
The mlx5tool(8) utility has been added, which is used to manage Connect-X 4 and Connect-X 5 devices supported by mlx5io(4).
The ifconfig(8) utility has been updated to include a random option, which when used with the ether option, generates a random MAC address for an interface.
The dwatch(1) utility has been introduced
The efibootmgr(8) utility has been added, which is used to manipulate the EFI boot manager.
The etdump(1) utility has been added, which is used to view El Torito boot catalog information.
The linux(4) ABI compatibility layer has been updated to include support for musl consumers.
The fdescfs(5) filesystem has been updated to support Linux®-specific fd(4) /dev/fd and /proc/self/fd behavior
Support for virtio_console(4) has been added to bhyve(4).
The length of GELI passphrases entered when booting a system with encrypted disks is now hidden by default. See the configuration options in geli(8) to restore the previous behavior.

  • In addition to the usual CD/DVD ISO, Memstick, and prebuilt VM images (raw, qcow2, vhd, and vmdk), FreeBSD 11.2 is also available on:
    • Amazon EC2
    • Google Compute Engine
    • Hashicorp/Atlas Vagrant
    • Microsoft Azure
  • In addition to a generic ARM64 image for devices like the Pine64 and Raspberry Pi 3, specific images are provided for:
    • GUMSTIX
    • BANANAPI
    • BEAGLEBONE
    • CUBIEBOARD
    • CUBIEBOARD2
    • CUBOX-HUMMINGBOARD
    • RASPBERRY PI 2
    • PANDABOARD
    • WANDBOARD
  • Full Release Notes

###Setting up an MTA Behind Tor

This article will document how to set up OpenSMTPD behind a fully Tor-ified network. Given that Tor’s DNS resolver code does not support MX record lookups, care must be taken for setting up an MTA behind a fully Tor-ified network. OpenSMTPD was chosen because it was easy to modify to force it to fall back to A/AAAA lookups when MX lookups failed with a DNS result code of NOTIMP (4).

Note that as of 08 May 2018, the OpenSMTPD project is planning a configuration file language change. The proposed change has not landed. Once it does, this article will be updated to reflect both the old language and new.

The reason to use an MTA behing a fully Tor-ified network is to be able to support email behind the .onion TLD. This setup will only allow us to send and receive email to and from the .onion TLD.

  • Requirements:

  • A fully Tor-ified network

  • HardenedBSD as the operating system

  • A server (or VM) running HardenedBSD behind the fully Tor-ified network.

  • /usr/ports is empty

  • Or is already pre-populated with the HardenedBSD Ports tree

  • Why use HardenedBSD? We get all the features of FreeBSD (ZFS, DTrace, bhyve, and jails) with enhanced security through exploit mitigations and system hardening. Tor has a very unique threat landscape and using a hardened ecosystem is crucial to mitigating risks and threats.

Also note that this article reflects how I’ve set up my MTA. I’ve included configuration files verbatim. You will need to replace the text that refers to my .onion domain with yours.

On 08 May 2018, HardenedBSD’s version of OpenSMTPD just gained support for running an MTA behind Tor. The package repositories do not yet contain the patch, so we will compile OpenSMTPD from ports.

  • Steps
  • Installation
  • Generating Cryptographic Key Material
  • Tor Configuration
  • OpenSMTPD Configuration
  • Dovecot Configuration
  • Testing your configuration
  • Optional: Webmail Access

iXsystems
https://www.forbes.com/sites/forbestechcouncil/2018/06/21/strings-attached-knowing-when-and-when-not-to-accept-vc-funding/#30f9f18f46ec
https://www.ixsystems.com/blog/self-2018-recap/

###Running pfSense on a Digital Ocean Droplet

I love pfSense (and opnSense, no discrimination here). I use it for just about anything, from homelab to large scale deployments and I’ll give out on any fancy <enter brand name fw appliance here> for a pfSense setup on a decent hardware.

I also love DigitalOcean, if you ever used them, you know why, if you never did, head over and try, you’ll understand why.
<shameless plug: head over to JupiterBroadcasting.com, the best technology content out there, they have coupon codes to get you started with DO>.

Unfortunately, while DO offers tremendous amount of useful distros and applications, pfSense isn’t one of them. But, where there’s a will, there’s a way, and here’s how to get pfSense up and running on DO so you can have it as the gatekeeper to your kingdom.

Start by creating a FreeBSD droplet, choose your droplet size (for modest setups, I find the 5$ to be quite awesome):

There are many useful things you can do with pfSense on your droplet, from OpenVPN, squid, firewalling, fancy routing, url filtering, dns black listing and much much more.

  • One note though, before we wrap up:

You have two ways to initiate the initial setup wizard of the web-configurator:
Spin up another droplet, log into it and browse your way to the INTERNAL ip address of the internal NIC you’ve set up. This is the long and tedious way, but it’s also somewhat safer as it eliminates the small window of risk the second method poses.
or
Once your WAN address is all setup, your pfSense is ready to accept https connection to start the initial web-configurator setup.
Thing is, there’s a default, well known set of credential to this initial wizard (admin:pfsense), so, there is a slight window of opportunity that someone can swoop in (assuming they know you’ve installed pfsense + your wan IP address + the exact time window between setting up the WAN interface and completing the wizard) and do <enter scary thing here>.

I leave it up to you which of the path you’d like to go, either way, once you’re done with the web-configurator wizard, you’ll have a shiny new pfSense installation at your disposal running on your favorite VPS.

Hopefully this was helpful for someone, I hope to get a similar post soon detailing how to get FreeNAS up and running on DO.
Many thanks to Tubsta and his blogpost as well as to Allan Jude, Kris Moore and Benedict Reuschling for their AWESOME and inspiring podcast, BSD Now.

##News Roundup
###One year of C

It’s now nearly a year that I started writing non-trivial amounts of C code again (the first sokol_gfx.h commit was on the 14-Jul-2017), so I guess it’s time for a little retrospective.

In the beginning it was more of an experiment: I wanted to see how much I would miss some of the more useful C++ features (for instance namespaces, function overloading, ‘simple’ template code for containers, …), and whether it is possible to write non-trivial codebases in C without going mad.

Here are all the github projects I wrote in C:

  • sokol: a slowly growing set of platform-abstraction headers
  • sokol-samples – examples for Sokol
  • chips – 8-bit chip emulators
  • chips-test – tests and examples for the chip- emulators, including some complete home computer emulators (minus sound)

All in all these are around 32k lines of code (not including 3rd party code like flextGL and HandmadeMath). I think I wrote more C code in the recent 10 months than any other language.

So one thing seems to be clear: yes, it’s possible to write a non-trivial amount of C code that does something useful without going mad (and it’s even quite enjoyable I might add).

  • Here’s a few things I learned:

  • Pick the right language for a problem

  • C is a perfect match for WebAssembly

  • C99 is a huge improvement over C89

  • The dangers of pointers and explicit memory management are overrated

  • Less Boilerplate Code

  • Less Language Feature ‘Anxiety’

  • Conclusion

All in all my “C experiment” is a success. For a lot of problems, picking C over C++ may be the better choice since C is a much simpler language (btw, did you notice how there are hardly any books, conferences or discussions about C despite being a fairly popular language? Apart from the neverending bickering about undefined behaviour from the compiler people of course 😉 There simply isn’t much to discuss about a language that can be learned in an afternoon.

I don’t like some of the old POSIX or Linux APIs as much as the next guy (e.g. ioctl(), the socket API or some of the CRT library functions), but that’s an API design problem, not a language problem. It’s possible to build friendly C APIs with a bit of care and thinking, especially when C99’s designated initialization can be used (C++ should really make sure that the full C99 language can be used from inside C++ instead of continuing to wander off into an entirely different direction).

###Configuring OpenBGPD to announce VM’s virtual networks

We use BGP quite heavily at work, and even though I’m not interacting with that directly, it feels like it’s something very useful to learn at least on some basic level. The most effective and fun way of learning technology is finding some practical application, so I decided to see if it could help to improve networking management for my Virtual Machines.

My setup is fairly simple: I have a host that runs bhyve VMs and I have a desktop system from where I ssh to VMs, both hosts run FreeBSD. All VMs are connected to each other through a bridge and have a common network 10.0.1/24. The point of this exercise is to be able to ssh to these VMs from desktop without adding static routes and without adding vmhost’s external interfaces to the VMs bridge.

I’ve installed openbgpd on both hosts and configured it like this:

vmhost: /usr/local/etc/bgpd.conf
AS 65002
router-id 192.168.87.48
fib-update no

network 10.0.1.1/24

neighbor 192.168.87.41 {
    descr "desktop"
    remote-as 65001
}

Here, router-id is set vmhost’s IP address in my home network (192.168.87/24), fib-update no is set to forbid routing table update, which I initially set for testing, but keeping it as vmhost is not supposed to learn new routes from desktop anyway. network announces my VMs network and neighbor describes my desktop box. Now the desktop box:

desktop: /usr/local/etc/bgpd.conf
AS 65001
router-id 192.168.87.41
fib-update yes

neighbor 192.168.87.48 {                                                                                                                                                                                           
        descr "vmhost"                                                                                                                                                                                             
        remote-as 65002                                                                                                                                                                                            
}

It’s pretty similar to vmhost’s bgpd.conf, but no networks are announced here, and fib-update is set to yes because the whole point is to get VM routes added. Both hosts have to have the openbgpd service enabled:

/etc/rc.conf.local
openbgpd_enable="YES"
  • Conclusion

As mentioned already, similar result could be achieved without using BGP by using either static routes or bridging interfaces differently, but the purpose of this exercise is to get some basic hands-on experience with BGP. Right now I’m looking into extending my setup in order to try more complex BGP schema. I’m thinking about adding some software switches in front of my VMs or maybe adding a second VM host (if budget allows). You’re welcome to comment if you have some ideas how to extend this setup for educational purposes in the context of BGP and networking.

As a side note, I really like openbgpd so far. Its configuration file format is clean and simple, documentation is good, error and information messages are clear, and CLI has intuitive syntax.

Digital Ocean

###The Power to Serve

All people within the IT Industry should known where the slogan “The Power To Serve” is exposed every day to millions of people. But maybe too much wishful thinking from me. But without “The Power To Serve” the IT industry today will look totally different. Companies like Apple, Juniper, Cisco and even WatsApp would not exist in their current form.

I provide IT architecture services to make your complex IT landscape manageable and I love to solve complex security and privacy challenges. Complex challenges where people, processes and systems are heavily interrelated. For this knowledge intensive work I often run some IT experiments. When you run experiments nowadays you have a choice:

  • Rent some cloud based services or
  • DIY (Do IT Yourself) on premise

Running your own developments experiments on your own infrastructure can be time consuming. However smart automation saves time and money. And by creating your own CICD pipeline (Continuous Integration, Continuous Deployment) you stay on top of core infrastructure developments. Even hands-on. Knowing how things work from a technical ‘hands-on’ perspective gives great advantages when it comes to solving complex business IT problems. Making a clear distinguish between a business problem or IT problem is useless. Business and IT problems are related. Sometimes causal related, but more often indirect by one or more non linear feedback loops. Almost every business depends of IT systems. Bad IT means often that your customers will leave your business.

One of the things of FeeBSD for me is still FreeBSD Jails. In 2015 I had luck to attend to a presentation of the legendary hacker Poul-Henning Kamp . Check his BSD bio to see what he has done for the FreeBSD community! FreeBSD jails are a light way to visualize your system without enormous overhead. Now that the development on Linux for LXD/LXD is more mature (lxd is the next generation system container manager on linux) there is finally again an alternative for a nice chroot Linux based system again. At least when you do not need the overhead and management complexity that comes with Kubernetes or Docker.

FreeBSD means control and quality for me. When there is an open source package I need, I want to install it from source. It gives me more control and always some extra knowledge on how things work. So no precompiled binaries for me on my BSD systems! If a build on FreeBSD fails most of the time this is an alert regarding the quality for me.

If a complex OSS package is not available at all in the FreeBSD ports collection there should be a reason for it. Is it really that nobody on the world wants to do this dirty maintenance work? Or is there another cause that running this software on FreeBSD is not possible…There are currently 32644 ports available on FreeBSD. So all the major programming language, databases and middleware libraries are present. The FreeBSD organization is a mature organization and since this is one of the largest OSS projects worldwide learning how this community manages to keep innovation and creates and maintains software is a good entrance for learning how complex IT systems function.

FreeBSD is of course BSD licensed. It worked well! There is still a strong community with lots of strong commercial sponsors around the community. Of course: sometimes a GPL license makes more sense. So beside FreeBSD I also love GPL software and the rationale and principles behind it. So my hope is that maybe within the next 25 years the hard battle between BSD vs GPL churches will be more rationalized and normalized. Principles are good, but as all good IT architects know: With good principles alone you never make a good system. So use requirements and not only principles to figure out what OSS license fits your project. There is never one size fits all.

June 19, 1993 was the day the official name for FreeBSD was agreed upon. So this blog is written to celebrate 25th anniversary of FreeBSD.

###Dave’s BSDCan trip report

  • So far, only one person has bothered to send in a BSDCan trip report. Our warmest thanks to Dave for doing his part.

Hello guys! During the last show, you asked for a trip report regarding BSDCan 2018.
This was my first time attending BSDCan. However, BSDCan was my second BSD conference overall, my first being vBSDCon 2017 in Reston, VA.
Arriving early Thursday evening and after checking into the hotel, I headed straight to the Red Lion for the registration, picked up my badge and swag and then headed towards the ‘DMS’ building for the newbies talk. The only thing is, I couldn’t find the DMS building! Fortunately I found a BSDCan veteran who was heading there themselves. My only suggestion is to include the full building name and address on the BSDCan web site, or even a link to Google maps to help out with the navigation. The on-campus street maps didn’t have ‘DMS’ written on them anywhere. But I digress.
Once I made it to the newbies talk hosted by Dan Langille and Michael W Lucas, it highlighted places to meet, an overview of what is happening, details about the ‘BSDCan widow/widower tours’ and most importantly, the 6-2-1 rule!
The following morning, we were present with tea/coffee, muffins and other goodies to help prepare us for the day ahead.
The first talk, “The Tragedy of systemd” covered what systemd did wrong and how the BSD community could improve on the ideas behind it.
With the exception of Michael W Lucas, SSH Key Management and Kirk McKusick, The Evolution of FreeBSD Governance talk, I pretty much attended all of the ZFS talks including the lunchtime BoF session, hosted by Allan Jude. Coming from FreeNAS and being involved in the community, this is where my main interest and motivation lies. Since then I have been able to share some of that information with the FreeNAS community forums and chatroom.
I also attended the “Speculating about Intel” lunchtime BoF session hosted by Theo de Raddt, which proved to be “interesting”.
The talks ended with the wrap up session with a few words from Dan, covering the record attendance and made very clear there “was no cabal”. Followed by the the handing over of Groff the BSD goat to a new owner, thank you’s from the FreeBSD Foundation to various community committers and maintainers, finally ending with the charity auction, where a things like a Canadian $20 bill sold for $40, a signed FreeBSD Foundation shirt originally worn by George Neville-Neil, a lost laptop charger, Michael’s used gelato spoon, various books, the last cookie and more importantly, the second to last cookie!
After the auction, we all headed to the Red Lion for food and drinks, sponsored by iXsystems.
I would like to thank the BSDCan organizers, speakers and sponsors for a great conference. I will certainly hope to attend next year!
Regards,
Dave (aka m0nkey_)

  • Thanks to Dave for sharing his experiences with us and our viewers

##Beastie Bits

Tarsnap

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

The post Goes to 11.2 | BSD Now 252 first appeared on Jupiter Broadcasting.

]]> Puffy in a Box | BSD Now 81 https://original.jupiterbroadcasting.net/79142/puffy-in-a-box-bsd-now-81/ Thu, 19 Mar 2015 09:37:38 +0000 https://original.jupiterbroadcasting.net/?p=79142 We’re back from AsiaBSDCon! This week on the show, we’ll be talking to Lawrence Teo about how Calyptix uses OpenBSD in their line of commercial routers. They’re getting BSD in the hands of Windows admins who don’t even realize it. We also have all this week’s news and answer to your emails, on BSD Now […]

The post Puffy in a Box | BSD Now 81 first appeared on Jupiter Broadcasting.

]]>

post thumbnail

We’re back from AsiaBSDCon! This week on the show, we’ll be talking to Lawrence Teo about how Calyptix uses OpenBSD in their line of commercial routers. They’re getting BSD in the hands of Windows admins who don’t even realize it. We also have all this week’s news and answer to your emails, on BSD Now – the place to B.. SD.

Thanks to:


DigitalOcean


iXsystems

Tarsnap

Direct Download:

Video | HD Video | MP3 Audio | OGG Audio | Torrent | YouTube

RSS Feeds:

MP3 Feed | OGG Feed | iTunes Feed | Video Feed | HD Vid Feed | HD Torrent Feed

– Show Notes: –

Headlines

Using OpenBGPD to distribute pf table updates

  • For those not familiar, OpenBGPD is a daemon for the Border Gateway Protocol – a way for routers on the internet to discover and exchange routes to different addresses
  • This post, inspired by a talk about using BGP to distribute spam lists, details how to use the protocol to distribute some other useful lists and information
  • It begins with “One of the challenges faced when managing our OpenBSD firewalls is the distribution of IPs to pf tables without manually modifying /etc/pf.conf on each of the firewalls every time. This task becomes quite tedious, specifically when you want to distribute different types of changes to different systems (eg administrative IPs to a firewall and spammer IPs to a mail server), or if you need to distribute real time blacklists to a large number of systems.”
  • If you manage a lot of BSD boxes, this might be an interesting alternative to some of the other ways to distribute configuration files
  • OpenBGPD is part of the OpenBSD base system, but there’s also an unofficial port to FreeBSD and a “work in progress” pkgsrc version

Mounting removable media with autofs

  • The FreeBSD foundation has a new article in the “FreeBSD from the trenches” series, this time about the sponsored autofs tool
  • It’s written by one of the autofs developers, and he details his work on creating and using the utility
  • “The purpose of autofs(5) is to mount filesystems on access, in a way that’s transparent to the application. In other words, filesystems get mounted when they are first accessed, and then unmounted after some time passes.”
  • He talks about all the components that need to work together for smooth operation, how to configure it and how to enable it by default for removable drives
  • It ends with a real-world example of something we’re all probably familiar with: plugging in USB drives and watching the magic happen
  • There’s also some more advanced bonus material on GEOM classes and all the more technical details

The Tor Browser on BSD

  • The Tor Project has provided a “browser bundle” for a long time, which is more or less a repackaged Firefox with many security and privacy-related settings preconfigured and some patches applied to the source
  • Just tunneling your browser through a transparent Tor proxy is not safe enough – many things can lead to passive fingerprinting or, even worse, anonymity being completely lost
  • It has, however, only been released for Windows, OS X and Linux – no BSD version
  • “[…] we are pushing back against an emerging monoculture, and this is always a healthy thing. Monocultures are dangerous for many reasons, most importantly to themselves.”
  • Some work has begun to get a working port on BSD going, and this document tells about the process and how it all got started
  • If you’ve got porting skills, or are interested in online privacy, any help would be appreciated of course (see the post for details on getting involved)

OpenSSH 6.8 released

  • Continuing their “tick tock” pattern of releases alternating between new features and bugfixes, the OpenSSH team has released 6.8 – it’s a major upgrade, focused on new features (we like those better of course)
  • Most of the codebase has gone through refactoring, making it easier for regression tests and improving the general readability
  • This release adds support for SHA256-hashed, base64-encoded host key fingerprints, as well as making that the default – a big step up from the previously hex-encoded MD5 fingerprints
  • Experimental host key rotation support also makes it debut, allowing for easy in-place upgrading of old keys to newer (or refreshed) keys
  • You can now require multiple, different public keys to be verified for a user to authenticate (useful if you’re extra paranoid or don’t have 100% confidence in any single key type)
  • The native version will be in OpenBSD 5.7, and the portable version should hit a ports tree near you soon
  • Speaking of the portable version, it now has a configure option to build without OpenSSL or LibreSSL, but doing so limits you to Ed25519 key types and ChaCha20 and AES-CTR ciphers

NetBSD at AsiaBSDCon

  • The NetBSD guys already have a wrap-up of the recent event, complete with all the pictures and weird devices you’d expect
  • It covers their BoF session, the six NetBSD-related presentations and finally their “work in progress” session
  • There was a grand total of 34 different NetBSD gadgets on display at the event

Interview – Lawrence Teo – lteo@openbsd.org / @lteo

OpenBSD at Calyptix

News Roundup

HardenedBSD introduces Integriforce

  • A little bit of background on this one first: NetBSD has something called veriexec, used for checking file integrity at the kernel level
  • By doing it at the kernel level, similar to securelevels, it offers some level of protection even when the root account is compromised
  • HardenedBSD has introduced a similar mechanism into their “secadm” utility
  • You can list binaries in the config file that you want to be protected from changes, then specify whether those can’t be run at all, or if they just print a warning
  • They’re looking for some more extensive testing of this new feature

More s2k15 hackathon reports

  • A couple more Australian hackathon reports have poured in since the last time
  • The first comes from Jonathan Gray, who’s done a lot of graphics-related work in OpenBSD recently
  • He worked on getting some newer “Southern Islands” and “Graphics Core Next” AMD GPUs working, as well as some OpenGL and DRM-related things
  • Also on his todo list was to continue hitting various parts of the tree with American Fuzzy Lop, which ended up fixing a few crashes in mandoc
  • Ted Unangst also sent in a report to detail what he hacked on at the event
  • With a strong focus on improving SMP scalability, he tackled the virtual memory layer
  • His goal was to speed up some syscalls that are used heavily during code compilation, much of which will probably end up in 5.8
  • All the trip reports are much more detailed than our short summaries, so give them a read if you’re interested in all the technicalities

DragonFly 4.0.4 and IPFW3

  • DragonFly BSD has put out a small point release to the 4.x branch, 4.0.4
  • It includes a minor list of fixes, some of which include a HAMMER FS history fix, removing the no-longer-needed “new xorg” and “with kms” variables and a few LAGG fixes
  • There was also a bug in the installer that prevented the rescue image from being installed correctly, which also gets fixed in this version
  • Shortly after it was released, their new IPFW2 firewall was added to the tree and subsequently renamed to IPFW3 (since it’s technically the third revision)

NetBSD gets Raspberry Pi 2 support

  • NetBSD has announced initial support for the second revision of the ever-popular Raspberry Pi board
  • There are -current snapshots available for download, and multiprocessor support is also on the way
  • The NetBSD wiki page about the Raspberry Pi also has some more information and an installation guide
  • The usual Hacker News discussion on the subject
  • If anyone has one of these little boards, let us know – maybe write up a blog post about your experience with BSD on it

OpenIKED as a VPN gateway

  • In our first discussion segment, we talked about a few different ways to tunnel your traffic
  • While we’ve done full tutorials on things like SSH tunnels, OpenVPN and Tor, we haven’t talked a whole lot about OpenBSD’s IPSEC suite
  • This article should help fill that gap – it walks you through the complete IKED setup
  • From creating the public key infrastructure to configuring the firewall to configuring both the VPN server and client, this guide’s got it all

Feedback/Questions

Mailing List Gold

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
  • If you’re in or around the Troy, New York area, our listener Brian is giving a presentation about ports on OpenBSD at the Rensselaer Polytechnic Institute this Friday at 4:00PM
  • If anyone else in the audience is doing something similar or organizing any kind of BSD event, let us know and we’ll be glad to mention it
  • Look forward to seeing the AsiaBSDCon interviews in upcoming episodes

The post Puffy in a Box | BSD Now 81 first appeared on Jupiter Broadcasting.

]]>