arm64 – Jupiter Broadcasting

Some highlights of the NetBSD 8.0 release are:
USB stack rework, USB3 support added.
In-kernel audio mixer (audio_system(9)).
Reproducible builds (MKREPRO, see mk.conf(5)).
Full userland debug information (MKDEBUG, see mk.conf(5)) available. While most install media do not come with them (for size reasons), the debug and xdebug sets can be downloaded and extracted as needed later. They provide full symbol information for all base system and X binaries and libraries and allow better error reporting and (userland) crash analysis.
PaX MPROTECT (W^X) memory protection enforced by default on some architectures with fine-grained memory protection and suitable ELF formats: i386, amd64, evbarm, landisk.
PaX ASLR (Address Space Layout Randomization) enabled by default on: i386, amd64, evbarm, landisk, sparc64.
Position independent executables by default for userland on: i386, amd64, arm, m68k, mips, sh3, sparc64.
A new socket layer can(4) has been added for communication of devices on a CAN bus.
A special pseudo interface ipsecif(4) for route-based VPNs has been added.
Parts of the network stack have been made MP-safe. The kernel option NET_MPSAFE is required to enable this.
Hardening of the network stack in general.
Various WAPBL (the NetBSD file system “log” option) stability and performance improvements.
Specific to i386 and amd64 CPUs:
Meltdown mitigation: SVS (Separate Virtual Space), enabled by default.
SpectreV2 mitigation: retpoline (support in gcc), used by default for kernels. Other hardware mitigations are also available.
SpectreV4 mitigations available for Intel and AMD.
PopSS workaround: user access to debug registers is turned off by default.
Lazy FPU saving disabled on vulnerable Intel CPUs (“eagerfpu”).
SMAP support.
Improvement and hardening of the memory layout: W^X, fewer writable pages, better consistency, better performance.
(U)EFI bootloader.
Many evbarm kernels now use FDT (flat device tree) information (loadable at boot time from an external file) for device configuration, the number of kernels has decreased but the number of boards has vastly increased.
Lots of updates to 3rd party software included:
GCC 5.5 with support for Address Sanitizer and Undefined Behavior Sanitizer
GDB 7.12
GNU binutils 2.27
Clang/LLVM 3.8.1
OpenSSH 7.6
OpenSSL 1.0.2k
mdocml 1.14.1
acpica 20170303
ntp 4.2.8p11-o
dhcpcd 7.0.6
Lua 5.3.4

###Running FreeBSD on the ARM64 VPS from Scaleway

I’ve been thinking about this 6 since 2017, but only yesterday signed up for an account and played around with the ARM64 offering.
Turns out it’s pretty great! KVM boots into UEFI, there’s a local VirtIO disk attached, no NBD junk required. So we can definitely run FreeBSD.
I managed to “depenguinate” a running instance, the notes are below. Would be great if Scaleway offered an official image instead
For some reason, unlike on x86 4, mounting additional volumes is not allowed 4 on ARM64 instances. So we’ll have to move the running Linux to a ramdisk using pivot_root and then we can do whatever to our one and only disk.
Spin up an instance with Ubuntu Zesty and ssh in.

Prepare the system and change the root to a tmpfs:

apt install gdisk
mount -t tmpfs tmpfs /tmp
cp -r /bin /sbin /etc /dev /root /home /lib /run /usr /var /tmp
mkdir /tmp/proc /tmp/sys /tmp/oldroot
mount /dev/vda /tmp/oldroot
mount --make-rprivate /
pivot_root /tmp /tmp/oldroot
for i in dev proc sys run; do mount --move /oldroot/$i /$i; done
systemctl daemon-reload
systemctl restart sshd

Now reconnect to ssh from a second terminal (note: rm the connection file if you use ControlPersist in ssh config), then exit the old session. Kill the old sshd process, restart or stop the rest of the stuff using the old disk:

pkill -f notty
sed -ibak 's/RefuseManualStart.*$//g' /lib/systemd/system/dbus.service
systemctl daemon-reload
systemctl restart dbus
systemctl daemon-reexec
systemctl stop user@0 ntp cron systemd-logind
systemctl restart systemd-journald systemd-udevd
pkill agetty
pkill rsyslogd

Check that nothing is touching /oldroot:

lsof | grep oldroot

There will probably be an old dbus-daemon, kill it.
And finally, unmount the old root and overwrite the hard disk with a memstick image:

umount -R /oldroot
wget https://download.freebsd.org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/12.0/FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz
xzcat FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz | dd if=/dev/stdin of=/dev/vda bs=1M

(Look for the newest snapshot, don’t copy paste the July 19 link above if you’re reading this in the future. Actually maybe use a release instead of CURRENT…)
Now, fix the GPT: move the secondary table to the end of the disk and resize the table.
It’s important to resize here, as FreeBSD does not do that and silently creates partitions that won’t persist across reboots

gdisk /dev/vda
x
e
s
4
w
y

And reboot. (You might actually want to hard reboot here: for some reason on the first reboot from Linux, pressing the any-key to enter the prompt in the loader hangs the console for me.)

I didn’t have to go into the ESC menu and choose the local disk in the boot manager, it seems to boot from disk automatically.

Now we’re in the FreeBSD EFI loader.
For some reason, the (recently fixed? 2) serial autodetection from EFI is not working correctly. Or something.
So you don’t get console output by default.
To fix, you have to run these commands in the boot loader command prompt:

set console=comconsole,efi
boot

Ignore the warning about comconsole not being a valid console.
Since there’s at least one (efi) that the loader thinks is valid, it sets the whole variable.)

(UPD: shouldn’t be necessary in the next snapshot)

Now it’s a regular installation process!
When asked about partitioning, choose Shell, and manually add a partition and set up a root filesystem:

gpart add -t freebsd-zfs -a 4k -l zroot vtbd0
zpool create -R /mnt -O mountpoint=none -O atime=off zroot /dev/gpt/zroot
zfs create -o canmount=off -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=/ zroot/ROOT/default
zfs create -o mountpoint=/usr zroot/ROOT/default/usr
zfs create -o mountpoint=/var zroot/ROOT/default/var
zfs create -o mountpoint=/var/log zroot/ROOT/default/var/log
zfs create -o mountpoint=/usr/home zroot/home
zpool set bootfs=zroot/ROOT/default zroot
exit

(In this example, I set up ZFS with a beadm-compatible layout which allows me to use Boot Environments.)

In the post-install chroot shell, fix some configs like so:

echo 'zfs_load="YES"' >> /boot/loader.conf
echo 'console="comconsole,efi"' >> /boot/loader.conf
echo 'vfs.zfs.arc_max="512M"' >> /boot/loader.conf
sysrc zfs_enable=YES
exit

(Yeah, for some reason, the loader does not load zfs.ko’s dependency opensolaris.ko automatically here. idk what even. It does on my desktop and laptop.)

Now you can reboot into the installed system!!

Here’s how you can set up IPv6 (and root’s ssh key) auto configuration on boot:

Pkg bootstrap
pkg install curl
curl https://raw.githubusercontent.com/scaleway/image-tools/master/bases/overlay-common/usr/local/bin/scw-metadata > /usr/local/bin/scw-metadata
chmod +x /usr/local/bin/scw-metadata
echo '#\!/bin/sh' > /etc/rc.local
echo 'PATH=/usr/local/bin:$PATH' >> /etc/rc.local
echo 'eval $(scw-metadata)' >> /etc/rc.local
echo 'echo $SSH_PUBLIC_KEYS_0_KEY > /root/.ssh/authorized_keys' >> /etc/rc.local
echo 'chmod 0400 /root/.ssh/authorized_keys' >> /etc/rc.local
echo 'ifconfig vtnet0 inet6 $IPV6_ADDRESS/$IPV6_NETMASK' >> /etc/rc.local
echo 'route -6 add default $IPV6_GATEWAY' >> /etc/rc.local
mkdir /run
mkdir /root/.ssh
sh /etc/rc.local

And to fix incoming TCP connections, configure the DHCP client to change the broadcast address:

echo 'interface "vtnet0" { supersede broadcast-address 255.255.255.255; }' >> /etc/dhclient.conf
killall dhclient
dhclient vtnet0

Other random notes:
keep in mind that -CURRENT snapshots come with a debugging kernel by default, which limits syscall performance by a lot, you might want to build your own 2 with config GENERIC-NODEBUG
also disable heavy malloc debugging features by running ln -s ‘abort:false,junk:false’ /etc/malloc.conf (yes that’s storing config in a symlink)
you can reuse the installer’s partition for swap

** Digital Ocean **
https://do.co/bsdnow

###Easy encrypted backups on OpenBSD with base tools

Today’s topic is “Encrypted backups” using only OpenBSD base tools. I am planning to write a bigger article later about backups but it’s a wide topic with a lot of software to cover and a lot of explanations about the differents uses cases, needs, issues an solutions. Here I will stick on explaining how to make reliable backups for an OpenBSD system (my laptop).
What we need is the dump command (see man 8 dump for its man page). It’s an utility to make a backup for a filesystem, it can only make a backup of one filesystem at a time. On my laptop I only backup /home partition so this solution is suitable for me while still being easy.
Dump can do incremental backups, it means that it will only save what changed since the last backup of lower level. If you do not understand this, please refer to the dump man page.
What is very interesting with dump is that it honors nodump flag which is an extended attribute of a FFS filesystem. One can use the command chflags nodump /home/solene/Downloads to tells dump not do save that folder (under some circumstances). By default, dump will not save thoses files, EXCEPT for a level 0 backup.

Important features of this backup solution:
save files with attributes, permissions and flags
can recreate a partition from a dump, restore files interactively, from a list or from its inode number (useful when you have files in lost+found)
one dump = one file

My process is to make a huge dump of level 0 and keep it on a remote server, then, once a week I make a level 1 backup which will contain everything changed since the last dump of level 0, and everyday I do a level 2 backup of my files. The level 2 will contain latest files and the files changing a lot, which are often the most interesting. The level 1 backup is important because it will offload a lot of changes for the level 2.
Let me explain: let says my full backup is 60 GB, full of pictures, sources files, GUI applications data files etc… A level 1 backup will contain every new picture, new projects, new GUI files etc… since the full backup, which will produce bigger and bigger dump over time, usually it is only 100 MB to 1GB. As I don’t add new pictures everyday or use new software everyday, the level 2 will take care of most littles changes to my data, like source code edited, little works on files etc… The level 2 backup is really small, I try to keep it under 50 MB so I can easily send it on my remote server everyday.
One could you more dump level, up to level 9, but keep in mind that those are incremental. In my case, if I need to restore all my partition, I will need to use level 0, 1 and 2 to get up to latest backup state. If you want to restore a file deleted a few days ago, you need to remember in which level its latest version is.
History note: dump was designed to be used with magnetic tapes.

See the article for the remainder of the article

##News Roundup
###Status of DFly server storage upgrades (Matt Dillon)

Last month we did some storage upgrades, particularly of internet-facing machines for package and OS distribution. Yesterday we did a number of additional upgrades, described below. All using funds generously donated by everyone!

The main repository server received a 2TB SSD to replace the HDDs it was using before. This will improve access to a number of things maintained by this server, including the mail archives, and gives the main repo server more breathing room for repository expansion. Space was at a premium before. Now there’s plenty.

Monster, the quad socket opteron which we currently use as the database builder and repository that we export to our public grok service (grok.dragonflybsd.org) received a 512G SSD to add swap space for swapcache, to help cache the grok meta-data. It now has 600GB of swapcache configured. Over the next few weeks we will also be changing the grok updates to ping-pong between the two 4TB data drives it received in the last upgrade so we can do concurrent updates and web accesses without them tripping over each other performance-wise.

The main developer box, Leaf, received a 2TB SSD and we are currently in the midst of migrating all the developer accounts in /home and /build from its old HDDs to its new SSD. This machine serves developer repos, developer web stuff, our home page and wiki, etc, so those will become snappier as well.

Hard drives are becoming real dinosaurs. We still have a few left from the old days but in terms of active use the only HDDs we feel we really need to keep now are the ones we use for backups and grok data, owing to the amount of storage needed for those functions.

Five years ago when we received the blade server that now sits in the colo, we had a small 256G SSD for root on every blade, and everything else used HDDs. To make things operate smoothly, most of that 256G root SSD was assigned to swapcache (200G of it, in fact, in most cases). Even just 2 years ago replacing all those HDDs with SSDs, even just the ones being used to actively serve data and support developers, would have been cost prohibitive. But today it isn’t and the only HDDs we really need anywhere are for backups or certain very large bits of bulk data (aka the grok source repository and index). The way things are going, even the backup drives will probably become SSDs over the next two years.

###iX ad spot
OSCON 2018 Recap

###zpool checkpoints

In March, to FreeBSD landed a very interesting feature called ‘zpool checkpoints’. Before we jump straight into the topic, let’s take a step back and look at another ZFS feature called ‘snapshot’. Snapshot allows us to create an image of our single file systems. This gives us the option to modify data on the dataset without the fear of losing some data.

A very good example of how to use ZFS snapshot is during an upgrade of database schema. Let us consider a situation where we have a few scripts which change our schema. Sometimes we are unable to upgrade in one transaction (for example, when we attempt to alter a table and then update it in single transaction). If our database is on dataset, we can just snapshot it, and if something goes wrong, simply rollback the file system to its previous state.

The problem with snapshot is that it works only on a single dataset. If we added some dataset, we wouldn’t then be able to create the snapshot which would rollback that operation. The same with changing the attributes of a dataset. If we change the compression on the dataset, we cannot rollback it. We would need to change that manually.

Another interesting problem involves upgrading the whole operating system when we upgrade system with a new ZFS version. What if we start upgrading our dataset and our kernel begins to crash? (If you use FreeBSD, I doubt you will ever have had that experience but still…). If we rollback to the old kernel, there is a chance the dataset will stop working because the new kernel doesn’t know how to use the new features.

Zpool checkpoints is the solution to all those problems. Instead of taking a single snapshot of the dataset, we can now take a snapshot of the whole pool. That means we will not only rollback the data but also all the metadata. If we rewind to the checkpoint, all our ZFS properties will be rolled back; the upgrade will be rolledback, and even the creation/deletion of the dataset, and the snapshot, will be rolledback.

Zpool Checkpoint has introduced a few simple functions:
For a creating checkpoint:

zpool checkpoint

Rollbacks state to checkpoint and remove the checkpoint:

zpool import -- rewind-to-checkpoint

Mount the pool read only – this does not rollback the data:

zpool import --read-only=on --rewind-to-checkpoint

Remove the checkpoint

zpool checkpoint --discard or zpool checkpoint -d

With this powerful feature we need to remember some safety rules:
Scrub will work only on data that isn’t in checkpool.
You can’t remove vdev if you have a checkpoint.
You can’t split mirror.
Reguid will not work either.
Create a checkpoint when one of the disks is removed…

For me, this feature is incredibly useful, especially when upgrading an operating system, or when I need to experiment with additional data sets. If you speak Polish, I have some additional information for you. During the first Polish BSD user group meeting, I had the opportunity to give a short talk about this feature. Here you find the video of that talk, and here is the slideshow.

I would like to offer my thanks to Serapheim Dimitropoulos for developing this feature, and for being so kind in sharing with me so many of its intricacies. If you are interested in knowing more about the technical details of this feature, you should check out Serapheim’s blog, and his video about checkpoints.

###g2k18 Reports

##Beastie Bits

Tarsnap

##Feedback/Questions

Dale – L2ARC recommendations & drive age question
Todd – ZFS & S3
efraim – License Poem
Henrick – Yet another ZFS question

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

The post Great NetBSD 8 | BSD Now 257 first appeared on Jupiter Broadcasting.

Linux Action News 36

chris — Sun, 14 Jan 2018 17:09:59 +0000

RSS Feeds:

HD Video Feed | MP3 Feed | iTunes Feed

Become a supporter on Patreon:

Episode Links

Barcelona to switch to FOSS — According to the news report, the city plans to first replace all its user applications with alternative open source applications. This will go on until the only remaining proprietary software will be Windows where it will finally be replaced with a Linux distribution.
Nextcloud Talk — We’re very proud to announce today Nextcloud Talk, the first enterprise-ready, self-hosted communication technology giving users the highest degree of control over their data and communication.
AMP changes aren’t good enough for some web devs — The Web is not Google, and should not be just Google.
Ubuntu 17.10 ISOs available again — The Ubuntu 17.10.1 ISO re-spin is for disabling the SPI kernel driver to avoid messing up select laptops.
lkml.org is hosted on a home connection — The site’s backend is hosted on a machine at home which is waiting for someone to enter a luks passphrase after a power outage while on vacation.
Bad news for the fans of lkml.org — Bad news for the fans of https://lkml.org : the main board of the server somehow did not survive the outage
Expect prolonged downtime while I source replacement parts.
Fedora might make AArch64 a primary architecture — Fedora developers are looking to promote their AArch64 / ARM64 / ARMv8 server offerings to being a “primary architecture” for this next Fedora release.

The post Linux Action News 36 first appeared on Jupiter Broadcasting.

Our Code is Your Code | BSD Now 98

chris — Thu, 16 Jul 2015 09:19:26 +0000

To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video

Coming up this time on the show, we’ll be talking with the CTO of Xinuos, David Meyer, about their adoption of FreeBSD. We also discuss the BSD license model for businesses & the benefits of contributing changes back.

Thanks to:

Get Paid to Write for DigitalOcean

Direct Download:

RSS Feeds:

– Show Notes: –

Headlines

Enabling FreeBSD on AArch64

One of the things the FreeBSD foundation has been dumping money into lately is ARM64 support, but we haven’t heard too much about it – this article should change that
Since it’s on a mainstream ARM site, the article begins with a bit of FreeBSD history, leading up to the current work on ARM64
There’s also a summary of some of the ARM work done at this year’s BSDCan, including details about running it on the Cavium ThunderX platform (which has 48 cores)
As of just a couple months ago, dtrace is even working on this new architecture
Come 11.0-RELEASE, the plan is for ARM64 to get the same “tier 1” treatment as X86, which would imply binary updates for base and ports – something Raspberry Pi users often complain about not having

OpenBSD’s tcpdump detailed

Most people are probably familiar with tcpdump, a very useful packet sniffing and capturing utility that’s included in all the main BSD base systems
This video guide is specifically about the version in OpenBSD, which has gone through some major changes (it’s pretty much a fork with no version number anymore)
Unlike on the other platforms, OpenBSD’s tcpdump will always run in a chroot as an unprivileged user – this has saved it from a number of high-profile exploits
It also has support for the “pf.os” system, allowing you to filter out operating system fingerprints in the packet captures
There’s also PF (and pflog) integration, letting you see which line in your ruleset triggered a specific match
Being able to run tcpdump directly on your router is pretty awesome for troubleshooting

More FreeBSD foundation at BSDCan

The FreeBSD foundation has another round of trip reports from this year’s BSDCan
First up is Kamil Czekirda, who gives a good summary of some of the devsummit, FreeBSD-related presentations, some tutorials, getting freebsd-update bugs fixed and of course eating cake
A second post from Christian Brueffer, who cleverly planned ahead to avoid jetlag, details how he got some things done during the FreeBSD devsummit
Their third report is from our buddy Warren Block, who (unsurprisingly) worked on a lot of documentation-related things, including getting more people involved with writing them
In true doc team style, his report is the most well-written of the bunch, including lots of links and a clear separation of topics (doc lounge, contributing to the wiki, presentations…)
Finally, the fourth one comes to us from Shonali Balakrishna, who also gives an outline of some of the talks
“Not only does a BSD conference have way too many very smart people in one room, but also some of the nicest.”

DragonFly on the Chromebook C720

If you’ve got one of the Chromebook laptops and weren’t happy with the OS it came with, DragonFlyBSD might be worth a go
This article is a “mini-report” on how DragonFly functions on the device as a desktop, and
While the 2GB of RAM proved to be a bit limiting, most of the hardware is well-supported
DragonFly’s wiki has a full guide on getting set up on one of these devices as well

Interview – David Meyer – info@xinuos.com / @xinuos

Xinuos, BSD license model vs. others, community interaction

News Roundup

Introducing LiteBSD

We definitely don’t talk about 4.4BSD a lot on the show
LiteBSD is “a variant of [the] 4.4BSD operating system adapted for microcontrollers”
If you’ve got really, really old hardware (or are working in the embedded space) then this might be an interesting hobby project to look info

HardenedBSD announces ASLR completion

HardenedBSD, now officially a full-on fork of FreeBSD, has declared their ASLR patchset to be complete
The latest and last addition to the work was VDSO (Virtual Dynamic Shared Object) randomization, which is now configurable with a sysctl
This post gives a summary of the six main features they’ve added since the beginning
Only a few small things are left to do – man page cleanups, possibly shared object load order improvements

Unlock the reaper

In the ongoing quest to make more of OpenBSD SMP-friendly, a new patch was posted that unlocks the reaper in the kernel
When there’s a zombie process causing a resource leak, it’s the reaper’s job to deallocate their resources (and yes we’re still talking about computers, not horror movies)
Initial testing has yielded positive results and no regressions
They’re looking for testers, so you can install a -current snapshot and get it automatically
An updated version of the patch is coming soon too
A hackathon is going on right now, so you can expect more SMP improvements in the near future

The importance of mentoring

Adrian Chadd has a blog post up about mentoring new users, and it tells the story of how he originally got into FreeBSD
He tells the story of, at age 11, meeting someone else who knew about making crystal sets that became his role model
Eventually we get to his first FreeBSD 1.1 installation (which he temporarily abandoned for Linux, since it didn’t have a color “ls” command) and how he started using the OS
Nowadays, there’s a formal mentoring system in FreeBSD
While he talks about FreeBSD in the post, a lot of the concepts apply to all the BSDs (or even just life in general)

Feedback/Questions

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
If you’re in or around the Calgary, Alberta area in Canada, there’s an OpenBSD developer speaking event at the Southern Alberta Institute of Technology on July 20th
It’s right after the hackathon, and they’ll be discussing some of the work that was done (maybe with a Q&A session)
We’re looking for some new interviews – get in touch if you’re doing anything cool with BSD that you’d like to talk about (or want to suggest someone else)

The post Our Code is Your Code | BSD Now 98 first appeared on Jupiter Broadcasting.