Devops Discoveries

Installing Ubuntu 18.04 on an APU2

The PC Engines APU2 is a great little board to build a router on. We’ve used several of them at work for branch offices, and I’m using one for my home router now. But with no video, these devices can be a bit intimidating to get setup for a new user. This guide will take you step-by-step through updating the firmware and installing Ubuntu 18.04 on one. Before installing any software make sure you assemble the APU2 properly, pay special attention to how you install the CPU cooler.


Getting Started with systemd-nspawnd

I love container technologies. One of my most popular blog posts to date is my guide on FreeBSD Jails the Hard Way. This guide aims to be similar, but for creating containers on linux using systemd-nspawnd. Many people immediately think docker when they think linux container, but docker has a very specific vision for containers. Docker aims to build a collection of microservices where only a single process is running in a container.


VMs Containers and Docker

This post is an explanation of what containers are, how they compare to vms, and where specific container technologies like docker, nspawn, and FreeBSD jails fit. This post is mostly to clarify terminology and help to to bring anyone who is unfamiliar with these concepts up to speed. This explanation is based on my experience over the last several years using these technologies and writing programs to make use of them.


Moving to Hugo

If you notice things look a little different here, it’s because I’ve given up on Jekyll. Ever since github upgraded it I’ve not been happy with how it does syntax highlighting and I’ve been looking for alternatives. Hugo looks promising, and that’s what I’m trying now. Sometime soon I’ll blog about my CI setup once it’s running the way I like.


Salt git integration without gitfs

SaltStack has some pretty cool git integration. Unfortunately it also has quite a few bugs, especially when using gitfs for pillars. These issues can be annoying at small scale, but they can become very important as you add more minions. To work around these I looked for ways I could simplify our salt/git integration and now that it’s complete I couldn’t be happier. With a post-receive hook on my gitlab server and a salt master that is also a minion, the salt server updates it’s file root’s directory from git without the salt-master process having to do any interfacing with git at all.


Random root passwords with saltstack.

Common passwords for devices is a significant security risk, but maintaining unique passwords for every server is nearly impossible without some password manager. But manually generating passwords for hundreds of servers and putting them into a password manager is just not practical. Here is a way to have your salt master generate complex random passwords for each minion and store them in a password manager where you can retrieve them later.


Building your own Session Boarder Controller for SIP communication with Office 365.

If you’re looking to connect an on-premise VOIP phone system to Office 365 you’ll find several documents stating that you must purchase a session border controller, and a list of supported vendors. But if you’re the kind of person who would rather take an unsupported approach than install yet another black box on your network, it can be done. In this guide I’ll share what I’ve learned over the last two weeks in building my own session boarder controller with open source software.


FreeBSD Jails the hard way

There are many great options for managing FreeBSD Jails. iocage, warden and ez-jail aim to streamline the process and make it quick an easy to get going. But sometimes the tools built right into the OS are overlooked. This post goes over what is involved in creating and managing jails using only the tools built into FreeBSD. For this guide, I’m going to be putting my jails in /usr/local/jails.


Changing UPN to Email with Powershell

If you need a quick way to change the UPN of all your users in active directory to match their email address, PowerShell makes it easy. $users = get-aduser -SearchBase "OU=Users,DC=ad,DC=contoso,DC=com" -Filter * -Properties EmailAddress | where {$_.EmailAddress -ne $null -AND $_.EmailAddress.toLower() -ne $_.UserPrincipalName.toLower()} foreach ($user in $users) { $forest = Get-ADForest $email = $user.EmailAddress $username = $email.toLower().Split('@')[0] $userdomain = $email.toLower().Split('@')[1] if (-Not $($forest.UPNSuffixes).Contains($userdomain)) { $forest | Set-ADForest -UPNSuffixes @{Add="$userdomain"} } $user | Set-ADUser -UserPrincipalName "$username@$userdomain" }


Compiling the mongodb plugin for collectd

The MongoDB plugin for collectd is currently unfinished and hasn’t had active development since 2012. Fortunately the folks at Stackdriver have fixed some of the issues so that the plugin works for their stackdriver agent, which is based on collectd. Unfortunately this code has not been submitted back upstream to collectd. This means that if you want to monitor your own mongodb instances with collectd you’ll need to build it yourself.


Creating a Secure Corporate Apt Repository with Salt

There are many reasons an organization could use it’s own internal apt repository. But controlling access to this repository for clients that are outside your internal network can be difficult. But if your repository contains proprietary or confidential packages, securing access is not optional. Thankfully apt supports client authentication with SSL certificates. And with the new x509 module, managing these certificates can be made fully automatic. The x509 module is not yet in the latest release of salt, so you’ll need to manually add it to your custom paths.


X509 Certificates in Salt, implementation details

Saltstack has recently accepted my pull request and integrated the x509 module I’ve spent the last few weeks working on. Most of it’s functionality including a complete PKI example is explained in the documentation, this blog post is to go over some more of the details of how and why I made it for anyone who may not want to just read the source code. I started out with a need to manage X509 certificates for geographically distributed linux based VPN routers.


Embracing Binary for Beautiful Networks

During my time working for an MSP I got to see many different networks with many different techniques for mapping vlan numbers to subnets, but all of them left me dissatisfied. The biggest problem was that they were always designed to try to make some sense to people looking at the numbers as they’re most commonly written. Vlan numbers in decimal and IP addresses in dotted decimal. But these are just incompatible.


Managing Users with Salt

This post has gone through a few iterations. You can see the full history on the github repo. One of the great things about a configuration management solution like Salt is the ability to centrally manage local users. Sure LDAP and Kerberos are great, but sometimes it’s better to keep things simple, that’s what I’m doing with Salt. Leveraging Pillars I can define certain users to be added to servers of a given role.


Automated Ubuntu Installation with PXE, Preseeds and Apt-Cacher-NG (and UEFI compatible)

Doing repetitive installs can be a pain, and figuring out how to make it easier can be even more of a pain since everybody has their own preferred system. Most searching for how to do this for Ubuntu will lead you to Cobbler. Cobbler isn’t a bad tool, but it’s not a good fit for me. It takes away most of the ability to customize pxelinux without learning their templating language.


Using one pair of SSDs for both ZIL and L2ARC in FreeNAS

I’m a big fan of ZFS, and a big fan of FreeNAS. But some times the options avaliable in the FreeNAS GUI can’t quite do everything. Using one disk for more than one purpose is one of those things. At $dayjob we’re going to be using a new FreeNAS server for iSCSI datastores for VMWare. This is one of those instances where a ZIL can really improve performance because there is potential for a lot of synchronious writes from VMs hosting databases.