Migrating a Decade-Old Blog: From Ubuntu 16.04 to FreeBSD Jails
For many developers, a personal blog or a small project site is a "set it and forget it" endeavor. For one developer, this meant running a blog on Ubuntu 16.04 LTS for over a decade. While the system remained remarkably stable—boasting an uptime of nearly 1,500 days—the lack of security updates for over five years eventually made a migration inevitable.
This migration wasn't just about updating the OS; it was an opportunity to experiment with a different philosophy of system administration. By moving from a DigitalOcean VPS to a Hetzner instance and switching from Linux to FreeBSD, the author transformed a bloated, outdated server into a streamlined, containerized architecture using FreeBSD Jails.
The Motivation for Change
Running an end-of-life (EOL) distribution like Ubuntu 16.04 poses significant security risks. Once a release is out of support, the apt repositories are moved, leaving the system vulnerable to new exploits. While the author's site remained untouched by malicious actors, the risk of "SEO spam" (where bots inject gambling links into posts) is a common fate for neglected legacy servers.
Beyond security, the move was driven by a desire for better hardware value. A transition to Hetzner provided double the CPU and RAM for less than half the monthly cost of the previous DigitalOcean droplet.
Embracing the FreeBSD Philosophy
FreeBSD was chosen not just for stability, but for its integrated design and specific features that offer a more robust server experience than standard Linux distributions.
FreeBSD Jails vs. Docker
One of the standout features of FreeBSD is Jails, a form of OS-level virtualization that predates Docker by decades. While Docker is often used for ephemeral, immutable "packaging" of applications, Jails act more like mini-virtual machines that share the same kernel. They provide a high degree of isolation, allowing each site to run in its own sandbox. If one jail is compromised, the blast radius is limited, and the jail can be destroyed and recreated without affecting the host or other sites.
ZFS: The Gold Standard for Filesystems
Another primary motivator was ZFS. Compared to Linux alternatives like Btrfs, ZFS is widely regarded as more mature. Its ability to handle data integrity and provide near-instant snapshots allows for a backup strategy that is independent of the VPS provider's paid snapshot services.
Architecture and Implementation
The new stack is designed around a reverse-proxy architecture to ensure security and ease of management.
The Networking Layer
To allow Jails to communicate while remaining isolated, a virtual network interface (bastille0) was created using a cloned loopback interface. Traffic is managed via PF (Packet Filter), FreeBSD's powerful firewall. The configuration redirects all incoming HTTP (80) and HTTPS (443) traffic to a specific internal IP (10.0.0.5) where the primary web server resides.
Managing Jails with Bastille
To avoid the complexity of manual jail creation, the author utilized Bastille, a management framework that simplifies the lifecycle of jails. With Bastille, creating a new environment is as simple as:
bastille create caddy 14.3-RELEASE 10.0.0.5 bastille0
The Web Stack: Caddy and Nginx
- Caddy: Used as the primary entry point. Caddy was chosen over Nginx for the edge server because it handles SSL certificate issuance and renewal automatically, eliminating the need for manual
certbotconfigurations. - Nginx: Each individual site (such as the blog or a protest page) runs in its own jail with its own Nginx instance. The Caddy server then reverse-proxies traffic to these internal jail IPs.
Performance Benchmarking
To validate the move, the author conducted extensive load testing using wrk and hey from multiple global locations (London, São Paulo, Silicon Valley, and Tokyo) via Vultr VPS instances.
The Results
The performance difference was stark. Under heavy load (1 million requests with 10k concurrent connections), the old Ubuntu server struggled significantly, completing only about 7% of requests. In contrast, the FreeBSD server handled 94% of the requests successfully.
| Metric | Ubuntu 16.04 (Old) | FreeBSD 14.3 (New) |
|---|---|---|
| Success Rate | ~7% | ~94% |
| Requests/Sec | Lower | 3x to 11x Higher |
| Latency (p90) | Unpredictable | < 3.5s (Global) |
Technical Note: The author noted that the FreeBSD server initially failed under 10k concurrent connections due to the default kern.ipc.somaxconn limit of 128. Increasing this to 16,384 via sysctl resolved the bottleneck.
Critical Perspectives and Takeaways
While the author praised the experience, the community discussion on Hacker News provided important counterpoints:
- Benchmark Validity: Some users argued that the benchmarks were skewed because the new server had 4 CPU cores compared to the old server's one, and that a modern, properly configured Ubuntu installation would likely perform similarly to FreeBSD.
- The "Uptime Trap": One commenter warned against the danger of extreme uptime, noting that they once had a site up for 10 years, only to realize they had no idea how it was configured when the underlying hardware was finally sunset.
- Alternative Hosting: Some pointed out that for static sites generated by Hugo, using a VPS is overkill compared to free alternatives like GitHub Pages or S3+CloudFront.
Final Thoughts
Despite the debates over benchmarks, the migration served as a powerful learning exercise. The transition highlighted the value of documenting infrastructure as code and the benefits of moving away from "zombie" servers. As one user put it, migrating to FreeBSD provides "new eyes into what Linux was, is, and the awesomeness of FreeBSD."