I Like My Web Servers Basic AF

I’ve spent a surprisingly large portion of my career thinking about web servers. My first real exposure was Apache, back when I was getting into Slackware.

This was the era of page counters, guestbooks, and cgi-bin directories. The web was still weird enough that putting something online felt a little like discovering a secret passage. Apache was everywhere, and for a while, that was enough.

In the early 2000s, I got deeper into Microsoft technology and started working with IIS. IIS 4 was the first version I used and IIS 7 is probably the last one where I’d claim significant hands-on time. I was writing a lot of ASP and eventually ASP.net, so IIS became the natural choice.

Apache never really disappeared. Sometimes I was actively managing it. Other times it was just part of whatever environment I happened to be working in.

Once I moved into publishing, the web server suddenly had a much bigger job. NGINX entered the picture and so did Varnish. The difference between “this website works” and “this website continues working when several million people suddenly want to look at it” is substantial.

I maintained sites that were busy all the time and then became really busy when major entertainment events happened. The Oscars were particularly good at turning theoretical performance conversations into very practical ones. Proper caching, connection handling, and configurations were the difference between turning off the TV for some quiet and an editorial director throwing a fit over slower than normal response times.

The difference between a page being served quickly and a page being served slowly mattered a lot when thousands of people were asking for it at the same time and varnish was great for that. Varnish also gave me the opportunity to spend entirely too much time doing things like adding pointless headers to responses because, technically, I could.

That’s the thing about infrastructure - once you have a powerful tool, you start finding things for it to do. Sometimes that’s engineering, and sometimes it’s just doing some dumb shit because you can.

Complexity is the Answer - Sometimes

I don’t want to turn this into an “old technology good, new technology bad” argument; I’ve been around too long for that. Complexity is sometimes absolutely necessary.

If you’re serving dynamic applications, you may need an application runtime. If you’re dealing with large amounts of data, you may require databases and caching. Operating at scale often means load balancers, reverse proxies, CDNs, queues, microservices, observability platforms, and a collection of other things that would have sounded insane to the person I was when I first opened a Slackware installation. That’s progress.

Sometimes the problem genuinely is complicated. The mistake is assuming that the infrastructure should remain complicated after the problem becomes simple again.

When a Website Became a Folder

At some point I started moving toward static websites where possible. It’s not like I suddenly discovered the security benefits of static content - I just wanted a change of pace.

Hugo made it possible to take content and turn it into a collection of static files - and changed the nature of the problem for me. There wasn’t an application serving the site anymore, replaced by a build process that created the it instead. Once that build finished, I had HTML, CSS, JavaScript, images, feeds, and everything else a browser needed - and the website effectively become a directory.

That raised a question that took me a while to appreciate: why was I still thinking about the web server as an application platform? If the job is to return a file, maybe the web server should just return the file - and that’s where Lighttpd came into my Homelab.

The Temptation to Build More

One of the easiest things to do in technology is to add another layer. Need a website? Install a web server. Have something that needs to be dynamic? Go ahead and add a runtime in. Persistence? Well, let’s get a database in here somewhere. Performance issues? Add caching. Need authentication, observability, and a splash of routing? Let’s just pile it on!

Need to manage it all? Well, now you require another system to manage the systems you just added.

None of those decisions are inherently wrong, and that’s what can make this particularly dangerous. Every individual component can be perfectly reasonable. The problem is what happens when the accumulated infrastructure becomes more complicated than the thing you’re actually trying to accomplish.

It’s a bit like this person/people line from Men in Black:

A static website is a particularly good example because it gives us a relatively clean baseline. If the website consists of static files, I don’t need an application server to generate those files.

I don’t need to install something like PHP because I’m not executing any PHP code, nor do I need a database since the site is built before it even touches the server. My caching layer has disappeared because I have a relatively small collection of files that can be served directly.

Most importantly, I don’t need the web server to do anything particularly clever - I just need it to serve the files. That’s a service that’s doing one thing - and doing it well. It’s the “person vs. people” line from the MIB clip.

KISS Gets More Valuable as Technology Improves

The second CIO I answered to loved citing KISS, but he also installed KaZaa on a proxy server to download Limp Bizkit albums, so I never took much of what he said seriously. KISS was the one exception when it came to his advice, and that was a lifetime ago.

Technology has made it easier than ever to build complicated systems. Containers make it trivial to deploy another service, infrastructure-as-code makes it easier to spin up complicated environments, and Cloud platforms provide a wide array of accessible building blocks. Modern systems can be assembled from dozens or hundreds of independently managed components - and all of that can start with a simple prompt now.

All of that is incredibly useful. It also makes complexity cheap to create - and when complexity is cheap to create, simplicity becomes something you have to actively choose. That’s where I’ve come to appreciate KISS - Keep It Simple, Stupid - more as I’ve gotten older.

I don’t interpret KISS as “always use the simplest technology.” That’s nonsense; I interpret it as “don’t introduce complexity unless the problem requires it.” There’s a subtle but important difference - complexity should have to justify its existence. I’ll happily maintain something if I need it - but if I don’t need it, I’m not going to want to maintain it.

Security Is Part of the Argument

There’s an obvious security argument here, too. Every service is another thing to patch, and components have configurations to maintain that can become opportunities for mistakes. Processes have permissions and each network listener creates another potential path into the system.

Reducing unnecessary components therefore reduces some amount of risk, but I don’t think security is the most interesting reason to simplify. It’s just one of the reasons hovering around the larger reason - complexity consumes attention, and that’s a finite resource.

I Don’t Want to Maintain Infrastructure for Its Own Sake

This is probably the part of Homelabbing that has changed most for me over the past few years. When I was younger, there was a certain amount of joy in making things work simply because making things work was the point. I’d happily spend an evening figuring out why some ridiculous combination of software wouldn’t cooperate, and sometimes I still do.

Increasingly though, I want the infrastructure to get out of my way. There are more things I want to build and write, and plenty projects that are completely unrelated to technology. I don’t want to spend my time maintaining infrastructure that exists solely to support infrastructure that exists solely to serve a static website.

There’s a difference between working on infrastructure and maintaining infrastructure. The first can be fun. The second is often just rent. If I can eliminate some of that maintenance without sacrificing anything I actually need, I’m going to.

This is why I like the Lighttpd setup in my Homelab. It isn’t particularly impressive and that’s the point. Hugo builds the site and Lighttpd serves the resulting files. The surrounding infrastructure handles the things that should be handled elsewhere. The web server doesn’t need to understand the content, communicate with a database or execute application code. It just needs to answer HTTP requests - and when it does that reliably, I can stop thinking about it.

I don’t want to wake up one morning and think, “I wonder how my web server is feeling today.” I want to realize that I haven’t thought about my web server’s capabilities for a while because it has been quietly doing exactly what I told it to do. That’s what good infrastructure should feel like.

Defense in Depth Without Making It a Religion

There is another lesson here that I think is worth mentioning. Simplifying one layer doesn’t mean pretending the rest of the environment doesn’t matter.

I don’t expect Lighttpd to be my security boundary. The service has limited permissions, the content it serves is static, and the surrounding network determines how the service can be reached. TLS and access controls are handled according to the needs of the environment. Those are separate decisions that are kept separate on purpose.

That’s important because “simple” doesn’t have to mean “wide open.” In fact, simple systems can be easier to secure precisely because their responsibilities are easier to understand. If a web server only needs to read a directory and return files, it’s pretty easy to reason about what it should be allowed to do.

If that same server is also building applications, modifying content, talking to databases, executing scripts, handling authentication, proxying other applications, and doing seventeen other things, the security conversation becomes considerably more interesting - and not necessarily in a good way.

I’m not trying to convince you to use Lighttpd.

If NGINX solves your problem, use NGINX. If Apache solves your problem, use Apache. Looking to develop a twitch in an eye over time, go find a reason to work with Tomcat and Struts. If you need something else entirely, use that instead. The point is that the web server should match the problem.

I’ve used more complicated web infrastructure because I’ve had complicated problems, and now I have a relatively simple problem, so I use relatively simple infrastructure. It feels like a downgrade at first, but it’s an appropriate response to a change in requirements.

The Infrastructure Should Serve the Project

I think this is the part that took me the longest to learn. It’s easy to become attached to the infrastructure itself - especially in a Homelab. You can spend an entire Saturday building some spectacular system and then realize you haven’t actually used it for anything yet. I’ve done this. More than once. In the past month.

There’s nothing wrong with learning for the sake of learning. That’s one of the best reasons to have a Homelab - but eventually, I want the infrastructure to serve a purpose beyond its own existence.

My web server exists so I can publish websites for internal testing and review, and storage exists so I can store things. My network exists so my systems can communicate, and my automations exist, so I don’t have to do everything manually. The infrastructure exists to support the work - but it’s far from the only work to be done.

There Is a Cost to Every Moving Part

We tend to talk about infrastructure costs in terms of CPU, RAM, disk space, licensing, or Cloud bills.

Those are real costs, but there’s one that never shows up on a dashboard - attention. Every running service asks something of me - I need to understand it, updated it and know what to do when it breaks. I need to know what depends on it, why I installed it, and whether I still need it.

Eventually, enough little things become a significant amount of cognitive overhead and the cost I’m increasingly interested in eliminating, because I’d rather spend that attention somewhere useful. It’s also much easier to be a bit lazy now and then - when everything is working as expected.

A Web Server That I Don’t Think About

That’s ultimately what brought me to Lighttpd. After all the years of Apache, IIS, NGINX, Varnish, application servers, caching layers, and the assorted weirdness that comes with running websites at scale, I’ve ended up with a web server whose primary job is to return files.

It feels almost anticlimactic and I like that. I’ve learned enough about web infrastructure that I don’t need every Homelab project to demonstrate that I know how complicated infrastructure can become. Sometimes the better demonstration is showing that I knew when not to make it complicated.

If you want to see exactly how I built the Lighttpd + Hugo setup I’m talking about here, I’ve documented the practical details separately: Lighttpd + Certbot + Rsync for Static Sites

That’s the actual configuration, commands, permissions, and deployment process I use. This post is the “why,” while that one is the “how” - and what the server running this site will be moved to in the near future.

If you follow along and have a web server quietly disappears into the background after you’ve finished setting it up, I’d say it did its job.