< the skull

THE NIGHT WE PUT FORGEJO BEHIND THE TAILNET
===========================================
2026-06-06

It started, as all good server adventures do, with a suspiciously high load
average and that sinking feeling that something somewhere was chewing through
CPU like it had been personally wronged.

At first glance, the usual suspects appeared: Nginx workers, database processes,
Docker containers, and a few background services minding their own business. But
one process kept standing out from the crowd: Forgejo, along with several
Git-related commands. The server was not merely busy; it was being worked hard.

The first move was blunt but effective: bring the Forgejo stack down with Docker
Compose. Almost immediately, the load began to fall. That was the smoking gun.
Forgejo was not just involved; Forgejo was the center of gravity.

From there, the investigation split into two questions:

  * Why was Forgejo consuming so much CPU?
  * How could it be locked down without making it useless?

The answer to the second question shaped the whole plan: Forgejo should not be
public. There was only one real user, so leaving a Git service exposed to the
entire internet made no sense. Bots, crawlers, scanners, and opportunistic
traffic were all invited to knock on the door, and some of them were clearly
doing more than knocking.

The goal became simple:

> Keep Forgejo fully usable, but only inside the Tailscale network.

First, the Docker Compose configuration was inspected. The web interface was
already bound safely to localhost, but the SSH Git port was still exposed
publicly. That was a problem. The SSH mapping was changed so it listened only on
the server’s Tailscale IP. After recreating the containers, the public Git SSH
port was gone, and SSH access was limited to the Tailnet.

Then came the Nginx side of the story.

At first, the wrong Nginx site file was edited. The configuration looked right,
but requests still returned `200 OK` from the public internet. That was the
clue: Nginx was not using the file that had just been changed. A full
configuration dump revealed the real active virtual host, generated under a
different filename. Once the correct file was found, the fix was applied
properly:

  allow 100.64.0.0/10;
  deny all;

That single rule changed everything. Public requests to the Forgejo domain began
returning:

  HTTP/2 403

Perfect. The internet was locked out.

But access through Tailscale still worked. A forced test resolving the Forgejo
domain to the server’s Tailscale address returned:

  HTTP/2 200

That proved the architecture was correct:

  Public internet -> blocked
  Tailscale network -> allowed

The CPU confirmed it too. Forgejo dropped from hundreds of percent of CPU usage
to almost nothing. The server began to breathe again.

The last challenge was DNS.

Editing every client’s hosts file would have worked, but it was ugly. Nobody
wants to maintain a pile of manual host overrides across several machines. The
better solution was split DNS through Tailscale.

A small `dnsmasq` service was installed on the server and configured to listen
only on the Tailscale interface. It answered one important question:

  Forgejo domain -> server Tailscale IP

Testing directly against that DNS server worked. Then the Tailscale admin
console was configured to use the server as a custom nameserver for the Forgejo
domain. After refreshing the Tailscale clients, machines inside the Tailnet
could resolve the Forgejo domain internally, without touching local hosts files.

At the end of the adventure, the setup looked like this:

  Forgejo web:
    available only through Tailscale
  
  Forgejo SSH:
    bound only to the server’s Tailscale IP
  
  Public domain:
    blocked by Nginx unless the client comes from the Tailnet
  
  Internal DNS:
    resolves the Forgejo domain to the Tailscale IP
  
  Git clone, pull, and push:
    work normally from Tailnet devices
  
  Random internet bots:
    get a 403 and go bother someone else

The important lesson was simple: exposing a private Git server to the public
internet is usually unnecessary risk. Even if registration is disabled and there
is only one user, bots can still hammer expensive Git endpoints, trigger pack
generation, request archives, crawl diffs, and generally waste CPU.

The clean solution was not to harden the public door forever.

It was to remove the public door.

Forgejo stayed fully functional, but became private to the Tailnet. The server
load dropped, the attack surface shrank, and Git access became exactly what it
should have been from the start: available to trusted machines only.


< the skull