Cerberus — port management

Overview

Cerberus is the hub-managed UFW firewall for every Linux node — it replaces SSH-and-ufw allow PORT with one-click presets, drift detection, and audit-logged mutations. Every rule add, edit, delete, preset-apply, bootstrap, and reconcile lands in admin_audit with the actor email, the node id, the literal UFW command issued, and the result.

What is UFW?

UFW (Uncomplicated Firewall) is the standard Ubuntu front-end for the kernel’s netfilter / iptables rule engine. It exposes a small command vocabulary that the hub drives over SSH:

• ufw allow 22/tcp — permit TCP traffic to port 22 (SSH).
• ufw allow 53/udp — permit UDP traffic to port 53 (DNS). UDP is connectionless; UFW treats it as a separate transport from TCP, and the hub stores them as distinct rules.
• ufw allow 17890:17990/tcp — permit a port range on TCP. Port ranges count as ONE rule even though they cover many slots.
• ufw status numbered — print every active rule with its index. Cerberus parses this on every probe tick to detect drift.
• ufw --force enable / disable— turn enforcement on or off. Cerberus only enables; disable is gated behind an Ancient-Admin-only confirm flow (deferred to v.G.1.2 for the UI; the driver supports it today).

Default policy after Cerberus bootstrap is deny incoming, allow outgoing — no inbound traffic except what an explicit allow rule lets through. Cerberus inserts the three default rules (22/tcp, 80/tcp, 443/tcp) before enabling, so the box never locks out SSH at flip-time.

Cerberus auto-includes the SSH port the hub uses to reach this node in the seed defaults — for boxes that listen on a non-standard SSH port, bootstrapping never breaks the hub’s own access. The lock-out guard hinges on the actual port-of-record (e.g. 2222/tcp), not a hardcoded 22/tcp.

Why localhost ports don’t need UFW rules

UFW filters traffic that crosses a network interface. A listener bound to 127.0.0.1:PORT (or any address inside the loopback range 127.0.0.0/8) never touches a network interface — it lives entirely inside the kernel’s loopback path, which is invisible to iptables / netfilter.

Practical implication: a service that listens on 127.0.0.1:5984 (CouchDB), 127.0.0.1:6379 (Redis), or any other loopback-only address requires NO UFW allow rule. The port is unreachable from the network regardless of UFW policy. Only listeners bound to 0.0.0.0:PORT (every interface) or a specific public IP need an explicit allow rule.

Cerberus drift detection mirrors this: only network-facing listeners are checked against the rules table. The hub never raises an extra-on-box false positive for a loopback service that has no UFW row.

Tunnelees and Cerberus

A tunnelee is a node that connects to the cluster outbound through its parent tunneler. Tunnelees live behind CGNAT or restrictive firewalls and have no public-facing inbound surface — nothing reaches them from the outside that UFW could block or allow.

Cerberus refuses to manage tunnelees. The per-node panel for a tunnelee renders an offline view (“Cerberus is offline for tunnelees”) instead of the bootstrap and rules surface, the bulk-apply page omits them from the selectable fleet, and every mutation endpoint returns a 409 against a tunnelee target so a hand-crafted request can’t install rules that would never have any effect.

To control inbound traffic for a tunnelee, manage the firewall on the tunneler — that’s the box that actually terminates the public ports on the tunnelee’s behalf.

Master-only presets

Some firewall presets only make sense on a single role of box in the fleet. The mailcowpreset opens 10 inbound mail ports (SMTP, IMAP, POP3, HTTP/S, Sieve); Mailcow itself runs on exactly ONE box per fleet — the master server. Showing the preset on every node would let an operator install 10 inbound mail rules on a chainweb box that will never accept mail.

The hub gates the preset on node_category = ‘master’. On a non-master node:

• The mailcow card is hidden in the per-node firewall tab.
• The bulk-apply page disables non-master rows whenever mailcow is in the preset selection.
• Direct API calls to POST / DELETE /api/hub/nodes/[id]/firewall/presets/mailcow return 409 preset_master_only without writing to the DB or audit log.

Operators who need a single mail port on a non-master box (e.g. opening 25/tcpfor an outbound MTA on a forwarder) can still allow it via the custom-rule form — only the named mailcow PRESET is gated, not the underlying ports.

Cloud-panel and UFW — the two-tier posture

Cloud-panel firewalls (IONOS, Hetzner, Contabo) cap rule counts at 10–20 entries and conflict with the StoaChain port budget the moment more than two Tunnel slaves land on one host. Cerberus moves the policy down to UFW on the box and asks the cloud panel to step out of the way.

The cloud-panel guidance is verbatim: open TCP 1-65535 ALLOW + UDP 1-65535 ALLOW. That posture is only safe because UFW on the box becomes the real gate — default-deny on incoming, only the hub-intended rules open.

Tier 1 (cloud panel): one-time operator setup, fully open. Tier 2 (UFW on the box): hub-managed, audited, drift-checked. Cerberus owns Tier 2; Tier 1 stays a one-time human action outside the hub.

Custom rules and validation

• Port: integer 1–65535. Ranges use the low:high form with strict low < high — equal endpoints and inverted ranges are rejected.
• Protocol: tcp, udp, or any.
• Comment: up to 64 characters, whitelist ^[A-Za-z0-9 :_.-]*$ — printable-ASCII subset chosen to be safe inside a shell-quoted UFW comment field.
• Source CIDR: IPv4 only in v.G.1.1. IPv6 source restrictions are deferred.

Drift detection

The probe cycle reads ufw status numbered on every node and compares observed rules against the hub DB. Three drift kinds are tracked, each with its own reconcile path:

• missing-on-box— hub has the rule, UFW doesn’t. Reconcile action: re-apply-to-box.
• extra-on-box— UFW has a rule the hub doesn’t know about. Reconcile actions: import-from-box or mark-unmanaged.
• unmanaged— operator-marked rules. Reconcile action: re-apply-to-box as a verification pass; the hub stops trying to push it but still reads it back.
• in-sync— nothing to do.

Reconcile — in detail

When the probe-cycle drift step flags a rule as anything other than in-sync, the rules table on the per-node Cerberus Panel shows a Reconcilebutton on that row. Reconcile is the operator’s way of telling the hub what to do about the disagreement between hub state and UFW state — it never resolves drift on its own.

Concrete example: you bootstrap the firewall on a fresh node. The hub inserts three default rules (22/tcp, 80/tcp, 443/tcp) and applies them via SSH. If one of the SSH applies fails transiently — or if the apply succeeded but a probe ran beforethe apply finished writing UFW state — the rule shows up in the rules table with 22/tcp, source anywhere, preset default, drift missing-on-box. Clicking Reconcile opens an inline drawer with three actions:

• Re-apply to box— the hub re-issues the original ufw allow ... command over SSH and updates the row to in-syncon success. Use this when the rule is supposed to be on the box and you trust the hub’s record. Most common action for missing-on-box.
• Import from box— only available for extra-on-box drift. The hub adopts the existing UFW rule into its own state with preset_source = ‘custom’, drift becomes in-sync, and the rule is now hub-managed (it WILL be re-applied if it ever drifts again, and it can be deleted from the rules table). Use this when an operator manually added a rule via SSH and you want the hub to take ownership.
• Mark unmanaged — only available for extra-on-boxdrift. The hub records the rule’s existence but flags it with preset_source = ‘unmanaged’. The hub stops trying to push or remove it; UFW state is preserved as-is. Use this for rules you want to keep on the box without the hub touching them — e.g. a temporary debug port you’ll close manually.

Every reconcile click writes one row to admin_audit with the action verb, the rule id, the actor email, and (for re-apply) the literal UFW command issued. The audit trail is the source of truth for “who decided what about this rule and when” — UFW itself does not retain that history.

Tier gating

• Operator— per-node firewall tab on owned nodes only. The tab is hidden on non-owned nodes.
• Ancient Admin— per-node tab on every node, plus the bulk-apply page at /hub/firewall/bulk with node + preset multi-select and a dry-run preview.

v.G.1.1 scope-downs

• Edit-comment per-row is deferred. Comments are set at rule creation; changing one means delete + re-add for now.
• UFW disable UI is deferred. The driver supports it; the hub doesn’t expose the action in v.G.1.1.
• The per-node tab uses a cards + form layout, not the popover-with-tabs sketch from the original spec.
• Source CIDR validation is IPv4-only. IPv6 source rules are deferred (UFW handles v6 transparently for the no-source case via parallel ip6tables).