Day-to-day operation

Running compose

Always use vulnboxctl compose instead of docker compose for a managed service:

sudo vulnboxctl compose web ps
sudo vulnboxctl compose web logs -f --tail 100
sudo vulnboxctl compose web restart
sudo vulnboxctl compose web exec app sh

It runs docker compose for that service with vulnboxd's override applied, and hands over the terminal, so Ctrl+C and interactive commands behave as usual. A plain docker compose up in the service directory would give the containers their original ports back, where they clash with the proxy.

You rarely need up yourself: push to git and vulnboxd deploys at the end of the tick.

The compose guard

The installer puts a small wrapper in front of the docker compose plugin. In a service directory vulnboxd manages, it refuses what would bring the containers up without the override, and says what to run instead:

$ docker compose up -d
vulnboxd manages web: `docker compose up` here would give its containers their public ports back and
lock vulnboxd's proxy out.

    vulnboxctl compose web up ...

runs it with the override. To do it anyway: VULNBOXD_COMPOSE=1 docker compose ...

Only up, create and run --service-ports are refused. ps, logs, exec, down and the rest work as usual, and anything outside the services directory is not touched. If the guard itself cannot decide, because the config does not load say, it lets the command through rather than stand between you and docker.

Watching it: vulnboxctl top

vulnboxctl top

A terminal dashboard of what the running daemon is doing. The header shows totals for open connections and rule hits; below it, every service with its ports, open connections and rule hits, and for the selected service, each port's rules with how often each one has fired. It reads the daemon's metrics endpoint and the rules files, and refreshes every second.

key
↑ ↓	select a service
r	refresh now
q / Esc	quit

vulnboxctl dashboard is the same command.

Logs

vulnboxd logs to stderr, which systemd sends to the journal:

journalctl -fu vulnboxd

Verbosity is set with RUST_LOG, for example RUST_LOG=debug or, for one area, RUST_LOG=info,vulnboxd::proxy=debug. At info you see discoveries, builds, deploys, git updates and every rule that fires, with a preview of what it matched. Mirrors that agree are logged at debug.

The capture broker is its own service, with its own journal:

journalctl -fu vulnboxd-pcap

Changing rules during a game

1. Add the rule with action: log and save. It is live within a second.
2. Watch vulnboxctl top, or curl -s localhost:9330/metrics | grep rule_hits. A logging rule counts as action="matched".
3. Check that what it catches looks like exploits, and that the service stays up on the scoreboard while the rule is in. Then remove action: log.
4. Commit and push, so the rest of the team has it.

If the file does not parse, the error is logged and the previous rules stay active; nothing is dropped.

When a change breaks a service

sudo vulnboxctl rollback web

This undoes the latest change, code or rules, with a pushed commit. vulnboxd also does this by itself when a new version does not come up, or when the checker says down after a change. See Rollback.

Taking a service out of vulnboxd's hands

• Stop filtering one port but keep deploying: intercept: false on the port. Applied at the next deploy.
• Stop filtering the whole service: intercept: false at the top of its rules file.
• Stop managing it entirely: add its directory name to exclude in the config, or move it out of the services directory.

Troubleshooting

A service is unreachable after a deploy. If the new version failed its health check, vulnboxd has rolled it back already and sent an alert saying why. Otherwise, check vulnboxctl compose <service> ps and its logs. The proxy retries for deploy.grace and then gives up on that connection; vulnboxd_upstream_errors_total counts how often.

A service went down on the scoreboard after I added a rule. The usual cause, and the reason the docs keep saying to start with action: log. Find the rule in vulnboxctl top: a rule whose hit count climbs once per tick, rather than with attack traffic, is matching the checker. Set it to action: log and save; it is live within a second and the service recovers on the next tick. Then narrow the pattern until the count only moves when you attack the service yourself, and put the action back.

Deploys spill past the window. Look at vulnboxd_deploy_ms. The usual culprit is a container that ignores SIGTERM; see services that stop slowly.

Windows fall in the middle of ticks. Set tick.start to a real tick boundary; see lining up with the game.

Git updates stop arriving. vulnboxd_git_polls_total{result="failed"} goes up and the reason is logged. A rebase that conflicts is left for a human: resolve it in the service directory, and polling picks up from there.

The checker's verdicts have stopped. vulnboxd_sla_up keeps its last value rather than going to zero, and gameserver: ... is logged once. While the gameserver is quiet nothing is rolled back, so this is safe to wait out; see when the gameserver is quiet.

An upgrade did not switch. vulnboxctl upgrade says whether the daemon acknowledged it. A handover waits for the end-of-tick window unless you passed --now, and it will not land in the middle of a deploy. If the new build cannot run, the old one keeps going and alerts. Confirm which is running with vulnboxd_info{version=...} rather than vulnboxd --version, which only reports the file on disk. See Upgrading.

No packets reach my capture reader. Check the service is up at all — systemctl status vulnboxd-pcap — and then what it is capturing: pcap.ports defaults to the game's range, so traffic outside it is not captured. journalctl -u vulnboxd-pcap shows the exact tcpdump command. A reader that was disconnected rather than never fed is logged as having fallen too far behind; see slow readers.

Too many open files. vulnboxd raises its limit to the hard limit at startup and warns if that is below 65536. Raise LimitNOFILE in the systemd unit.