Streaming and performance

Streaming

Rules are matched as the data flows through. A pattern is found wherever it lands, even split across packets or megabytes apart, and nothing is buffered to find it: the state of a connection is a few bytes per rule. The proxy's memory stays flat however large the messages are: in testing it held at about 28 MB while 50 MB uploads streamed through a port with rules.

All the patterns of consecutive rules are compiled into one state machine ahead of time, when the rules load, so each byte is looked at once no matter how many rules there are.

A pattern that cannot be compiled into a state machine is rare, and logged when the rules load. That pattern alone is matched over a 64 KiB window instead, so for it a match has to fall within that distance. The rest of the rules keep streaming.

Writing fast patterns

Patterns that begin with plain text are matched with a SIMD search that skips over uninteresting data. A rule set made only of such patterns (everything from contains, and regexes that start with literal text) is matched around 17 times faster than one where a pattern starts with something else, such as \b or a character class.

- match: { regex: 'UNION\s+SELECT' }   # starts with literal text: fast
- match: { regex: '\bUNION\b' }        # starts with \b: every byte goes through the state machine

It is worth keeping in mind when a rule could be written either way. It is rarely worth contorting a rule for: even the slow path handles a 64 byte read in about 120 ns.

Latency

vulnboxd adds very little. With connection pooling (the default), an http request through the proxy costs one TCP handshake, the same as going to the service directly. The proxy is multi-threaded and has no per-connection limits, so it scales with the box.

HTTP bodies

Some rules need a whole http message, so for each request and response vulnboxd decides from the head alone what the body is needed for:

No applicable rule looks at the body. It is forwarded untouched, without buffering.

Only rules that can work on a stream: contains, regex, replace, or larger_than on a body whose length the head does not give. The body is buffered up to limits.buffered_body so that blocking can answer cleanly, and streamed beyond that. A blocked request then gets a 403 even if it was already partly forwarded; a blocked response is cut off mid-flight, because its head has already been sent.

A rule uses body or set_body. The whole body is needed. Larger messages are answered with 413 (request) or 502 (response) rather than passed through unchecked. Scope such rules with path so that large endpoints are not affected, or raise limits.buffered_body.

limits:
  buffered_body: 8388608   # 8 MiB, the default

A rule only counts if it applies to the message: a body rule scoped to path: '^/render' makes no difference to requests elsewhere.

Memory and file descriptors

Each proxied connection needs two file descriptors, one to the client and one to the service, which is what the systemd unit raises the limit for.