Plugins

Bytes can be passed through a pipeline of plugins on their way between the endpoints. A plugin can watch the stream, rewrite it, or drop parts of it.

Plugins are written like endpoints (a name, an optional direction, and a set of options) and go between the source and the sink on the command line.

name[:direction],option,option=value

$ tocat tcp-listen:8080,fork tee,format=hex tcp:example.com:80

The same entries can be given with -p, which is the only way to add one when both endpoints come from flags or a config file. Repeat it as needed; entries apply in the order written, after any in the config file.

$ tocat -f tcp-listen:8080,fork -t tcp:example.com:80 -p 'tee,format=hex'

Run tocat --list-plugins to see what your build has, and --no-plugins to ignore the ones in a config file.

What ships with tocat

“Datagram paths” is what each stage reports about itself, and it is what tocat checks when the destination on that path is a datagram endpoint. The default for a stage that says nothing, including any plugin from outside the binary, is that it is not safe.

A few stages go further and need message boundaries rather than merely preserving them. encrypt and decrypt in record mode are the two that ship, and an unmet requirement is an error rather than a warning: see When a stage needs boundaries below.

Writing an entry

On the command line an entry is comma-separated, as an endpoint is. A bare key means true (tee,append), a value that parses as an integer becomes one, true and false become booleans, and everything else stays a string. Sizes and durations are therefore written as they are elsewhere (bytes=10MiB, flush=250ms), and a value containing a comma has to go in a config file.

Option keys are matched leniently: case, dashes and underscores are noise, so at-limit, at_limit and atLimit are the same option. An option the plugin does not declare is an error rather than being ignored.

In a config file, an entry is a [[plugin]] table with the plugin’s own options alongside name, direction, as and detach:

[[plugin]]
name = "tee"
direction = "sink-to-source"
file = "/var/log/tocat-capture.bin"
format = "raw-binary"

Direction

An entry applies to one path or both. Omitting the direction means the forward path, from the source to the sink.

both builds two independent instances, one per path, so per-direction state (byte offsets, codec state) never leaks across paths. That also means two of whatever the stage counts: limit,bytes=1MiB,direction=both lets a megabyte past in each direction rather than a megabyte between them, and a ticking stage declared that way builds a timer per path per connection. Asking for both is therefore worth doing on purpose, which is why it is no longer what you get by saying nothing.

An asymmetric stage is not its own inverse, so both is wrong for it and the pair is written out:

$ tocat - compress decompress:reverse tcp:relay.internal:9000

The command line is a picture of the wire, read left to right, and the reverse path reads it right to left:

tocat SRC  a  b  c  SINK

source -> sink:   SRC -> a -> b -> c -> SINK
sink -> source:   SRC <- a <- b <- c <- SINK

So a stage written earlier sits nearer the source, and bytes coming back from the sink reach the later stages first. Write the stages in the order the forward path would see them, and the reverse path nests correctly, which is what you want for anything that wraps the payload.

The source/sink aliases are accepted but read badly: tee:sink means sink-to-source, not “tee at the sink”. Prefer forward and reverse.

Common options

Two options are handled by tocat rather than the plugin, and work on any entry.

The #n suffixes are only added where a name would otherwise appear twice on the same path, so a lone tee stays tee.

compress and decompress default to detached, and detach=false on one is honoured. process runs as a child process and always has its own task, so detach=false on one is rejected rather than ignored.

If nothing is declared on a path, that direction is copied straight through with no plugin machinery in the way.

Datagrams

On a byte stream a chunk is an arbitrary slice, and a stage may buffer, split or coalesce it freely. On a datagram path the chunk is the message: one call per datagram, and each unit the stage emits is sent as exactly one datagram. A stage that holds bytes across calls, or emits two messages’ worth from one, will produce well-formed datagrams containing nonsense.

Every stage declares whether it may sit on such a path. When one that may not is on a path whose destination is a datagram endpoint, tocat names it, warns, and relays anyway:

stage may not preserve message boundaries; datagrams sent to this endpoint may be split, merged, or malformed

That is a warning rather than a refusal because rewriting the message stream is sometimes the point: block,size=1400 on a UDP path emits one datagram per block, which is a reasonable thing to ask for and still not a preservation of what the peer sent.

The check is against the destination, not the source. Sending datagrams into a stream sink is unremarkable and draws no warning. So udp-listen:9000 compress:forward tcp:collector:9000 is fine, while udp-listen:9000 compress udp:peer:9000 will warn. A process stage always warns: its stdin and stdout are byte streams, so boundaries are gone the moment bytes cross the pipe.

When a stage needs boundaries

encrypt and decrypt do in record mode, a plugin from outside the binary can say that it does, and a WebAssembly guest declares it through TOCAT_NEEDS_UPSTREAM and TOCAT_NEEDS_DOWNSTREAM. A stage that reads whole messages needs one message per call; a stage that writes whole messages needs the ones it emits to arrive intact. Neither is a matter of taste, so an unmet requirement stops the relay before it starts rather than warning, and names both the stage and what took the boundaries away:

wasm on source-to-sink needs message boundaries arriving, and the source tcp://host:9000 is a byte
stream: put an unframe above it, or use a stage that does not need them

There are two ways to satisfy one. A datagram endpoint on the relevant side supplies boundaries by itself, so nothing more is needed on a UDP path. On a byte stream, frame and unframe are what put them there: unframe above a stage that reads messages, frame below one that writes them. Either in mode=null satisfies the requirement without writing any framing, which is what you want when the far end is a terminal rather than a peer, and is not what you want otherwise.

The scan stops at the first stage that settles the question, so a frame anywhere below covers everything under it: once the boundary is in the payload no later stage can lose it. A stage needing its units delivered is therefore happy with frame,mode=length below it even when a compress sits below that, and unhappy with the same two the other way round, since the units are gone by the time the framer sees them.

The check goes as far as confirming something downstream can put the boundaries on the wire. It cannot confirm the bytes reach the peer intact, or that the peer unframes them: see What framing survives.

One more thing a datagram sink does silently: an empty emission is not sent. At end of stream a pipeline is drained once more, and on a datagram sink that would otherwise put a spurious zero-length message on the wire, which many peers read as a close. The cost is that a genuine zero-length datagram is dropped rather than forwarded.