DocsReferenceThe wire protocol

The wire protocol

A dedicated control stream carries a small, fully-typed set of JSON messages between the agent and the edge. Every other stream — one per HTTP/1.1 exchange or native HTTP/2 connection — carries a single JSON preamble followed by the selected protocol bytes.

Framing & encoding#

Every control message and every stream preamble uses the same trivial framing: a big-endian length prefix followed by a JSON payload.

uint32 big-endian payload length | JSON payload (max frame: 64 KiB)

JSON was chosen over protobuf or gob — it's greppable in debug logs, control-plane volume is a few messages a minute so performance is irrelevant, and it avoids codegen for a handful of message types owned by one repo. HTTP/1.1 data streams carry raw HTTP/1.1 bytes after the preamble; native HTTP/2 streams carry the HTTP/2 preface and frames. JSON never touches either data path.

The message envelope#

Every control-stream message is wrapped in the same envelope:

internal/proto/proto.gogo
type Msg struct {
Type string `json:"type"` // discriminator
ID uint64 `json:"id,omitempty"` // correlates request→reply;
// per-direction ID spaces
Data json.RawMessage `json:"data,omitempty"` // type-specific payload
}

ID spaces are per-direction — the agent mints odd IDs, the edge mints even ones — so both peers can initiate a message (both send Ping) without colliding. Replies don't get a fresh ID: a reply carries the same Msg.ID as the request it answers — AuthOK echoes Auth's ID, BindOK echoes Bind's, Pong echoes Ping's.

Handshake#

The first exchange on every control stream. Anything else as the first message closes the connection.

type Auth struct {
Token string `json:"token"`
InstanceID string `json:"instance_id"` // stable per agent-process start
ClientVersion string `json:"client_version"` // e.g. "v0.1.0 (commit 3f1a9c2, built 2026-06-01T12:00:00Z)"
ProtoVersions []int `json:"proto_versions"` // [2, 1] — supported protocol majors
}
type AuthOK struct {
ProtoVersion int `json:"proto_version"` // server picks highest mutual
SessionID string `json:"session_id"` // minted per session
ServerVersion string `json:"server_version"`
}
type AuthErr struct {
Code string `json:"code"` // bad_token, no_common_version
Message string `json:"message"`
}

The first message must be Auth within 5 seconds of TLS establishment, or the edge closes the connection. In scoped-credential mode the edge hashes the bearer token, checks its expiration and revocation state, and attaches the resulting scope to the session. The legacy mode compares the token against the configured SHA-256 digest.

Tunnel lifecycle#

A bound session can bind, unbind, and be evicted by a takeover — six message types cover the whole lifecycle.

Bind / BindOK / BindErr

kindstring
"http" or "tcp". A "tcp" bind skips the subdomain/hostname machinery entirely and gets back an allocated port instead — see TCP tunnels.
subdomainstring
Empty string requests a random name, e.g. amber-fox-42. Ignored for kind: "tcp".
remote_portint
kind: "tcp" only. Requested remote port; 0 means server-assigned.
takeoverbool
Reclaim a binding — authorized by identity, not liveness.
basic_auth_sha256[]string
v2 only. SHA-256 digests, hex-encoded, of user:pass credentials — the edge never sees the plaintext. HTTP only; a TCP bind carrying this field is rejected. See Access control.
basic_auth_realmstring
v2 only. WWW-Authenticate realm string. Ignored unless basic_auth_sha256 is also set; falls back to a default realm if empty.
allow_cidrs[]string
v2 only. Source CIDRs allowed to reach the tunnel. Applies to both kind: "http" and kind: "tcp".
policyobject
v2 only and HTTP-only. Ordered method/path/header rules (allow, deny, redirect, header transform) and an optional CORS block sent from --policy-file. The edge evaluates the policy before opening a request stream. An older edge that does not understand these fields rejects the bind instead of ignoring them.
policy_digeststring
SHA-256 digest of the normalized policy. The agent includes it inBind, and the edge echoes it in BindOK so an older edge cannot silently ignore a policy.
http2bool
v2 only and HTTP-only. Selects the native HTTP/2 data path. The edge echoes this in BindOK; the agent fails closed if an older edge silently ignores the request.
multi_connectorbool
v2 only and HTTP-only. Requires an explicit subdomain and lets multiple sessions join one load-balanced connector pool. Every connector must match the pool's HTTP/2, access-control, and traffic-policy settings. The edge echoes this in BindOK.
connector_token_sha256string
v2 only and required with multi_connector. SHA-256 hex digest of the per-tunnel connector secret. The secret itself is never placed in the control message; a mismatch prevents a session from joining the pool.
hostnamestring
Returned in BindOK: the full bound hostname, e.g. api.ngstone.site.
urlstring
Returned in BindOK: the full public URL.
portint
Returned in BindOK for kind: "tcp": the allocated remote port.
codestring
Returned in BindErr: subdomain_taken, invalid_subdomain, kind_unsupported, port_unavailable, too_many_binds, invalid_access, or connector_config_mismatch, or credential_scope.

invalid_access

Returned when access-control or policy fields fail validation (a malformed CIDR, a digest that isn't 32 bytes, too many credentials/CIDRs, an invalid policy, or protected fields on a tcp bind), and also when a session negotiated protocol v1 but the Bind carries any protected field — see Versioning.

Unbind / UnbindOK / Evicted

hostnamestring
The agent drops one tunnel; the session itself stays up.
reasonstring
Sent only on Evicted: always "takeover" in v1 — this binding was reclaimed, stop advertising the URL.

Liveness & shutdown#

Both peers send Ping on the control stream every 15 seconds; Pongechoes the ping's timestamp so the sender can log RTT. Three consecutive misses (≈45 seconds) declares the peer dead.

A scoped credential is revalidated from the credential file on each edge heartbeat. If it expires, is revoked, or the file becomes unreadable, the edge sends Goodbyeand releases the session's bindings. This keeps revocation independent without putting the plaintext token on the wire again.

type Ping struct { TS int64 `json:"ts_unix_ms"` }
type Pong struct { TS int64 `json:"ts_unix_ms"` } // echoes Ping.TS

Graceful shutdown is fire-and-forget by design — there is no GoodbyeOK. The sender starts its own drain timer the moment it sends and behaves the same whether or not the peer honored the message.

type Goodbye struct {
Reason string `json:"reason"` // "shutdown", "reload", "idle"
DrainMS int `json:"drain_ms"` // sender stops opening new streams
}

Stream preamble#

Every data stream opens with one JSON frame — the only JSON on that stream — before raw bytes take over.

type StreamOpen struct {
Kind string `json:"kind"` // "http", "h2", or "tcp"
Host string `json:"host"` // which binding this request targets
ReqID string `json:"req_id"` // edge-minted ULID; ties edge logs ↔
// inspector entries
ClientAddr string `json:"client_addr"` // public client ip:port
}

A Kind: "h2" stream carries a native HTTP/2 connection after the preamble. The edge opens it with the HTTP/2 client transport; the agent serves it with an HTTP/2 server, allowing request trailers, response trailers, streaming, and full-duplex exchanges to pass through. A Kind: "http" stream continues to carry one HTTP/1.1 request/response exchange.

ClientAddr is carried here rather than derived from the stream, because a Kind: "tcp" stream has no X-Forwarded-For header to fall back on.

Versioning#

The protocol uses a single integer major version, currently 3. The agent offers every version it supports in Auth.ProtoVersions — [3, 2, 1]— and the edge picks the highest version present in both its own supported list and the agent's offer, returning it in AuthOK.ProtoVersion. An edge that only understands v1 still negotiates fine with a v2-capable agent; it just settles on 1.

v2 added the three Bind access-control fields above (basic_auth_sha256, basic_auth_realm, allow_cidrs) and the invalid_access error code. Access control and traffic policies are hard v2 requirements, not best-effort features: if a session negotiates v1, the edge rejects a Bind carrying any protected field with invalid_access rather than silently binding without protection, and the agent checks this client-side too — it refuses to send the Bind at all when --basic-auth, --allow-cidr, or --policy-file was requested but the negotiated version is below 2. A policy also requires a matching digest in BindOK. Either way the tunnel fails closed instead of coming up unprotected.

HTTP/2 mode is also v2-only and requires the http2 echo inBindOK, so an older edge cannot silently fall back to HTTP/1.1.

v3 added database sharing: Auth.share_secret as an alternative to token, AuthOK.scope (operator or connector), the share/share_ok/share_err, unshare, share_update, kick, approve, list_shares, watch_shares, connect/connect_ok/connect_err, share_event, share_closed and share_updated messages, a share stream kind and optional share_id and target fields on the stream preamble. A connector-scoped session may send only connect, ping, pong and goodbye; anything else is answered with forbidden and the session closes. The full message set is in docs/SHARE-DESIGN.md.

Additive JSON fields never bump the version — unknown fields are simply ignored. Semantic changes do. Within a negotiated version, an unknown Msg.Type or a malformed frame gets a generic Error reply and then the session closes — never a silent ignore.

Error is also a catch-all reply

Error{Code, Message}is sent as the reply to any request the receiver can't satisfy, and also right before a connection is closed for a protocol violation. Codes include unknown_type, malformed, not_bound, and internal.