Running the edge on a VPS
ngstoned is the half of ngstone that lives on the public internet. This is a runbook: work top to bottom on a fresh Linux server and you end with a managed service holding valid certificates, an open control plane, and an agent tunnelling through it. example.com stands in for your domain throughout.
Fast path installer#
On a fresh Ubuntu or Debian VPS, create the apex and wildcard DNS records first and create an API credential for your DNS provider, then install the edge from a checkout. The installer downloads both release binaries, verifies their checksums, creates the root-only secrets file, installs the hardened systemd unit, configures UFW, and starts ngstoned:
$ git clone https://github.com/emilio-kariuki/ngstone.git /opt/ngstone$ cd /opt/ngstone$ sudo ./deploy/install-edge.sh \ --domain example.com \ --email you@example.com \ --dns-provider cloudflare \ --acme-ca stagingIt prompts for the chosen provider's credentials without echoing them, initializes durable SQLite storage, and prints a scoped agent credential once. Set NGSTONE_DATABASE_URL before running it to use PostgreSQL. The installer does not create DNS records, so the apex and wildcard must already resolve to the VPS. Once the health output shows both certificates, switch to trusted certificates:
$ sudo ./deploy/install-edge.sh --acme-ca productionA rerun with only --acme-ca production keeps the domain, email, DNS provider, credentials, TCP port range, and auth token from the existing installation. The installer then prints the laptop commands, which connect to the bare domain, example.com:4443.
staging or production. Start with staging, then promote; neither browsers nor the agent trust staging certificates. A rerun keeps the current value.10000-10100. Adds -tcp-ports to the unit and opens the range in UFW; the range is kept across reruns and --tcp-ports off removes it. It must lie within 1024-65535 and must not include 4443.ngstoned with -quic so agents can use --transport quic. Opens 4443/udp in UFW and writes /etc/sysctl.d/99-ngstone-quic.conf with larger UDP socket buffers (net.core.rmem_max and net.core.wmem_max of 7500000). A key whose live value already exceeds that gets no entry, and no file is written if the installer has not written one before and both exceed it. Once the file exists, a rerun keeps its keys at 7500000. The installer never lowers a live value, but at boot the file overrides any earlier-sorting drop-in for the same key, though /etc/sysctl.conf or 99-sysctl.conf can still override it; edit or remove its key to keep a larger persistent value. off removes the flag, the rule and the file.ngstoned from the checkout with Go instead of downloading a release. Needed for providers other than Cloudflare until a release that includes them is published; the installer refuses to install a binary that does not support the chosen provider.v0.3.2.Passing credentials without a prompt
sudo drops variables exported in your shell, so put them on the sudo command line instead: sudo NGSTONE_CF_API_TOKEN=... ./deploy/install-edge.sh .... Either way they are stored only in the root-only /etc/ngstoned/env.Uninstalling#
From the checkout, remove the service, binary, and root-only secrets:
$ sudo ./deploy/uninstall.shCertificate, ACME, and default SQLite control-plane state is preserved by default under /var/lib/private/ngstoned, so a later reinstall can reuse it. To delete that state and remove matching public UFW allow rules, opt in explicitly:
$ sudo ./deploy/uninstall.sh --remove-state --remove-firewall-rulesSSH access is kept
80/tcp, 443/tcp, 4443/tcp, 4443/udp if QUIC was enabled, and the TCP tunnel range if one is configured. SSH rules and the source checkout are left in place. If another service uses the same allow rule, review it before selecting this option. Use --yes only for a deliberate non-interactive uninstall.Running it with Docker#
Instead of the installer, run the ghcr.io/emilio-kariuki/ngstoned image (amd64 and arm64) on any host with Docker and the Compose plugin. The DNS records, the provider credential, and the open ports (80, 443, 4443) are the same; the installer's UFW and DNS checks are yours to do. Fetch the Compose file and the environment template, fill in .env, and start it:
$ mkdir ngstone && cd ngstone$ curl -fsSLO https://raw.githubusercontent.com/emilio-kariuki/ngstone/main/deploy/docker/compose.yaml$ curl -fsSL -o .env https://raw.githubusercontent.com/emilio-kariuki/ngstone/main/deploy/docker/.env.example$ chmod 600 .env && $EDITOR .env$ docker compose up -d$ docker compose ps # STATUS turns "healthy" once both certificates load$ docker compose logs -f ngstoned.env holds the domain, email, DNS provider and its credentials, and the token hash. Compose passes the secrets to the edge as environment variables only, never as flags. When the container reports healthy, set NGSTONE_ACME_CA=production in .env and run docker compose up -d again. Certificates live in the ngstoned-data volume, so restarts and docker compose pull upgrades reuse them.
8080, 8443 and 4443 inside the container, so it needs no added capabilities; the host publishes them as 80, 443 and 4443. The ops endpoint stays on the container's loopback, where the image's health check (ngstoned healthcheck) probes /healthz.--allow-cidr work without -proxy-protocol. IPv6 clients are the exception unless IPv6 is enabled on the Compose network.65532. The named volume is writable by it; a bind mount in its place must be chowned to 65532:65532 first.-tcp-ports and the matching ports range in compose.yaml; the two ranges must be identical.Prerequisites#
- 1
A Linux VPS with a public IP
Anything that runs a static Go binary. It needs a stable public address — every DNS record below points at it — and root orsudofor installing a systemd unit and binding privileged ports. - 2
A domain you control
A registered domain, hereexample.com. Tunnels get hostnames one label below it, such asamber-fox-42.example.com. - 3
The zone hosted on a supported DNS provider
Certificates are issued through the ACME DNS-01 challenge, songstonedneeds API access to the zone. The domain's nameservers must point at one of the supported providers and the zone must be active there before anything below will work. - 4
A machine to run the agent
Your laptop, with thengstonebinary installed — see Installation.
DNS records#
Create two records in the example.comzone, both pointing at the VPS's public IP.
4443, which is what the agent dials, and the apex certificate that secures it.amber-fox-42.example.com at bind time and never touches DNS, so every possible tunnel name has to already resolve to the VPS through this one record.Use AAAA instead if the VPS is IPv6-only, or add both.
DNS-only is simplest; auto has a :443 fallback
:4443, but the default --transport auto can fall back to the authenticated WebSocket control endpoint on HTTPS :443 when WebSockets are enabled. Use DNS-only records when forcing --transport raw or when you want the simplest path. --transport quic needs UDP on 4443 and cannot pass through the proxy.Choosing a DNS provider#
The edge needs to create and delete _acme-challenge TXT records in the one zone. Pick the provider that hosts the zone with -dns-provider (default cloudflare) and create a credential scoped as tightly as that provider allows. Credentials are read from the environment only; ngstoned -dns-provider help lists the providers and the variables of the binary you have.
route53:ListHostedZonesByName, route53:ListResourceRecordSets, route53:ChangeResourceRecordSets, and route53:GetChange.dns.hetzner.com must be migrated first.ovh-eu, ovh-ca, or ovh-us. Limit the consumer key to GET, POST, and DELETE on /domain/zone/*.GoDaddy, Namecheap, Porkbun and others are not supported
libdns modules replace or delete every TXT value at a name, and the apex and wildcard certificates publish two values at _acme-challenge.example.com at the same time, so one issuance would break the other. Keep the registrar and move only the DNS hosting to a supported provider by changing the nameservers — Cloudflare and deSEC are free.The Cloudflare API token#
The default provider, and the example used below. Create a scoped API token for exactly the challenge records — never the Global API Key, which authenticates everything on the account.
- 1
Open the token editor
Cloudflare dashboard → My Profile → API Tokens → Create Token → Custom Token. - 2
Grant one permission
Zone/DNS/Edit. Nothing else. - 3
Scope it to one zone
Zone Resources:Include/Specific zone/example.com. Not “All zones”. - 4
Keep the value
Cloudflare shows the token once. It goes into the edge's environment file asNGSTONE_CF_API_TOKENbelow.
A token scoped this way can write challenge records in one zone and nothing more — it cannot read other zones, change nameservers, or touch account settings. If it leaks, the blast radius is one zone's DNS.
Two certificates, not one#
ngstoned calls ManageSync(ctx, ["*.example.com", "example.com"]) once, which reads like a single multi-domain certificate. It is not. certmagic does not obtain multi-SAN certificates: that call places two independent ACME orders and produces two certificates with two independent renewal schedules.
This is forced anyway, because a wildcard covers exactly one left-most label (RFC 6125) — *.example.com does not match SNI example.com. The wildcard serves public tunnel traffic; the apex certificate serves the WebSocket fallback on :443 and the raw control plane on :4443. Both names validate through the same _acme-challenge.example.com TXT record, which is why only providers that add and remove challenge values individually are supported.
A healthy wildcard can hide a dead apex
ngstoned checks at startup that an apex certificate is loadable with at least 24 hours of life left before accepting on :4443, and exits loudly rather than listening with a certificate it cannot serve. /healthzreports each certificate's expiry separately — check both, not just one.Durable storage and scoped tunnel credentials#
Issue one credential per tunnel
$ ngstone admin credentials issue --server http://127.0.0.1:4041 \$ --subdomain demo --ttl 24hcredential <id> issuedtoken: <save this value on the laptop>Copy the printed token to the laptop and run ngstone auth, then use ngstone port 3000 --subdomain demo. Set NGSTONE_DATABASE_URL=sqlite:///var/lib/ngstoned/control.dbin the unit's environment. For active/active edges, use the same PostgreSQL URL on every edge and a different NGSTONE_EDGE_ID. Forwarding a connection between edges also needs NGSTONE_PEER_ADDR.
The edge reads its storage settings, token hash, ACME email, and DNS provider's credentials from its environment. Put them in /etc/ngstoned/env, mode 0600, owned by root — for Cloudflare:
NGSTONE_DATABASE_URL=sqlite:///var/lib/ngstoned/control.dbNGSTONE_EDGE_ID=edge-aNGSTONE_CERT_EMAIL=you@example.comNGSTONE_CF_API_TOKEN=cf_fake_token_value_goes_hereFor active/active edges, point every edge at the same PostgreSQL URL and add NGSTONE_PEER_ADDR; the edge refuses to start with it on SQLite:
NGSTONE_DATABASE_URL=postgres://ngstone:<password>@db.internal/ngstone?sslmode=requireNGSTONE_EDGE_ID=edge-aNGSTONE_PEER_ADDR=10.0.0.11:4043<ip>:<port> of this edge, set on every edge that shares the PostgreSQL database. It must be an IP address literal and a port; hostnames and unspecified addresses are rejected. Requires PostgreSQL and is refused with -insecure. Equivalent flag -peer-addr. With it set, any edge forwards a tunnel's HTTPS, TLS-passthrough or raw TCP connection once to the edge holding its live connector, and that edge enforces every access rule using the real client address. Keep the port reachable only from the other edges. Empty disables forwarding.<ip>:<port> or :<port>) on which this edge accepts peer connections when that differs from NGSTONE_PEER_ADDR, for example inside a container. Requires NGSTONE_PEER_ADDR, which stays the address the other edges dial. Equivalent flag -peer-listen. Defaults to NGSTONE_PEER_ADDR.-token-sha256; use it only while migrating older installations.-cf-api-token; the environment wins when both are set. Other providers read the variables listed under Choosing a DNS provider instead, and have no flags at all.ngstoned directly — the unit interpolates it into -cert-email, which is required unless -insecure. It is the ACME account contact, not a secret.Never pass secrets as ExecStart flags
/proc/<pid>/cmdline is world-readable, so any local user can read a flag value with ps auxww — but not the process environment. That is precisely why Secrets belong in the environment file. Explicit flags override their environment defaults; other DNS providers' credentials are available only through environment variables. Keep secrets out of ExecStart.Installing the binary#
Release archives contain both binaries — ngstone and ngstoned. On the VPS, download the archive matching its architecture, verify the checksum, and install the edge binary:
$ VERSION=v0.2.0$ ARCHIVE=ngstone_linux_amd64.tar.gz # ngstone_linux_arm64.tar.gz on arm64$ BASE=https://github.com/emilio-kariuki/ngstone/releases/download/$VERSION$ curl -fsSLO "$BASE/$ARCHIVE"$ curl -fsSLO "$BASE/checksums.txt"$ sha256sum --ignore-missing -c checksums.txt$ tar -xzf "$ARCHIVE" ngstone ngstoned$ sudo install -m 0755 ngstone /usr/local/bin/ngstone$ sudo install -m 0755 ngstoned /usr/local/bin/ngstoned$ ngstoned -versionRunning it under systemd#
Install the unit, create the environment file from the section above, and start the service:
[Unit]Description=ngstoned edge serverAfter=network-online.targetWants=network-online.target [Service]Type=simpleExecStart=/usr/local/bin/ngstoned \ -domain example.com \ -cert-email ${NGSTONE_CERT_EMAIL} \ -acme-ca staging \ -cert-storage ${STATE_DIRECTORY}/certsEnvironmentFile=/etc/ngstoned/envRestart=alwaysRestartSec=2s StateDirectory=ngstonedStateDirectoryMode=0700 AmbientCapabilities=CAP_NET_BIND_SERVICENoNewPrivileges=yesProtectSystem=strictProtectHome=yesPrivateTmp=yesPrivateDevices=yesProtectKernelTunables=yesProtectKernelModules=yesProtectControlGroups=yesRestrictSUIDSGID=yesRestrictRealtime=yesLockPersonality=yes DynamicUser=yes LimitNOFILE=65536 [Install]WantedBy=multi-user.target$ sudo mkdir -p /etc/ngstoned && sudo chmod 700 /etc/ngstoned$ sudo $EDITOR /etc/ngstoned/env$ sudo chmod 600 /etc/ngstoned/env$ sudo systemctl daemon-reload$ sudo systemctl enable --now ngstoned$ systemctl status ngstonedDynamicUser=yes gives the service a transient, unprivileged UID allocated fresh on every start, and AmbientCapabilities=CAP_NET_BIND_SERVICE lets that unprivileged user still bind :80 and :443. The certificate cache has to survive across those changing UIDs, which is what StateDirectory=ngstoned is for: systemd creates /var/lib/private/ngstoned (with a /var/lib/ngstoned symlink) and re-chowns it to the current UID on every start. Point -cert-storage at ${STATE_DIRECTORY}/certs so the two always agree.
-domain is required and has no default. For a DNS host other than Cloudflare, add -dns-provider NAME to ExecStartand put that provider's variables in the environment file.
Do not substitute ReadWritePaths= for StateDirectory=
ReadWritePaths= gets no ownership handling from systemd, so a freshly assigned dynamic user cannot read the existing cache. The certificates are silently lost and re-issued on every restart, burning Let's Encrypt rate limits until issuance starts failing.-acme-ca defaults to staging and the unit above keeps it there deliberately. Staging certificates are not trusted by browsers but exercise the identical code path, so a misconfiguration costs nothing. Once systemctl status ngstoned is clean and /healthz reports sane expiry dates for both certificates, switch the unit to -acme-ca production and restart. Do not delete the staging certificates first — certmagic namespaces storage per issuer, so the two live in separate subtrees and cannot collide.
Firewall#
Four listeners, three of them public. The addresses below are the defaults; each has a flag if you need to move it.
--transport raw; --transport auto can fall back through the public :443 WebSocket path.-control address. Open 4443/udp; agents opt in with --transport quic.301 redirects to HTTPS. DNS-01 means no ACME traffic ever arrives here, so you can close it if you do not want the redirect.expvar, /healthz, and pprof. Loopback only — never open this port. Reach it over an SSH tunnel instead. An empty value disables it.$ sudo ufw allow 443/tcp$ sudo ufw allow 4443/tcp$ sudo ufw allow 80/tcp$ sudo ufw enableTCP tunnels#
Off by default. An operator opts in with -tcp-ports, which lets agents run ngstone tcp — see TCP tunnels for the client-side guide.
10000-10999. Empty disables TCP tunnels. The installer sets it with --tcp-ports MIN-MAX and opens the range in UFW for you.This range bypasses Traefik entirely
-tcp-ports are bound directly by ngstoned — real listeners, not routed through the :443 reverse proxy or any front proxy sitting in front of it. The range needs its own firewall opening, the same way 443/tcp does above:$ sudo ufw allow 10000:10999/tcpDatabase sharing needs none of this: shares ride the control port and open no listener. The edge knobs for it are -max-share-ttl (default 24h), the longest lifetime ngstone share --expires may ask for, and -share-audit-log PATH, which appends every share's audit events to a JSON Lines file on the VPS.
Behind an existing reverse proxy#
Skip this section if ngstoned owns :443 directly. If something else already holds that port, the edge still has to terminate its own TLS — it serves certificates it manages itself — so the proxy must pass the connection through at the TCP layer rather than terminating it. With Traefik that is an SNI-routed TCP router with tls.passthrough:
tcp: routers: ngstoned: entryPoints: - websecure rule: "HostSNIRegexp(`^.+\\.example\\.com$`)" tls: passthrough: true service: ngstoned services: ngstoned: loadBalancer: proxyProtocol: version: 2 servers: - address: "ngstoned:443"Passthrough is not transparent at the TCP layer. Traefik accepts the browser's connection and opens a new one to ngstoned, so without further configuration the edge sees the proxy's address as the client address — often a private one such as 10.0.1.69. Every X-Forwarded-For the edge stamps is then wrong, the local app cannot trust the forwarding headers, and the per-IP rate limiter collapses the entire internet into one shared bucket, so a single abusive client either escapes limiting or locks everyone out.
The fix is the PROXY protocol: the proxy prefixes each connection with the original endpoints and the edge uses them as the client address. That is the proxyProtocol.version: 2 block above, plus -proxy-protocol on the edge:
ExecStart=/usr/local/bin/ngstoned -proxy-protocol -public 127.0.0.1:443 ...Both sides must change together
-proxy-protocol on, the edge rejects any connection to the public listener that does not begin with a valid PROXY header; with it off, a proxy that sends one produces a TLS handshake failure. Enabling either side alone breaks every public request. Change the unit and the Traefik service in the same deployment.The public port must then be reachable only from the proxy
X-Forwarded-For, and rate-limits on. Bind the public listener to the proxy-facing interface only (-public 127.0.0.1:443, or a Docker network address) and make sure no firewall rule exposes it. Leaving the flag off — the default — preserves the normal behaviour exactly.If binding to a loopback/Docker-only address isn't possible, -proxy-protocol-trustedrestricts which peers the edge accepts a PROXY header from — it closes the connection immediately if the socket's real peer address isn't in the list, before even trying to parse a header:
ExecStart=/usr/local/bin/ngstoned -proxy-protocol -proxy-protocol-trusted 10.0.1.69/32 ...This also matters for per-tunnel --allow-cidr (see Access control): that check trusts whatever the edge believes is the source IP, and behind a proxy that's whatever the PROXY header says. Left empty, the edge accepts a PROXY header from any peer, so a direct client could forge one and spoof its way past an allowlist unless the public port is firewalled to the proxy alone.
The flag applies to the public listener only. Do not put the control listener on :4443behind a PROXY-protocol proxy: it does not parse the header and every agent connection would fail. Version 2 is Traefik's default when proxyProtocol is present at all; the edge also accepts the text-based version 1, and the LOCAL command used by health checks, falling back to the real socket address.
Connecting an agent#
On the agent machine, point ngstone at your edge and hand it the plaintext token. Dial the bare domain: the control port serves the apex certificate, and the agent verifies it against the host in --server.
$ ngstone auth "$TOKEN" --server example.com:4443verified against example.com:4443 and savedauth persists the token and server address to the config file, so later commands need no extra flags:
$ ngstone port 3000$ ngstone port 3000 --subdomain demoOlder agents
--server-name defaulted to the --server host verify against ngstone.site instead, and need --server-name example.com on every command.The agent prints the assigned URL. Verify it end to end from a third machine:
$ curl -i https://demo.example.com/HTTP/2 200Staging certificates fail curl
-acme-ca staging is in effect the chain is not in any trust store, so curl reports an unknown issuer. Use curl -k to confirm routing works, then promote to production for a certificate real clients accept.Troubleshooting#
Everything the edge does is logged to the journal:
$ journalctl -u ngstoned -f$ journalctl -u ngstoned -n 200 --no-pagerThe ops endpoint is loopback-only by design; reach it over SSH. /healthz reports the apex and wildcard expiry separately, which is the fastest way to tell a half-working certificate state from a healthy one:
$ ssh -L 4041:127.0.0.1:4041 root@example.com$ curl http://127.0.0.1:4041/healthz-dns-provider, and that the credential can edit this zone — for Cloudflare, Zone:DNS:Edit on it. A credential scoped to the wrong zone fails at the TXT-record write, which the log names explicitly.:4443 accepts connections and fails loudly if no apex certificate is loadable with 24 hours of life remaining. Fix issuance first; the wildcard being healthy proves nothing about the apex.-domain is required. The edge uses durable SQLite storage by default; -credentials-file and the legacy-token-sha256 mode remain available. TLS requires-cert-emailand the chosen DNS provider's credentials unless -insecure is set. Each missing value is named before exit; ngstoned -dns-provider help lists the variables each provider reads.echo instead of printf is the usual cause — and confirm the edge picked up the new value with a restart.4443 may be closed or the agent may be dialling the wrong host. With --transport auto, confirm that HTTPS 443 reaches the edge and that WebSockets are enabled; with --transport raw, confirm the raw firewall rule. Check the saved address with ngstone config get server.edge.example.com (which older installer versions printed) — use example.com:4443, and see Older agents above. It is also what a staging-issued apex certificate looks like from a client with a normal trust store.Restarting three times in a row should produce no new ACME traffic in the logs. If it does, the certificate cache is not persisting and the StateDirectory configuration is wrong.