DocsGuidesRunning the edge on a VPS

Running the edge on a VPS

ngstoned is the half of ngstone that lives on the public internet. This is a runbook: work top to bottom on a fresh Linux server and you end with a managed service holding valid certificates, an open control plane, and an agent tunnelling through it. example.com stands in for your domain throughout.

Fast path installer#

On a fresh Ubuntu or Debian VPS, create the apex and wildcard DNS records first and create an API credential for your DNS provider, then install the edge from a checkout. The installer downloads both release binaries, verifies their checksums, creates the root-only secrets file, installs the hardened systemd unit, configures UFW, and starts ngstoned:

$ git clone https://github.com/emilio-kariuki/ngstone.git /opt/ngstone
$ cd /opt/ngstone
$ sudo ./deploy/install-edge.sh \
--domain example.com \
--email you@example.com \
--dns-provider cloudflare \
--acme-ca staging

It prompts for the chosen provider's credentials without echoing them, initializes durable SQLite storage, and prints a scoped agent credential once. Set NGSTONE_DATABASE_URL before running it to use PostgreSQL. The installer does not create DNS records, so the apex and wildcard must already resolve to the VPS. Once the health output shows both certificates, switch to trusted certificates:

$ sudo ./deploy/install-edge.sh --acme-ca production

A rerun with only --acme-ca production keeps the domain, email, DNS provider, credentials, TCP port range, and auth token from the existing installation. The installer then prints the laptop commands, which connect to the bare domain, example.com:4443.

--domainstringrequired
Public base domain. Tunnels get hostnames one label below it.
--emailstringrequired
Let's Encrypt account email.
--dns-providerstringdefault: cloudflare
DNS host for the certificate challenges; see Choosing a DNS provider. The installer prompts for that provider's credentials.
--acme-castringdefault: staging
staging or production. Start with staging, then promote; neither browsers nor the agent trust staging certificates. A rerun keeps the current value.
--tcp-portsMIN-MAX
Enable raw TCP tunnels on this range, e.g. 10000-10100. Adds -tcp-ports to the unit and opens the range in UFW; the range is kept across reruns and --tcp-ports off removes it. It must lie within 1024-65535 and must not include 4443.
--quicon|offdefault: off
Run ngstoned with -quic so agents can use --transport quic. Opens 4443/udp in UFW and writes /etc/sysctl.d/99-ngstone-quic.conf with larger UDP socket buffers (net.core.rmem_max and net.core.wmem_max of 7500000). A key whose live value already exceeds that gets no entry, and no file is written if the installer has not written one before and both exceed it. Once the file exists, a rerun keeps its keys at 7500000. The installer never lowers a live value, but at boot the file overrides any earlier-sorting drop-in for the same key, though /etc/sysctl.conf or 99-sysctl.conf can still override it; edit or remove its key to keep a larger persistent value. off removes the flag, the rule and the file.
--from-sourcebool
Build ngstoned from the checkout with Go instead of downloading a release. Needed for providers other than Cloudflare until a release that includes them is published; the installer refuses to install a binary that does not support the chosen provider.
--versionstring
Release to download, for example v0.3.2.
--ssh-portintdefault: 22
SSH port to keep open in UFW.
--skip-firewallbool
Do not configure UFW.
--skip-dns-checkbool
Do not verify that the apex and wildcard records resolve first.

Passing credentials without a prompt

sudo drops variables exported in your shell, so put them on the sudo command line instead: sudo NGSTONE_CF_API_TOKEN=... ./deploy/install-edge.sh .... Either way they are stored only in the root-only /etc/ngstoned/env.

Uninstalling#

From the checkout, remove the service, binary, and root-only secrets:

$ sudo ./deploy/uninstall.sh

Certificate, ACME, and default SQLite control-plane state is preserved by default under /var/lib/private/ngstoned, so a later reinstall can reuse it. To delete that state and remove matching public UFW allow rules, opt in explicitly:

$ sudo ./deploy/uninstall.sh --remove-state --remove-firewall-rules

SSH access is kept

The firewall option removes matching allow rules for only 80/tcp, 443/tcp, 4443/tcp, 4443/udp if QUIC was enabled, and the TCP tunnel range if one is configured. SSH rules and the source checkout are left in place. If another service uses the same allow rule, review it before selecting this option. Use --yes only for a deliberate non-interactive uninstall.

Running it with Docker#

Instead of the installer, run the ghcr.io/emilio-kariuki/ngstoned image (amd64 and arm64) on any host with Docker and the Compose plugin. The DNS records, the provider credential, and the open ports (80, 443, 4443) are the same; the installer's UFW and DNS checks are yours to do. Fetch the Compose file and the environment template, fill in .env, and start it:

$ mkdir ngstone && cd ngstone
$ curl -fsSLO https://raw.githubusercontent.com/emilio-kariuki/ngstone/main/deploy/docker/compose.yaml
$ curl -fsSL -o .env https://raw.githubusercontent.com/emilio-kariuki/ngstone/main/deploy/docker/.env.example
$ chmod 600 .env && $EDITOR .env
$ docker compose up -d
$ docker compose ps # STATUS turns "healthy" once both certificates load
$ docker compose logs -f ngstoned

.env holds the domain, email, DNS provider and its credentials, and the token hash. Compose passes the secrets to the edge as environment variables only, never as flags. When the container reports healthy, set NGSTONE_ACME_CA=production in .env and run docker compose up -d again. Certificates live in the ngstoned-data volume, so restarts and docker compose pull upgrades reuse them.

Portscontainer
The edge listens on 8080, 8443 and 4443 inside the container, so it needs no added capabilities; the host publishes them as 80, 443 and 4443. The ops endpoint stays on the container's loopback, where the image's health check (ngstoned healthcheck) probes /healthz.
Client IPscontainer
Published ports keep the client's source address, so per-IP limits and --allow-cidr work without -proxy-protocol. IPv6 clients are the exception unless IPv6 is enabled on the Compose network.
Storagecontainer
The image runs as uid and gid 65532. The named volume is writable by it; a bind mount in its place must be chowned to 65532:65532 first.
TCP tunnelscontainer
Uncomment -tcp-ports and the matching ports range in compose.yaml; the two ranges must be identical.

Prerequisites#

  1. 1

    A Linux VPS with a public IP

    Anything that runs a static Go binary. It needs a stable public address — every DNS record below points at it — and root or sudo for installing a systemd unit and binding privileged ports.
  2. 2

    A domain you control

    A registered domain, here example.com. Tunnels get hostnames one label below it, such as amber-fox-42.example.com.
  3. 3

    The zone hosted on a supported DNS provider

    Certificates are issued through the ACME DNS-01 challenge, so ngstonedneeds API access to the zone. The domain's nameservers must point at one of the supported providers and the zone must be active there before anything below will work.
  4. 4

    A machine to run the agent

    Your laptop, with the ngstone binary installed — see Installation.

DNS records#

Create two records in the example.comzone, both pointing at the VPS's public IP.

example.comA recordrequired
The apex. Backs the control plane on port 4443, which is what the agent dials, and the apex certificate that secures it.
*.example.comA recordrequired
The wildcard. This is what makes per-tunnel hostnames resolve: the edge invents names like amber-fox-42.example.com at bind time and never touches DNS, so every possible tunnel name has to already resolve to the VPS through this one record.

Use AAAA instead if the VPS is IPv6-only, or add both.

DNS-only is simplest; auto has a :443 fallback

Cloudflare's orange-cloud proxy cannot carry the raw agent control connection on :4443, but the default --transport auto can fall back to the authenticated WebSocket control endpoint on HTTPS :443 when WebSockets are enabled. Use DNS-only records when forcing --transport raw or when you want the simplest path. --transport quic needs UDP on 4443 and cannot pass through the proxy.

Choosing a DNS provider#

The edge needs to create and delete _acme-challenge TXT records in the one zone. Pick the provider that hosts the zone with -dns-provider (default cloudflare) and create a credential scoped as tightly as that provider allows. Credentials are read from the environment only; ngstoned -dns-provider help lists the providers and the variables of the binary you have.

cloudflareNGSTONE_CF_API_TOKEN
The default. API token with Zone / DNS / Edit on this zone only. See below.
route53NGSTONE_AWS_REGION, NGSTONE_AWS_ACCESS_KEY_ID, NGSTONE_AWS_SECRET_ACCESS_KEY
An IAM user whose policy allows only route53:ListHostedZonesByName, route53:ListResourceRecordSets, route53:ChangeResourceRecordSets, and route53:GetChange.
digitaloceanNGSTONE_DO_API_TOKEN
Personal access token with domain read and write scopes.
hetznerNGSTONE_HETZNER_API_TOKEN
Read and write API token of the Hetzner Console project that holds the zone. Zones still on the deprecated dns.hetzner.com must be migrated first.
vultrNGSTONE_VULTR_API_KEY
API key; allow only the VPS's IP in the key's access control.
ovhNGSTONE_OVH_ENDPOINT, NGSTONE_OVH_APPLICATION_KEY, NGSTONE_OVH_APPLICATION_SECRET, NGSTONE_OVH_CONSUMER_KEY
The endpoint is ovh-eu, ovh-ca, or ovh-us. Limit the consumer key to GET, POST, and DELETE on /domain/zone/*.
desecNGSTONE_DESEC_TOKEN
Token from desec.io. deSEC enforces a 3600 s minimum TTL and rate limits, so issuance can take a few minutes.
bunnyNGSTONE_BUNNY_API_KEY
Bunny DNS account API key.

GoDaddy, Namecheap, Porkbun and others are not supported

GoDaddy, Namecheap, Porkbun, IONOS, Azure, DNSimple, and netcup are left out deliberately: their libdns modules replace or delete every TXT value at a name, and the apex and wildcard certificates publish two values at _acme-challenge.example.com at the same time, so one issuance would break the other. Keep the registrar and move only the DNS hosting to a supported provider by changing the nameservers — Cloudflare and deSEC are free.

The Cloudflare API token#

The default provider, and the example used below. Create a scoped API token for exactly the challenge records — never the Global API Key, which authenticates everything on the account.

  1. 1

    Open the token editor

    Cloudflare dashboard → My Profile → API Tokens → Create Token → Custom Token.
  2. 2

    Grant one permission

    Zone / DNS / Edit. Nothing else.
  3. 3

    Scope it to one zone

    Zone Resources: Include / Specific zone / example.com. Not “All zones”.
  4. 4

    Keep the value

    Cloudflare shows the token once. It goes into the edge's environment file as NGSTONE_CF_API_TOKEN below.

A token scoped this way can write challenge records in one zone and nothing more — it cannot read other zones, change nameservers, or touch account settings. If it leaks, the blast radius is one zone's DNS.

Two certificates, not one#

ngstoned calls ManageSync(ctx, ["*.example.com", "example.com"]) once, which reads like a single multi-domain certificate. It is not. certmagic does not obtain multi-SAN certificates: that call places two independent ACME orders and produces two certificates with two independent renewal schedules.

This is forced anyway, because a wildcard covers exactly one left-most label (RFC 6125) — *.example.com does not match SNI example.com. The wildcard serves public tunnel traffic; the apex certificate serves the WebSocket fallback on :443 and the raw control plane on :4443. Both names validate through the same _acme-challenge.example.com TXT record, which is why only providers that add and remove challenge values individually are supported.

A healthy wildcard can hide a dead apex

If the apex order fails while the wildcard renews fine, public traffic looks perfect and no agent can connect. ngstoned checks at startup that an apex certificate is loadable with at least 24 hours of life left before accepting on :4443, and exits loudly rather than listening with a certificate it cannot serve. /healthzreports each certificate's expiry separately — check both, not just one.

Durable storage and scoped tunnel credentials#

Issue one credential per tunnel

SQLite is the default control-plane store. It persists tunnel names, ownership, credentials, connector leases, audit events, traffic policies, and custom-domain registrations. The edge stores only a hash of each scoped credential; the plaintext is printed once by the management API and never stored.
$ ngstone admin credentials issue --server http://127.0.0.1:4041 \
$ --subdomain demo --ttl 24h
credential <id> issued
token: <save this value on the laptop>

Copy the printed token to the laptop and run ngstone auth, then use ngstone port 3000 --subdomain demo. Set NGSTONE_DATABASE_URL=sqlite:///var/lib/ngstoned/control.dbin the unit's environment. For active/active edges, use the same PostgreSQL URL on every edge and a different NGSTONE_EDGE_ID. Forwarding a connection between edges also needs NGSTONE_PEER_ADDR.

The edge reads its storage settings, token hash, ACME email, and DNS provider's credentials from its environment. Put them in /etc/ngstoned/env, mode 0600, owned by root — for Cloudflare:

/etc/ngstoned/envini
NGSTONE_DATABASE_URL=sqlite:///var/lib/ngstoned/control.db
NGSTONE_EDGE_ID=edge-a
NGSTONE_CERT_EMAIL=you@example.com
NGSTONE_CF_API_TOKEN=cf_fake_token_value_goes_here

For active/active edges, point every edge at the same PostgreSQL URL and add NGSTONE_PEER_ADDR; the edge refuses to start with it on SQLite:

/etc/ngstoned/envini
NGSTONE_DATABASE_URL=postgres://ngstone:<password>@db.internal/ngstone?sslmode=require
NGSTONE_EDGE_ID=edge-a
NGSTONE_PEER_ADDR=10.0.0.11:4043
NGSTONE_DATABASE_URLenvrequired
SQLite or PostgreSQL control-plane URL. Use a shared PostgreSQL URL for multiple edge servers.
NGSTONE_EDGE_IDenvrequired
Stable identity for this edge in connector leases and audit events.
NGSTONE_PEER_ADDRenv
Private <ip>:<port> of this edge, set on every edge that shares the PostgreSQL database. It must be an IP address literal and a port; hostnames and unspecified addresses are rejected. Requires PostgreSQL and is refused with -insecure. Equivalent flag -peer-addr. With it set, any edge forwards a tunnel's HTTPS, TLS-passthrough or raw TCP connection once to the edge holding its live connector, and that edge enforces every access rule using the real client address. Keep the port reachable only from the other edges. Empty disables forwarding.
NGSTONE_PEER_LISTENenv
Address (<ip>:<port> or :<port>) on which this edge accepts peer connections when that differs from NGSTONE_PEER_ADDR, for example inside a container. Requires NGSTONE_PEER_ADDR, which stays the address the other edges dial. Equivalent flag -peer-listen. Defaults to NGSTONE_PEER_ADDR.
NGSTONE_CREDENTIALS_FILEenv
Compatibility path to the versioned scoped credential file.
NGSTONE_TOKEN_SHA256env
Legacy shared-token SHA-256. Equivalent flag -token-sha256; use it only while migrating older installations.
NGSTONE_CF_API_TOKENenvrequired
The scoped Cloudflare token. Equivalent legacy flag -cf-api-token; the environment wins when both are set. Other providers read the variables listed under Choosing a DNS provider instead, and have no flags at all.
NGSTONE_CERT_EMAILenvrequired
Not read by ngstoned directly — the unit interpolates it into -cert-email, which is required unless -insecure. It is the ACME account contact, not a secret.

Never pass secrets as ExecStart flags

/proc/<pid>/cmdline is world-readable, so any local user can read a flag value with ps auxww — but not the process environment. That is precisely why Secrets belong in the environment file. Explicit flags override their environment defaults; other DNS providers' credentials are available only through environment variables. Keep secrets out of ExecStart.

Installing the binary#

Release archives contain both binaries — ngstone and ngstoned. On the VPS, download the archive matching its architecture, verify the checksum, and install the edge binary:

$ VERSION=v0.2.0
$ ARCHIVE=ngstone_linux_amd64.tar.gz # ngstone_linux_arm64.tar.gz on arm64
$ BASE=https://github.com/emilio-kariuki/ngstone/releases/download/$VERSION
$ curl -fsSLO "$BASE/$ARCHIVE"
$ curl -fsSLO "$BASE/checksums.txt"
$ sha256sum --ignore-missing -c checksums.txt
$ tar -xzf "$ARCHIVE" ngstone ngstoned
$ sudo install -m 0755 ngstone /usr/local/bin/ngstone
$ sudo install -m 0755 ngstoned /usr/local/bin/ngstoned
$ ngstoned -version

Running it under systemd#

Install the unit, create the environment file from the section above, and start the service:

/etc/systemd/system/ngstoned.serviceini
[Unit]
Description=ngstoned edge server
After=network-online.target
Wants=network-online.target
 
[Service]
Type=simple
ExecStart=/usr/local/bin/ngstoned \
-domain example.com \
-cert-email ${NGSTONE_CERT_EMAIL} \
-acme-ca staging \
-cert-storage ${STATE_DIRECTORY}/certs
EnvironmentFile=/etc/ngstoned/env
Restart=always
RestartSec=2s
 
StateDirectory=ngstoned
StateDirectoryMode=0700
 
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictSUIDSGID=yes
RestrictRealtime=yes
LockPersonality=yes
 
DynamicUser=yes
 
LimitNOFILE=65536
 
[Install]
WantedBy=multi-user.target
$ sudo mkdir -p /etc/ngstoned && sudo chmod 700 /etc/ngstoned
$ sudo $EDITOR /etc/ngstoned/env
$ sudo chmod 600 /etc/ngstoned/env
$ sudo systemctl daemon-reload
$ sudo systemctl enable --now ngstoned
$ systemctl status ngstoned

DynamicUser=yes gives the service a transient, unprivileged UID allocated fresh on every start, and AmbientCapabilities=CAP_NET_BIND_SERVICE lets that unprivileged user still bind :80 and :443. The certificate cache has to survive across those changing UIDs, which is what StateDirectory=ngstoned is for: systemd creates /var/lib/private/ngstoned (with a /var/lib/ngstoned symlink) and re-chowns it to the current UID on every start. Point -cert-storage at ${STATE_DIRECTORY}/certs so the two always agree.

-domain is required and has no default. For a DNS host other than Cloudflare, add -dns-provider NAME to ExecStartand put that provider's variables in the environment file.

Do not substitute ReadWritePaths= for StateDirectory=

ReadWritePaths= gets no ownership handling from systemd, so a freshly assigned dynamic user cannot read the existing cache. The certificates are silently lost and re-issued on every restart, burning Let's Encrypt rate limits until issuance starts failing.

-acme-ca defaults to staging and the unit above keeps it there deliberately. Staging certificates are not trusted by browsers but exercise the identical code path, so a misconfiguration costs nothing. Once systemctl status ngstoned is clean and /healthz reports sane expiry dates for both certificates, switch the unit to -acme-ca production and restart. Do not delete the staging certificates first — certmagic namespaces storage per issuer, so the two live in separate subtrees and cannot collide.

Firewall#

Four listeners, three of them public. The addresses below are the defaults; each has a flag if you need to move it.

-publicstringdefault: :443
Public HTTPS. Must be reachable from the internet — this is where tunnel traffic lands.
-controlstringdefault: :4443
Raw control plane. Must be reachable from wherever agents run when using --transport raw; --transport auto can fall back through the public :443 WebSocket path.
-quicbooldefault: false
Also accept agent control connections over QUIC on UDP at the -control address. Open 4443/udp; agents opt in with --transport quic.
-httpstringdefault: :80
Plain HTTP, which only issues 301 redirects to HTTPS. DNS-01 means no ACME traffic ever arrives here, so you can close it if you do not want the redirect.
-opsstringdefault: 127.0.0.1:4041
expvar, /healthz, and pprof. Loopback only — never open this port. Reach it over an SSH tunnel instead. An empty value disables it.
$ sudo ufw allow 443/tcp
$ sudo ufw allow 4443/tcp
$ sudo ufw allow 80/tcp
$ sudo ufw enable

TCP tunnels#

Off by default. An operator opts in with -tcp-ports, which lets agents run ngstone tcp — see TCP tunnels for the client-side guide.

-tcp-portsstring
Port range for raw TCP tunnels, e.g. 10000-10999. Empty disables TCP tunnels. The installer sets it with --tcp-ports MIN-MAX and opens the range in UFW for you.
-tcp-bindstringdefault: 0.0.0.0
Interface address the TCP tunnel port range binds on.

This range bypasses Traefik entirely

Ports in -tcp-ports are bound directly by ngstoned — real listeners, not routed through the :443 reverse proxy or any front proxy sitting in front of it. The range needs its own firewall opening, the same way 443/tcp does above:
$ sudo ufw allow 10000:10999/tcp

Database sharing needs none of this: shares ride the control port and open no listener. The edge knobs for it are -max-share-ttl (default 24h), the longest lifetime ngstone share --expires may ask for, and -share-audit-log PATH, which appends every share's audit events to a JSON Lines file on the VPS.

Behind an existing reverse proxy#

Skip this section if ngstoned owns :443 directly. If something else already holds that port, the edge still has to terminate its own TLS — it serves certificates it manages itself — so the proxy must pass the connection through at the TCP layer rather than terminating it. With Traefik that is an SNI-routed TCP router with tls.passthrough:

traefik dynamic configurationyaml
tcp:
routers:
ngstoned:
entryPoints:
- websecure
rule: "HostSNIRegexp(`^.+\\.example\\.com$`)"
tls:
passthrough: true
service: ngstoned
services:
ngstoned:
loadBalancer:
proxyProtocol:
version: 2
servers:
- address: "ngstoned:443"

Passthrough is not transparent at the TCP layer. Traefik accepts the browser's connection and opens a new one to ngstoned, so without further configuration the edge sees the proxy's address as the client address — often a private one such as 10.0.1.69. Every X-Forwarded-For the edge stamps is then wrong, the local app cannot trust the forwarding headers, and the per-IP rate limiter collapses the entire internet into one shared bucket, so a single abusive client either escapes limiting or locks everyone out.

The fix is the PROXY protocol: the proxy prefixes each connection with the original endpoints and the edge uses them as the client address. That is the proxyProtocol.version: 2 block above, plus -proxy-protocol on the edge:

ExecStart=/usr/local/bin/ngstoned -proxy-protocol -public 127.0.0.1:443 ...

Both sides must change together

With -proxy-protocol on, the edge rejects any connection to the public listener that does not begin with a valid PROXY header; with it off, a proxy that sends one produces a TLS handshake failure. Enabling either side alone breaks every public request. Change the unit and the Traefik service in the same deployment.

The public port must then be reachable only from the proxy

Whoever can open a TCP connection to that port chooses the client IP the edge logs, stamps into X-Forwarded-For, and rate-limits on. Bind the public listener to the proxy-facing interface only (-public 127.0.0.1:443, or a Docker network address) and make sure no firewall rule exposes it. Leaving the flag off — the default — preserves the normal behaviour exactly.

If binding to a loopback/Docker-only address isn't possible, -proxy-protocol-trustedrestricts which peers the edge accepts a PROXY header from — it closes the connection immediately if the socket's real peer address isn't in the list, before even trying to parse a header:

ExecStart=/usr/local/bin/ngstoned -proxy-protocol -proxy-protocol-trusted 10.0.1.69/32 ...

This also matters for per-tunnel --allow-cidr (see Access control): that check trusts whatever the edge believes is the source IP, and behind a proxy that's whatever the PROXY header says. Left empty, the edge accepts a PROXY header from any peer, so a direct client could forge one and spoof its way past an allowlist unless the public port is firewalled to the proxy alone.

The flag applies to the public listener only. Do not put the control listener on :4443behind a PROXY-protocol proxy: it does not parse the header and every agent connection would fail. Version 2 is Traefik's default when proxyProtocol is present at all; the edge also accepts the text-based version 1, and the LOCAL command used by health checks, falling back to the real socket address.

Connecting an agent#

On the agent machine, point ngstone at your edge and hand it the plaintext token. Dial the bare domain: the control port serves the apex certificate, and the agent verifies it against the host in --server.

$ ngstone auth "$TOKEN" --server example.com:4443
verified against example.com:4443 and saved

auth persists the token and server address to the config file, so later commands need no extra flags:

$ ngstone port 3000
$ ngstone port 3000 --subdomain demo

Older agents

Agent releases from before --server-name defaulted to the --server host verify against ngstone.site instead, and need --server-name example.com on every command.

The agent prints the assigned URL. Verify it end to end from a third machine:

$ curl -i https://demo.example.com/
HTTP/2 200

Staging certificates fail curl

While -acme-ca staging is in effect the chain is not in any trust store, so curl reports an unknown issuer. Use curl -k to confirm routing works, then promote to production for a certificate real clients accept.

Troubleshooting#

Everything the edge does is logged to the journal:

$ journalctl -u ngstoned -f
$ journalctl -u ngstoned -n 200 --no-pager

The ops endpoint is loopback-only by design; reach it over SSH. /healthz reports the apex and wildcard expiry separately, which is the fastest way to tell a half-working certificate state from a healthy one:

$ ssh -L 4041:127.0.0.1:4041 root@example.com
$ curl http://127.0.0.1:4041/healthz
Certificate issuance failsstartup
Check that both DNS records exist and have propagated, that the zone is actually served by the provider named in -dns-provider, and that the credential can edit this zone — for Cloudflare, Zone:DNS:Edit on it. A credential scoped to the wrong zone fails at the TXT-record write, which the log names explicitly.
The service exits at startupstartup
The apex certificate check runs before :4443 accepts connections and fails loudly if no apex certificate is loadable with 24 hours of life remaining. Fix issuance first; the wildcard being healthy proves nothing about the apex.
ngstoned refuses to start with no ACME trafficconfig
-domain is required. The edge uses durable SQLite storage by default; -credentials-file and the legacy-token-sha256 mode remain available. TLS requires-cert-emailand the chosen DNS provider's credentials unless -insecure is set. Each missing value is named before exit; ngstoned -dns-provider help lists the variables each provider reads.
rejected this tokenagent
The agent hashed a different string than the edge was given. Re-derive the hash from the exact plaintext — a trailing newline from echo instead of printf is the usual cause — and confirm the edge picked up the new value with a restart.
could not reach ...agent
Port 4443 may be closed or the agent may be dialling the wrong host. With --transport auto, confirm that HTTPS 443 reaches the edge and that WebSockets are enabled; with --transport raw, confirm the raw firewall rule. Check the saved address with ngstone config get server.
TLS handshake failures from the agentagent
Usually the agent is dialling a name the apex certificate does not cover, such as edge.example.com (which older installer versions printed) — use example.com:4443, and see Older agents above. It is also what a staging-issued apex certificate looks like from a client with a normal trust store.

Restarting three times in a row should produce no new ACME traffic in the logs. If it does, the certificate cache is not persisting and the StateDirectory configuration is wrong.