- Many servers per proxy. Any number of servers (“nodes”) join a proxy. DNZ pushes the configuration to all of them the moment it changes, and DNS answers the proxy’s names with the addresses of the nodes that are up.
- Certificates from DNZ, issued over ACME with DNS-01 through DNZ’s own zones or a Cloudflare token per domain, or HTTP-01 answered by every node. See Certificates.
- Analytics and request logs for every proxied hostname.
- Caching, a firewall, and an “under attack” challenge as middlewares.
- TCP and UDP routing.
Proxying is off by default. Turn it on with Settings → Proxy → dnz-proxy, or
DNZ_PROXY_BACKEND=dnz. See Turning proxying on.Nodes
A node is a server running thednz-proxy binary. It ships with DNZ, in the same Docker
image as dnz. A node needs only DNZ’s HTTP address. It never has to be on the
same machine.
Running a node
Settings → Proxy → Set up dnz-proxy shows these commands filled in, for systemd, Docker and a plain command line.- Command line
- systemd
- Docker
Joining
A node joins with the connect key, in one of three ways:- Added by address. On the proxy’s Nodes tab, Add node with the address it calls DNZ from. A node calling from there joins at once.
- Approved. Any other node shows up as New node waiting on the Proxy page. Approve it into a proxy (check its public addresses), or ignore it. At most 20 wait at once.
- Automatically. A proxy with Let nodes join without approval (its Settings tab) takes every node that names it.
Configuration and failures
Each node keeps a stream open to DNZ (GET /api/proxy/dnz/stream). When anything the proxy serves
changes, DNZ rebuilds its configuration within a second and sends it to every node, which swaps it in
without dropping connections.
A node stores the last configuration on disk and keeps serving it while DNZ is away or restarting. It
reconnects by itself.
Nodes report every 5 seconds: the configuration they serve, their listeners, the health of each
upstream, cache use, warnings (a rule or middleware they couldn’t use), and their traffic.
DNS and health
With dnz-proxy, a proxied name is answered with the public addresses of the proxy’s healthy nodes and a short TTL, so a node that goes down leaves the answers quickly. The TTL is TTL of proxied answers on the proxy’s Settings tab, 60 seconds by default. A node is healthy while it reports (within 15 seconds) and, with Check the nodes’ ports from DNZ on, while DNZ can connect to its HTTPS (or HTTP) port.- If no node is healthy, DNS answers with all of them rather than with nothing.
- A proxy without nodes answers with its own addresses.
Managing nodes from the CLI
Running one on DNZ’s server
Settings → Proxy → Run dnz-proxy on this server starts a node as a child of DNZ, joined to the chosen proxy aslocal. DNZ uses the dnz-proxy next to its own binary, or on the PATH, or the
one DNZ_PROXY_BIN names. It restarts the node if it exits and shows its last log lines.
Ports 80 and 443 need root or
CAP_NET_BIND_SERVICE. The systemd unit and the Docker image allow
it.Entry points
A proxy’s nodes listen on its entry points, on the proxy’s Entry points tab. Each has a name, an address (:443 listens on every address, IPv6 and IPv4), and a protocol: http, https, tcp
or udp.
A proxy starts with web (:80, HTTP) and websecure (:443, HTTPS). Routers name entry points
as they do with Traefik. A router with TLS answers on HTTPS entry points, one without on HTTP ones.
Per entry point:
- PROXY protocol from: addresses or ranges of load balancers in front of the nodes that send a PROXY protocol header (v1 or v2). The visitor’s address comes from it.
- Trusted forwarders: peers whose
X-Forwarded-*headers are kept. From anyone else they are replaced. - Timeouts for idle connections and for reading a request.
Routing
Routers, services and middlewares work as described in Routers, services and middlewares. dnz-proxy reads them as Traefik v3 does:- Rules:
Host,HostRegexp,Path,PathPrefix,PathRegexp,Method,Header,HeaderRegexp,Query,QueryRegexp,ClientIPwith&&,||,!and parentheses. Priority is by the rule’s length unless set. - Services: load balancing (
wrr,p2c,hrw,leasttime), sticky cookies, active and passive health checks, weighted, mirroring (without request bodies) and failover services. HTTP/1.1, HTTP/2 and h2c upstreams, WebSocket and gRPC. - Headers to the upstream:
X-Forwarded-For,-Proto,-Host,-Port,-ServerandX-Real-Ip. - Certificates: HTTPS entry points pick the certificate by the name the visitor asks for (SNI), in this order: DNZ’s issued and uploaded certificates, the TLS certificates of the proxy, the default certificate, then a self-signed one.
Client certificates
Client certificates followclientAuth in the router’s TLS options: clientAuthType, and caFiles
as paths on the node or PEM text. CAs alone mean RequireAndVerifyClientCert, as in Traefik.
- The handshake asks for a certificate when the name the visitor asks for belongs to such a router, and checks that the visitor holds its key.
- Whether one is required and signed by the CAs is checked once the visit is routed. A visit that fails gets a 403 (Traefik ends the handshake instead), or a 421 when it came over a connection made for another name (HTTP/2 reuses connections), so the browser retries on one of its own.
- A name served by routers with different TLS options gets the
defaultoptions. - Routers without a
Hostrule get thedefaultoptions’ client certificate setting. - TCP routers that end TLS don’t ask for client certificates.
Middlewares it runs
redirectScheme, redirectRegex, addPrefix, stripPrefix(Regex), replacePath(Regex), encodedCharacters, chain, headers (custom, CORS, security), basicAuth, digestAuth, forwardAuth, ipAllowList, passTLSClientCert, rateLimit, inFlightReq, compress, contentType, grpcWeb, errors, retry, circuitBreaker, buffering.encodedCharacters
encodedCharacters
Answers 400 to paths with an encoded
/, \, ;, %, ? or # that isn’t allowed. The query
isn’t checked.passTLSClientCert
passTLSClientCert
Writes
X-Forwarded-Tls-Client-Cert (each certificate as base64 DER, joined with ,) and
X-Forwarded-Tls-Client-Cert-Info (URL-escaped Subject="…";Issuer="…";…) as Traefik does.
Copies of these headers sent by the visitor are dropped.grpcWeb
grpcWeb
Turns
application/grpc-web and application/grpc-web-text (base64) requests into gRPC, and the
answer back, with the trailers written into the body. CORS preflights from allowOrigins are
answered. The service needs an HTTP/2 server (h2c:// or https://).rateLimit
rateLimit
Counts per node in memory, or with
redis in Redis, shared by every node that uses it. One
endpoint is a single Redis, several are a cluster. User, password, database, timeouts and TLS
(CA, client certificate and key, insecureSkipVerify) work as in Traefik.Buckets are keyed by the middleware’s name and the client, so proxies sharing a Redis share the
counts of same-named middlewares. While Redis can’t be reached, each node counts on its own and
logs it.TCP and UDP
- TCP routers on
tcpentry points route byHostSNI(TLS passthrough, or TLS ended on the node with its certificates),HostSNI(`*`)for everything,ClientIPandALPN. - TCP services balance by weight, send a PROXY protocol header if asked, and may speak TLS to the server.
- TCP middlewares: ipAllowList and inFlightConn.
- UDP routers on
udpentry points send each client’s datagrams to one server of their service. Sessions end after 3 seconds without traffic.
dnz-proxy’s middlewares
These appear under dnz-proxy in Add middleware while dnz-proxy is the backend. Traefik never gets them.Cache (dnzCache)
Caches answers on each node.
- Answers are stored for as long as
Cache-Control(s-maxage,max-age) orExpiresallows, or for the default TTL when the origin says nothing. With Respect the origin off, the default TTL always applies. - Never stored:
no-store,private, answers that set cookies or vary on anything butAccept-Encoding, and statuses outside the list. - Visits with
Authorizationor one of the bypass cookies go to the origin. Prefixes in the style ofwordpress_logged_in*work.
- Stale while revalidate serves an expired answer while one visit fetches it again.
- Stale if error serves it while the origin fails.
X-Cache: HIT | MISS | STALE | BYPASS and Age.
Each node caches in memory (256 MiB by default) and, with a disk cache directory and size, on disk
too. Both are on the proxy’s Settings tab.
Purge cache on the proxy clears everything, a hostname, a URL or a path prefix on every node:
- Dashboard
- CLI
Proxy → open the proxy → Purge cache.
Firewall (dnzFirewall)
An ordered list of rules; the first that matches decides.
A rule has conditions, all or any of which must hold, on the client’s address or range, country,
host, path, method, a header, a query parameter or the user agent. Each condition uses is, is
not, in range, contains, starts with or matches (a regular expression).
Countries need a GeoIP database. Set GeoIP database URL on the proxy’s Settings tab to an
.mmdb (or .mmdb.gz) file, such as MaxMind GeoLite2 Country or DB-IP’s IP to Country Lite. Each
node downloads it.
Under attack (dnzChallenge)
Browsers get a short page that solves a small proof of work. A cookie, signed for the visitor’s
network and valid for Valid for, then lets them through.
- Bots can be let through by user agent and by address.
- Requests that aren’t a browser page (APIs, forms) get a 403 until a page has been visited.
- Difficulty is in bits: each one doubles the work. 16 takes well under a second.
Analytics
Proxy → Analytics shows, per proxy and hostname, over the last hour to 30 days:- requests by status class, bandwidth, and latency (p50, p95, p99)
- cache hit ratio, blocked and challenged visits
- top hosts, paths, clients, countries, user agents and statuses
- a request log with filters and a live view
Troubleshooting
A node doesn't show up
A node doesn't show up
Check that it can reach DNZ at
--dnz and has the current connect key. A node that calls from an
address no node was added for waits under New node waiting on the Proxy page. At most 20
wait at once. With a self-signed certificate on DNZ, add --insecure.Nodes stop getting changes
Nodes stop getting changes
The configuration stream goes through
GET /api/proxy/dnz/stream. A reverse proxy in front of
DNZ that buffers responses holds it back. Turn buffering off for DNZ.DNS still answers with a node that is down
DNS still answers with a node that is down
A node leaves the answers once it stops reporting for 15 seconds, or once DNZ can’t connect to
its port when Check the nodes’ ports from DNZ is on. Resolvers may keep the old answer for the
TTL of proxied answers. If no node is healthy, DNS answers with all of them.
A router is off with a warning on the node
A router is off with a warning on the node
It uses something the node couldn’t run, such as a plugin middleware. The node’s report lists
the warning.
The local node doesn't start
The local node doesn't start
DNZ looks for
dnz-proxy next to its own binary, then on the PATH; set DNZ_PROXY_BIN to
point at it. Ports 80 and 443 need root or CAP_NET_BIND_SERVICE. DNZ shows the node’s last log
lines.Next steps
Certificates
Let DNZ issue and renew certificates for the nodes.
dnz-proxy API
Nodes, entry points, analytics and purges over HTTP.