05 – Configuration

Principles

• One declarative YAML file is the source of truth. Discovery adapters and the admin API only supplement the backend lists and states at runtime.
• Validate before activating. A faulty config is rejected; the running one stays active.
• Hot reload via SIGHUP or file watch. Listeners are reconciled by name: an added listener is spawned, a removed one is stopped, and one whose definition changed (bind, protocol, routes, …) is stopped and re-spawned. A listener whose definition is unchanged keeps running untouched. backend_sources discovery refresh tasks are reconciled the same way (added / removed / re-parameterised → started / stopped / restarted).
• Environment-variable interpolation (${VAR}) for secrets/tokens.

Schema (reference)

Implemented subset (roadmap phase 2 + phase 3 routing, partial). wayhouse-config currently accepts a reduced, flatter schema: pools[].targets or a pools[].source naming a backend_sources[] entry (static / dns_srv / consul / kubernetes / tunnel, flat fields, refresh_interval_sec); balancer: round_robin | least_conn | consistent_hash | weighted (scalar, not an object) — consistent_hash also reads a pool-level hash_on: src_ip | src_ip_port (default src_ip), and weighted a pool-level weights: { "ip:port": N } map (weight >= 1, default 1), each rejected on the other balancers; health_check.type: tcp_connect | udp_probe | none with send_hex / expect_hex_prefix for udp_probe (see “Health checks” below); per_backend.max_sessions; and proxy_protocol: none | v1 | v2 | v2-udp (scalar). v1/v2 are TCP-only, v2-udp UDP-only; the form must match the transport of the listeners that statically route to the pool (validated).

A listener maps to a pool either with the pool: <name> shorthand or with a priority-ordered routes: list — [{ match, action }], first match wins, action: { pool: <name> } — but not both. Implemented matchers: match: { type: always }, { type: client_cidr, cidrs: [<prefix>, ...] } (source IP), { type: dst, cidrs: [<prefix>, ...] } (destination IP the client connected to; a plain listener sees only its bind IP, but a UDP listener with prefix: <cidr> serves a whole routed prefix on one IP_PKTINFO socket and replies from the address that was hit), { type: port, ports: [<int> | "lo-hi", ...] } (destination port from the accepting socket), { type: first_bytes, prefix: "hex:ff.." | "ascii:..", length: { min, max } } — a ≤ 512-byte prefix and/or a byte-count window on the connection’s first bytes (TCP MSG_PEEK / first UDP datagram; at least one of prefix / length), and { type: sni, host: ["exact", "*.suffix", ".suffix"] } — the server_name from the peeked (not terminated) TLS ClientHello, TCP listeners only. The { type: sniffer, sniffer: <name>, host: [...] } matcher parses (a listener may use several sniffers, see docs/03-routing.md) but matches nothing until a sniffer is loaded — no sniffers are built in; the name is checked at listener start, not by validate(). The sniffer loader is Phase 9; regex-over-first-bytes is a Phase 9 sniffer concern, not a first_bytes sub-form.

External resolver (phase 4, slices 1–4): a top-level resolvers: list of { name, type: http|grpc, endpoint, timeout_ms, on_error: reject|fallback_route|stale_ok, proxy_protocol?, target_connect_timeout_ms?, target_idle_timeout_sec?, cache? } and a route action: { resolver: <name> } (exactly one of pool / resolver per action). The proxy POSTs {listener, src, dst, sni?, first_bytes_b64, routing_key?} and expects {pool?, target?, ttl_sec?} — target (“ip:port”) wins over pool and connects straight to that instance (no pool / health / cap). proxy_protocol: none | v1 | v2 | v2-udp (default none) is the PROXY protocol header to prepend to a target connection — there is no pool to read it from — so the backend still sees the real client IP; a resolver-chosen pool uses that pool’s own proxy_protocol. Same transport rule as pools: v1/v2 need TCP listeners on that resolver’s routes, v2-udp needs UDP. target_connect_timeout_ms (default 300) / target_idle_timeout_sec (default 90) are the connect / idle timeouts for a target connection or UDP session — again, no pool to read them from; a resolver-chosen pool uses its own. fallback_route continues the route list on failure; reject drops; stale_ok serves the last (expired) cached answer if there is one, else drops. Optional cache: { key: [<part>, ...], positive_ttl_sec, negative_ttl_sec, max_entries } — a TTL’d LRU keyed by the joined parts (src_ip / src_ip_port / sni / routing_key / first_bytes:a:b); a response ttl_sec overrides positive_ttl_sec; a request missing a key part bypasses the cache. For type: grpc the endpoint is an http://host:port URI and the contract is proto/resolver.proto (wayhouse.resolver.v1.Resolver/Resolve).

Push resolver: POST /route-hint (admin API) with { "src_ip": "...", "pool": "...", "ttl_sec": 30 } records a short-lived src_ip → pool hint; a listener with route_hint: true applies it before its route list. pool must exist; ttl_sec 1..=3600.

Listener options: prefix: <cidr> (UDP only) → prefix mode as above, with a wildcard bind; freebind: true (TCP only) → bind with IP_FREEBIND / IPV6_FREEBIND; transparent: true (TCP or UDP, Linux) → TPROXY mode: IP_TRANSPARENT on the listen socket, the original destination read per connection / datagram, and the real client address bound as the upstream source (needs CAP_NET_ADMIN; mutually exclusive with prefix); route_hint: true → consult the push resolver.

Filter chain (phase 7): allow: [<cidr>, …] / deny: [<cidr>, …] on a listener (TCP or UDP) are checked against the client source IP before routing. deny is checked first and wins; a non-empty allow makes the listener default-deny for any source it does not cover. A blocked connection / new UDP session is dropped silently (no error reply — no reflection) and counted by wayhouse_filter_blocked_total{listener,filter="acl"}. Established UDP sessions are not re-checked per datagram. allow / deny are compiled to a radix trie, so large block lists (bogons + a threat feed) match in bounded time.

rate_limit: { per_ip: { rate, burst }, per_net: { rate, burst } } (at least one of per_ip / per_net) is a token bucket on new connections / new UDP sessions, checked right after the ACL. rate is permits/second sustained, burst the bucket capacity (defaults to rate). per_net aggregates by the client’s /24 (IPv4) or /64 (IPv6). A permit is taken only when every configured bucket can afford it; excess is dropped silently and counted by wayhouse_filter_blocked_total{listener,filter="rate_ip"|"rate_net"}. Bucket state is per proxy instance (size it per node behind anycast HA). Each table is capped at 200 000 buckets (one for IPs, one for networks) to bound memory; idle buckets are pruned first, but if a flood of distinct sources (an IPv6 /64 sweep, spoofed UDP) fills a table with live buckets, a source with no bucket yet is refused (fail closed) until some expire, which can also turn away legitimate new clients while the flood lasts. Those refusals are counted as rate_ip / rate_net like any other over-rate drop, and the proxy logs a rate limiter table is full warning (at most once a minute per listener and table) so the two can be told apart.

Per-source concurrent cap: per_source: { max_per_ip, max_per_net } (at least one) bounds how many connections / UDP sessions are live at once from one client IP / one /24 (v4) / /64 (v6) — where rate_limit bounds the rate of new ones. Checked after rate_limit, before allocation; over the cap ⇒ silent drop + wayhouse_filter_blocked_total{listener,filter="src_conn_ip"|"src_conn_net"}. The slot is released when the connection closes / the UDP session idles out.

GeoIP filter: geo: { allow: [CC], deny: [CC] } on a listener (ISO 3166-1 alpha-2 country codes, case-insensitive) is checked on the client source IP right after the CIDR ACL. Same precedence as allow/deny: deny wins, a non-empty allow is default-deny, and an IP the database can’t place is admitted only when there is no allow list. Requires settings.geo_db — the path to a MaxMind Country .mmdb (e.g. MaxMind’s free GeoLite2-Country); --check and startup fail if it can’t be opened, and a listener whose DB somehow isn’t loaded fails closed. Blocked ⇒ wayhouse_filter_blocked_total{listener,filter="geo"}. Startup-only, like settings.workers.

Sniffer loader (phase 9; config schema slice 2, the wasmtime loader slice 3, live dir rescanning slice 4 — all landed): settings.sniffers: { dir, call_timeout_ms, max_memory_bytes, modules: [{ name, sha256, config? }] }. Absent ⇒ no sniffers load and a sniffer: route never matches. dir is a directory of *.wasm modules, scanned at startup and rescanned on every reload (added modules load, removed ones drop, changed ones recompile — swapped in atomically, no listener restart); call_timeout_ms (default 20) bounds a sniffer’s wall-clock time per call via wasmtime epoch interruption; max_memory_bytes (default 16 MiB) caps a call’s instance memory; modules: optionally pins each module’s SHA-256 for supply-chain verification — when non-empty, a dir entry not listed there (or whose hash doesn’t match) is refused (and, on a rescan, keeps the previous sniffer set rather than applying a half-updated one). modules[].config is an optional opaque string handed to that sniffer on every sniff call (the sniffer parses it itself — e.g. a match pattern for regex-firstbytes); a module needs a modules[] entry (hence its sha256) to carry a config. validate() rejects an empty dir, a zero call_timeout_ms / max_memory_bytes, a non-64-hex-char sha256, and an empty config string. call_timeout_ms / max_memory_bytes themselves, and the settings.sniffers block appearing/disappearing, are startup-only — the wasmtime::Engine and its epoch-ticker thread are built once (like settings.workers). Modules within an already-configured dir can also be managed over HTTP: GET/POST /admin/sniffers + DELETE /admin/sniffers/{name} + POST /admin/sniffers/{name}/rollback (swaps in the kept previous version, docs/sniffers.md “Updates and rollback”) on the instance’s own admin API (409 if settings.sniffers is absent). An upload is checked before it is written and written atomically: anything that is not a loadable module (not wasm, has imports, lacks the memory / alloc / sniff exports, declares no ABI version or another one than the host’s, fails to instantiate, empty) is 400, a body over 8 MiB is 413, and on an instance with settings.sniffers.modules pins an unlisted name or a different sha256 is 409. A bad .wasm file already in dir is logged and skipped by every rescan when modules is not set. When modules pins are set, a pin violation, a pinned module that fails validation, or a pinned path that is not a readable file (a directory, a dangling symlink) fails the whole scan (a pin whose file has not been uploaded yet is fine): the process refuses to start (naming the module), and a reload keeps the previous sniffer set. Uploads are fanned out fleet-wide via wayhouse-aggregator’s POST/DELETE /fleet/sniffers[/{name}] and the admin GUI’s Sniffers page — see docs/sniffers.md “Installing by hand”.

Tier-2 regional health fabric (phase 13, docs/10 “Tier 2” — built): settings.failure_domain (a string identity — this instance’s reachability-equivalence class, e.g. an AZ or region) and settings.gossip: { bind, seeds?, quorum_fraction?, psk } must be set together, or not at all — validate() rejects either one alone. bind / seeds[] are UDP socket addresses; seeds is optional (any subset of a domain’s live members is enough to join, membership then discovers the rest) and defaults to empty. quorum_fraction (default 0.66) must be > 0.5 and <= 1.0 — the fraction of the domain that must report a backend down before that verdict can override this instance’s own “up” reading; a same-or-under-half quorum could contradict itself between two overlapping majorities, hence the > 0.5 floor. psk (a pre-shared key, must not be empty) authenticates every gossip datagram with an HMAC-SHA256 tag. When the mesh itself lands, it will only ever be able to push a backend’s flag down, never revive it — a backend returns healthy only on this instance’s own local rise streak, and a Tier-1 force-down (AdminState::Disabled) always wins over the domain view. Startup-only, like settings.workers.

Fleet grouping: settings.group (optional string) is this instance’s self-reported organization path — e.g. "eu/frankfurt/cluster-a" — pushed to wayhouse-aggregator alongside its IngestPayload so the admin GUI can render a grouped/tree fleet view. /-separated, non-empty segments, no leading/trailing /; validate() rejects anything else. Purely a display label — never consulted by routing/forwarding, and independent of failure_domain (which is about health-fabric membership, not fleet organization).

Global caps: settings.limits: { max_connections, max_udp_sessions, max_new_sessions_per_sec } are process-wide ceilings (all optional; omit for no cap). max_connections / max_udp_sessions bound the live counts across every listener; max_new_sessions_per_sec is a token bucket (burst = the rate) over new connections and new UDP sessions combined. A new connection / session that would breach a cap is dropped before it is allocated (existing ones keep running) and counted by wayhouse_filter_blocked_total{filter="max_conn"|"max_udp"|"max_new_rate"}. Startup-only, like settings.workers.

Web sites (schema 3): the optional top-level web: section describes HTTP(S) sites published through a Caddy that sits behind a Wayhouse pool (design: issue #297, spec docs/superpowers/specs/2026-10-15-web-routing-design.md). It is validated and resolved as Config.web and needs schema_version: 3. The only routing it adds are the loopback listeners for via: wayhouse upstreams (below); everything else is read by wayhouse-web-sync through GET /web (docs/06) and rendered into Caddy.

web:
  pool: web                       # pool in front of Caddy (default "web"); must set proxy_protocol: v2
  listen: { http: 127.0.0.1:8080, https: 127.0.0.1:8443 }   # where Caddy listens (defaults shown)
  trusted_proxy: []               # addresses / CIDRs Caddy trusts PROXY headers from
  internal_bind: 127.0.0.1        # loopback only (default shown)
  internal_ports: "19000-19999"   # lo-hi, at most 1000 ports (default shown)
  sites:
    - hosts: [panel.example.com, "*.apps.example.com"]
      upstreams:
        - addr: 10.0.0.5:8080     # a plain address, or
        - pool: panel             # a pool (via defaults to direct), or
        - { pool: wings1, via: wayhouse, port: 19010 }   # port pins the loopback listener
      tls: { cert: /certs/panel.pem, key: /certs/panel.key }   # served by Caddy (optional)
      upstream_tls: skip          # verify | skip | { ca_file: <path> }; absent = plain HTTP
      health_path: /healthz       # active health check path on each upstream (optional)

Rules: pool must exist and set proxy_protocol: v2 (Caddy’s PROXY listener rejects plain connections). Hosts are plain hostnames or *.suffix (labels of a-z0-9_-; no port, no IP literal, no leading-dot form), lowercased, and unique across sites; each site needs a host and an upstream. An upstream is exactly one of addr / pool; via (direct | wayhouse) and port apply only to a pool, and port only with via: wayhouse, inside internal_ports, one port per pool and one pool per port, and not one of listen’s ports; a pool used via: wayhouse must not set proxy_protocol (the next hop is plain HTTP). health_path starts with / and has no whitespace, control characters, { or }. internal_bind must be a loopback address.

listen may be a routable address: on a gateway node Caddy cannot listen on loopback only. trusted_proxy is a list of IPs or CIDRs (a bare address is a /32 or /128); with a web pool on another node, put that node’s address in it, because Caddy’s proxy_protocol allow must contain the peer. tls.cert and tls.key are PEM files that wayhouse-web-sync reads (paths on its filesystem), both non-empty; without tls Caddy has no certificate for the site’s hosts and serves them over plain HTTP. upstream_tls is per site: skip turns off verification (acceptable on an authenticated tunnel), ca_file is a path on Caddy’s side. health_path must start with /. A non-loopback listen with an empty trusted_proxy logs a warning, as does a web.pool that no TCP listener routes to (--check, startup and reload).

Listeners for via: wayhouse upstreams. For each distinct pool used with via: wayhouse the validator appends a TCP listener web-up-<pool> to the config’s listeners, bound to internal_bind:<port> with one always route to that pool. Caddy’s upstream is that loopback address, so the connection takes the normal path: pool balancing, health checks, drains, tunnel backends. Several sites sharing a pool share one listener. The port is the pin if there is one, otherwise the first 8 bytes of the SHA-256 of the pool name (big-endian) modulo the range size, moving on to the next free port when taken. Taken means: another pool’s pin or generated port, an operator TCP listener bound to the same address or to the wildcard address of the same family, or the admin address. Adding or removing another pool does not move an existing pool’s port, except when two names hash to the same port; pools are then handled in name order, so pin a port if you need it fixed whatever else changes. A pin on a port an operator listener or the admin API already holds is an error, as are a listener name that clashes with web-up-<pool> and a range with no free port left. Reload reconciles these listeners by name like any other.

Client address. Caddy sets X-Forwarded-For from the PROXY v2 header, so an origin sees the client address in HTTP with either via mode. On the via: wayhouse hop the TCP peer is Caddy’s loopback address, which is expected: Wayhouse does not parse PROXY headers inbound and none is needed for HTTP.

Plugin routes (schema 2): plugin_routes: { type: sni } (or { type: sniffer, sniffer: <name> }) on a TCP listener lets enabled controller plugins add routes to it (docs/plugins.md “Routes”). They are matched by the hostname the connection announces, appended after the listener’s own routes (so any operator route, including an always fallback, wins), and each backend gets a one-target pool named plugin-<hash of the address>. Only meaningful with --controller; the field needs schema_version: 2.

UDP first-packet gate: first_packet_gate: true on a UDP listener makes it create a session only when the first datagram is positively recognised — a sniffer hint that is not reject, or a first_bytes route on the listener that matches the datagram. Applied before the route_hint lookup (a spoofable src_ip hint must not bypass it). Unrecognised datagrams are dropped with no session and no reply, counted by wayhouse_datagrams_dropped_total{reason="first_packet_gate"}. Requires at least one first_bytes route or a sniffer (else it would drop everything).

A UDP session reads the routed pool’s idle_timeout_sec once when it is created. UDP affinity (a client that comes back after its session was evicted reaches the same backend) comes from the pool: use balancer: consistent_hash with hash_on: src_ip | src_ip_port. There is no per-listener affinity key and no sticky table (removed in #56: it was per worker, so SO_REUSEPORT kept only ~34% of clients on 4 workers). A round_robin pool gives a returning client a new backend. See config.example.yaml.

Handshake-only sniffers (UDP, #131): the quic, wireguard, openvpn, raknet and teamspeak3 sniffers recognise only a flow’s handshake datagram. The route is decided when a session opens, not per connection, so a flow whose session was evicted by idle_timeout_sec (or whose NAT mapping changed, or that reached this proxy after a restart or an ECMP move) reopens on a transport datagram that no sniffer recognises. Without a gate it falls to the next matching route; with first_packet_gate: true it is dropped until the client handshakes again (WireGuard rekeys after about two minutes). Per protocol: keep the pool’s idle_timeout_sec well above the keepalive (the config warns below 60 s, at load, reload and --check); a catch-all always route after such a sniffer that goes to a different pool is rejected unless first_packet_gate: true is set; and use route_hint, or a pool with balancer: consistent_hash (hash_on: src_ip), so a reopened session still reaches the same backend. The check goes by the sniffer’s module name.

Admin API auth (phase 10+11 slice 10): settings.admin.auth_token (a flat string, not the target schema’s auth: { mode, token } object below) gates every admin API request except GET /healthz with a bearer check. None (the default) leaves the API open — network-boundary-only auth, same as always. A single shared secret, not RBAC/mTLS — appropriate for gating a control-plane API that’s meant to stay behind its own network boundary regardless; needed once something calls in from outside that boundary, which today means wayhouse-aggregator’s intent-verb fan-out (docs/10 “The aggregator”), given the same token to present.

Admin API TLS (2026-10-02): settings.admin.tls: { cert, key } (PEM chain, leaf first; PEM private key — both required) serves the admin API over HTTPS. Startup-only like listen; the files are re-read every 30 s, so a renewed certificate needs no restart. wayhouse --check loads the pair; errors name settings.admin.tls.cert/.key. Optional keys under tls bound the TLS handshakes: max_pending (512), max_pending_per_source (16), new_per_source_per_sec (20; 0 = no rate limit), new_per_source_burst (64); see docs/12 “Native TLS”. The admin URL reported to --aggregator becomes https://<listen>, so the certificate must be valid for the listen address (an IP SAN), and the aggregator needs --ca-file for a private CA. With --controller every instance gets the same YAML, so the paths are the same on every host while each host’s file must hold a certificate for its own listen address. See docs/12 “TLS for the fleet services”.

The full schema below is the target.

# Optional: the minimum config schema this document needs (whole number, 1 or
# higher; absent = 1). See "Schema version" below.
schema_version: 1

# global
settings:
  workers: 0                 # 0 = number of CPU cores
  shutdown_grace_sec: 30     # wait this long for in-flight conns on SIGTERM
  admin:
    listen: "127.0.0.1:9900"
    auth: { mode: "bearer", token: "${ADMIN_TOKEN}" }   # or mode: mtls
  metrics: { path: "/metrics" }
  log:
    format: "json"
    connection_log: { enabled: true, sample: 1.0 }
  limits:
    max_connections: 500000
    max_udp_sessions: 1000000
    max_new_sessions_per_sec: 50000   # new conns + UDP sessions/s (burst = rate)

# reusable filters
filters:
  - name: block-bogons
    type: deny_cidr
    cidrs: ["10.0.0.0/8", "192.168.0.0/16", "100.64.0.0/10"]
  - name: conn-rate
    type: rate_limit
    scope: src_ip            # src_ip | src_/24 | global
    new_per_sec: 50
    burst: 100

# backend sources (phase 8) — a pool takes its backends from `targets:` OR a
# named `source:`, never both. Level-triggered: the adapter returns the current
# address set; the runtime diffs it against the live set. A refresh that errors
# or returns empty keeps the last-known-good set. `static` is folded into the
# pool's targets at load time; the dynamic kinds each get one control-plane
# refresh task (never on the data path).
backend_sources:
  - name: static-eu
    type: static
    targets: ["10.1.0.11:7777", "10.1.0.12:7777"]
  - name: k8s-match
    type: kubernetes            # LIST .../endpointslices for the service, plus a watch for instant updates
    service: "match-server"
    namespace: "games"          # default: "default"
    port_name: "game"           # optional; else each slice's first port
    api: "https://kubernetes.default.svc"   # default; SA token + CA read in-pod
    refresh_interval_sec: 10   # k8s: resync interval; changes arrive via the watch
  - name: consul-eu
    type: consul               # GET /v1/health/service/<service>?passing=true, plus a blocking query for instant updates
    service: "match-server"
    consul_addr: "http://127.0.0.1:8500"    # default
    tag: "prod"                # optional
    consul_token_file: "/run/secrets/consul-token"   # optional ACL token (X-Consul-Token), re-read as it rotates
    refresh_interval_sec: 10   # consul: resync interval; changes arrive via the blocking query
  - name: srv-us
    type: dns_srv              # resolves the SRV record; port from the record
    record: "_game._udp.us.internal.example.com"
    refresh_interval_sec: 10
  - name: home-origin
    type: tunnel               # phase 14 (built) — the `tunnel` BackendSource.
                                # Resolves backend addresses a `wayhouse-agent`-managed
                                # origin behind a WireGuard tunnel has registered
                                # with the controller's backend-peers registry.
    pubkey: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="  # origin's WG pubkey
    refresh_interval_sec: 10

# pools
pools:
  - name: match-eu
    source: static-eu
    balancer: { strategy: consistent_hash, hash_on: src_ip }
    affinity: { table_ttl_sec: 120 }
    health_check:
      type: udp_probe          # tcp_connect | udp_probe | none | http
      send_hex: "01"
      expect_hex_prefix: "02"
      interval_sec: 2
      timeout_ms: 500
      rise: 2
      fall: 3
    per_backend:
      max_sessions: 200
      max_new_per_sec: 20
    proxy_protocol: "none"     # none | v1 | v2 | v2-udp
    connect_timeout_ms: 300
    idle_timeout_sec: 90

  - name: match-us
    source: srv-us
    balancer: { strategy: least_conn }
    health_check: { type: tcp_connect, interval_sec: 2, timeout_ms: 400, rise: 2, fall: 3 }

  - name: source-query
    source: static-eu
    balancer: { strategy: round_robin }
    idle_timeout_sec: 10

# resolvers (external routing logic)
resolvers:
  - name: matchmaker
    type: grpc                 # grpc | http
    endpoint: "https://matchmaker.internal:8443"
    timeout_ms: 40
    proxy_protocol: "none"     # none | v1 | v2 | v2-udp — header for a `target` result
    target_connect_timeout_ms: 300  # connect / idle timeout for a `target` result
    target_idle_timeout_sec: 90     #   (no pool to read them from)
    cache:
      key: ["first_bytes:0:16"]     # e.g. a session-token prefix
      positive_ttl_sec: 30
      negative_ttl_sec: 2
      max_entries: 200000
    on_error: "reject"         # reject | fallback | stale_ok

# listeners + routes
listeners:
  - name: public-udp
    bind: "0.0.0.0:7777"
    protocol: udp
    filters: ["block-bogons", "conn-rate"]
    peek_max_bytes: 64
    routes:
      - match: { type: always }
        action: { resolver: matchmaker }
      - match: { type: always }        # fallback if resolver on_error=fallback
        action: { pool: match-eu }

  - name: public-tls
    bind: "0.0.0.0:443"
    protocol: tcp
    peek_max_bytes: 2048
    peek_timeout_ms: 200
    routes:
      - match: { type: sni, suffix: ".eu.example.com" }
        action: { pool: match-eu }
      - match: { type: sni, suffix: ".us.example.com" }
        action: { pool: match-us }
      - match: { type: always }
        action: { pool: match-eu }

  - name: public-mc
    bind: "0.0.0.0:25565"
    protocol: tcp
    peek_max_bytes: 512
    routes:
      - match: { type: sniffer, name: minecraft, host: "survival.example.net" }
        action: { pool: match-eu }
      - match: { type: always }
        action: { pool: match-us }

  - name: source-27015
    bind: "0.0.0.0:27015"
    protocol: udp
    routes:
      - match: { type: first_bytes, prefix: "hex:FFFFFFFF" }
        action: { pool: source-query }
      - match: { type: always }
        action: { pool: match-eu }

  # Raw UDP with no protocol hint: subdomain by destination IP (see docs/03 scheme A)
  - name: raw-udp
    bind: "[::]:7777"                     # wildcard bind, one socket
    protocol: udp
    prefix: "2001:db8:ace:1::/64"         # enables IP_PKTINFO / IPV6_RECVPKTINFO;
                                           # routes by the real per-datagram destination
    routes:
      - match: { type: dst, cidrs: ["2001:db8:ace:1::1/128"] }   # survival.example.net
        action: { pool: match-eu }
      - match: { type: dst, cidrs: ["2001:db8:ace:1::2/128"] }   # creative.example.net
        action: { pool: match-us }
      # No `always`/catch-all route and no `reject` action — there isn't one;
      # an unmatched destination IP simply has no matching route and is
      # dropped (no_route).

  # IPv4-only variant: subdomain by port (scheme B), SRV hands out the port.
  # One listener, one `bind: "host:lo-hi"` port range (F1.4) — one real socket
  # per port, all sharing this listener's routes; the `port` route matcher
  # picks the pool by the port the datagram actually arrived on. Ports in the
  # range with no matching route (anything but 30001/30002 here) are dropped.
  - name: raw-udp-v4
    bind: "0.0.0.0:30000-30099"
    protocol: udp
    routes:
      - match: { type: port, ports: [30001] }
        action: { pool: match-eu }
      - match: { type: port, ports: [30002] }
        action: { pool: match-us }

A kubernetes source reads the service’s EndpointSlices (discovery.k8s.io/v1, selected by the kubernetes.io/service-name label) and merges them: ready endpoints only (an unset ready counts as ready), duplicates across slices collapsed, FQDN slices skipped, and a slice without the wanted port ignored. Its service account needs list and watch on endpointslices in the namespace; deploy/k8s/45-wayhouse-rbac.yaml is a ready-made Role. Without watch the source logs one warning per outage and keeps converging on the poll interval, so a missing grant slows updates but never breaks discovery. The watch resumes from the version of the last list and reopens itself when the API server ends it; a 410 Gone triggers an immediate list, which renews the version. A failing watch retries every 5 s, doubling up to 60 s. The service-account token is read from its file again every 60 s, so the kubelet’s rotation of the bound token (about hourly) is followed without a restart.

A consul source lists the passing instances of the service and then holds a blocking query (?index=<X-Consul-Index>&wait=300s) on the index of its last list; a new index triggers a list at once, and the poll interval stays as the resync net. An index that goes backwards, or a response without one, is treated as a reset and gets a fresh list. A failing query retries like the Kubernetes watch (5 s doubling to 60 s). With ACLs enabled, put the token in a file and point consul_token_file at it; the file (not the token) is what the YAML and GET /config carry, and it is read again every 60 s. The token needs service:read on the service and node:read on the nodes running it. The service name and the tag are URL-escaped.

Both pushed signals are coalesced: after the first change the source is fetched once it has been quiet for 500 ms (at most 5 s after the first change), so a rolling update of many pods costs one fetch, not one per event.

Validation rules (excerpt)

• A pool needs exactly one of targets: / source:; source must point to a backend_sources[].name. refresh_interval_sec >= 1. dns_srv needs record; consul / kubernetes need service (consul_token_file, if given, must not be empty); a Kubernetes api (which receives the pod’s service-account token) and a consul_addr used together with consul_token_file must be https:// or http:// to a loopback address, with no user info; tunnel needs pubkey (a base64-encoded 32-byte WireGuard key).
• Every action.pool / action.resolver must exist.
• Every listener needs at least one route. There is currently no warning for a route list with no trailing always — an unmatched connection/datagram is silently dropped (no_route); a “did you forget a catch-all route?” lint is a plausible small future addition, not built today.
• proxy_protocol: v2-udp only together with protocol: udp (for a pool, and for a resolver whose routes are on UDP listeners); v1/v2 only with TCP.
• A resolver’s timeout_ms, target_connect_timeout_ms and target_idle_timeout_sec must all be > 0.
• transparent: true (TCP or UDP) may not be combined with prefix, and may not be set on a listener whose pool: or a route’s pool action uses a tunnel backend source (the client and tunnel address families can differ). A resolver route can still return such a pool at runtime; there the proxy connects without the client’s source address.
• Every entry in a listener’s allow / deny must be a valid CIDR.
• rate_limit, if present, needs at least one of per_ip / per_net, each with rate >= 1.
• per_source, if present, needs at least one of max_per_ip / max_per_net, each >= 1.
• Every settings.limits.* value, if present, must be >= 1 (0 would block all traffic — omit the key for no cap).
• first_packet_gate: true is UDP-only and needs at least one first_bytes route or a sniffer on the listener.
• A UDP listener with a handshake-only sniffer (quic, wireguard, openvpn, raknet, teamspeak3) must not have a later always route to a different action unless first_packet_gate: true is set (#131).
• A listener geo filter requires settings.geo_db, a non-empty allow or deny, and 2-letter country codes; the DB file must open at startup / --check.
• consistent_hash requires hash_on.
• weights is accepted only with balancer: weighted; every key must parse as ip:port and every value must be >= 1.
• settings.sniffers.dir must not be empty; call_timeout_ms / max_memory_bytes must be >= 1; each modules[].sha256 must be a 64-char hex digest and name must not be empty; a modules[].config, if given, must be non-empty.
• settings.failure_domain and settings.gossip must both be set, or neither; gossip.bind / each gossip.seeds[] entry must be a valid socket address; quorum_fraction must be > 0.5 and <= 1.0; psk must not be empty.
• match.type: dst needs the destination address to be known: on TCP it comes from getsockname() for free; on UDP it needs either transparent: true (any dest) or prefix: <cidr> (dest within that prefix) on the listener — there is no separate recv_dst_addr flag, prefix/transparent themselves turn on the recv path. freebind is TCP-only and is rejected together with prefix (a UDP prefix listener needs the routed prefix reachable to the host, not IP_FREEBIND).
• match.type: port matches the destination port of the accepting socket; its own ports: list supports a "lo-hi" range.
• A listener’s bind is either one host:port socket address, or a host:lo-hi port range (F1.4) — one real socket per port (TCP: one per worker per port; UDP: same, SO_REUSEPORT-shared), all sharing that listener’s routes/filters/pool selection; a route’s port matcher still sees the real port a connection/datagram arrived on. Capped at 1024 ports per range (a typo like 0-65535 would otherwise try to open tens of thousands of sockets). Mutually exclusive with prefix (which needs exactly one wildcard socket).
• Bind addresses must not overlap between listeners (same IP:port:proto, checked per port of a range too); a prefix bind must not cover a single bind address of another listener.
• Numeric ranges: timeouts > 0, rise/fall ≥ 1, TTLs ≥ 0.

Schema version

schema_version (top level, optional) is the minimum config schema the document needs; absent means 1. A build supports up to the schema in wayhouse_config::version::CONFIG_SCHEMA_VERSION (currently 3). A document that asks for more is refused with config schema_version N is newer than this build supports (max M), on startup, on a file reload (the running config stays active) and on a controller-delivered revision. A string, 0 or a fractional value is an error that says so. deny_unknown_fields stays on, so an older node also rejects a newer field loudly instead of ignoring it.

Bump rule for contributors: a PR that adds a config field, even an optional one, raises CONFIG_SCHEMA_VERSION and adds the field’s dotted path to FIELD_SINCE (wayhouse-config/src/version.rs) with the new number. The parser then refuses a document that uses that field without a schema_version that covers it, so nobody adopts a new field by accident while older proxies still run.

Mixed fleets. Proxies that register with the controller (the tunnel source) report the newest schema they understand (max_config_schema) and how often they re-register. POST /config (and POST /config/promote/{revision}, for a staged revision) answers 422 when the document’s schema_version is above the lowest value reported by any live proxy: one that re-registered within three of its own intervals, so a stopped or deleted proxy stops counting. A live proxy that reports nothing predates the field and cannot parse a document that carries the schema_version key, so it holds such documents back until it upgrades or expires; a document without the key still goes through. If the controller cannot read its proxy registry the check fails closed (503). Upgrade every proxy before submitting a document that declares schema_version or uses a field introduced by the new version. Proxies that do not register (no tunnel) are not tracked: they still refuse a document they cannot parse and keep their previous config.

Reload semantics

Health checks

• health_check.type defaults to tcp_connect, which can never succeed against a UDP-only backend. A pool that only UDP listeners route to and that leaves type out is therefore rejected at load: pick udp_probe (with send_hex, and expect_hex_prefix if the reply is recognisable) or none. A pool shared with a TCP listener keeps the default; an explicit type is always accepted.
• none runs no active probe, for a game with no probe payload. Health then comes from passive observations alone: a UDP ICMP port-unreachable, a failed upstream send or a TCP connect failure counts against a backend, and a backend marked unhealthy is given another chance after one interval_sec (re-admitted after rise such intervals), since nothing else could ever bring it back. A truly dead backend therefore flaps: down on passive failures, back up after rise intervals, down again on the next failure.
• For UDP, a successful send() is not a passive success (it succeeds whether or not anything listens). The first reply datagram of a session is.
• A proxy-side shortage (EMFILE, ENFILE, ENOMEM, ENOBUFS, EADDRNOTAVAIL) while opening a socket, for a probe or for a session, is not counted for or against the backend.
• Probes run in the background, at most 256 at once; the first check of each backend is spread over one interval_sec. A slow probe does not delay the others, and a backend is never probed twice at once.
• idle_timeout_sec must be greater than 0 (as for the other timeouts).