Skip to content

Deployment

Three ways to run humpyard: a release binary, the container image, or the macOS launchd service (which uses the binary). Releases are described at the end.

Whatever you use, you need a config file (examples/quickstart.toml is the smallest; examples/config.toml adds failover across two providers; check one with humpyard check-config) and the provider API keys in environment variables named by the config (api_key_env). Keys are never stored in the config.

First config: humpyard init

humpyard init (0.10.0 and later) writes a working config.toml for you. Export your provider key, then name the provider:

export GROQ_API_KEY=...
humpyard init --provider groq --out config.toml
humpyard serve --config config.toml
  • Built-in providers are groq, openrouter and nvidia; --custom NAME,BASE_URL,KEY_ENV,MODEL adds any OpenAI-compatible endpoint. Run it without --provider in a terminal to be asked.
  • Each key is read only from the environment variable the config names. It is never printed, stored or asked for.
  • It lists each provider's models and writes a model only if the provider lists it and it answers one small request, so a retired model never reaches the file. A provider whose variable is unset is written unchecked.
  • The file listens on 127.0.0.1 and has two targets, fast and deep. It sends one test request through the result and prints the reply and the target that served it. An existing file is never overwritten.
  • Pass --out - to print the configuration to standard output instead of creating a file, with progress messages routed to standard error. An existing file is never checked in that mode.
  • --listen 0.0.0.0:8080 (anything not on loopback) also creates a virtual key with a daily token budget and prints it once; the file holds only its hash.

Release binary

Each GitHub release has tarballs for macOS arm64 (aarch64-apple-darwin) and Linux x86_64 / arm64 (*-unknown-linux-gnu), plus SHA256SUMS.

shasum -a 256 -c SHA256SUMS --ignore-missing      # sha256sum -c on Linux
tar xzf humpyard-v0.12.0-aarch64-apple-darwin.tar.gz
install -m 755 humpyard-v0.12.0-aarch64-apple-darwin/humpyard ~/.local/bin/
  • The binaries are not signed. If macOS quarantines a downloaded one: xattr -d com.apple.quarantine ~/.local/bin/humpyard.
  • Linux binaries are built on Ubuntu 24.04 and need glibc 2.39 or newer. On an older distribution use the container image.

Container image

docker run -d --name humpyard -p 127.0.0.1:8080:8080 \
  -v "$PWD/config.toml:/etc/humpyard/config.toml:ro" \
  -v humpyard-data:/data \
  -e GROQ_API_KEY -e OPENROUTER_API_KEY \
  ghcr.io/gschaetz/humpyard:0.12.0

In the config, set listen = "0.0.0.0:8080" (inside the container the default loopback address is unreachable; quickstart.toml already does) and the ledger path to /data/ledger.db so usage survives upgrades. The command publishes the port on 127.0.0.1 only; drop that prefix (and add virtual keys first) to reach it from other machines.

  • The image is distroless, runs as a non-root user (65532), has no shell, config or secrets, and is about 70 MB. docker stop triggers the graceful shutdown (the default stop timeout of 10 s is shorter than the 30 s grace period; use docker stop -t 45 or lower shutdown_grace_secs).
  • There is no built-in container health check (no shell or curl in the image); probe GET /healthz from outside.
  • Tags: X.Y.Z for every release; latest only moves for releases at 1.0 or later.
  • A bind-mounted /data must be writable by uid 65532.
  • The package is public (verified with an anonymous pull of 0.1.0): the image's org.opencontainers.image.source label links it to this public repository. Each architecture carries a build provenance attestation (buildx default), shown as an unknown/unknown entry in docker buildx imagetools inspect.

macOS service (launchd)

Layout (all under your home directory, nothing needs root):

Path What
~/.local/bin/humpyard the binary
~/.config/humpyard/config.toml the config
~/.config/humpyard/env provider keys, chmod 600: lines like OPENCODE_API_KEY=... or export ...
~/Library/Logs/humpyard.log stdout and stderr 
deploy/launchd/install.sh          # checks the files exist, installs the agent, starts it
tail -f ~/Library/Logs/humpyard.log
launchctl bootout gui/$(id -u)/dev.humpyard.gateway    # stop and unload

The service starts at login, restarts if it dies (10 s throttle) and loads the env file in the shell that launches it, so keys never appear in the plist, in launchctl print or in logs. ExitTimeOut is 50 s: launchd waits that long after SIGTERM, which covers the default 30 s grace period plus the ledger flush. Change the environment file, then launchctl kickstart -k gui/$(id -u)/dev.humpyard.gateway to restart.

Reloading the configuration

Edits to routes, selector rules, targets, providers, keys, budgets and health settings apply to a running gateway without a restart; requests already running (streams included) finish on the configuration they started with. A file that does not load, validate or build is never applied: the previous configuration keeps serving and the failure is logged and shown in GET /v1/health (reload.ok, reload.message, and config.fingerprint for the version in use).

Environment How to reload
any set reload_poll_secs = 15 (top level, 0 = off): the file is re-read on that interval and reloaded when its content changed. A broken file is attempted once per change.
macOS binary or launchd service kill -HUP $(pgrep -x humpyard) (the service replaces its shell with the binary, so this finds the right process)
Docker docker kill -s HUP humpyard (a bind-mounted file must be edited in place, or the container restarted, if your editor replaces the file)
Kubernetes mount the config from a ConfigMap and set reload_poll_secs: the image has no shell or kill, and the kubelet refreshes the mounted file on its own schedule (up to about a minute), after which the next poll picks it up (allow a minute or two for the kubelet)

Not reloadable, and refused as a whole with a message naming the setting: listen, the ledger path, shutdown_grace_secs, reload_poll_secs, and adding the first key to (or removing the last from) a keyless gateway. Provider API keys come from the process environment, which a running process cannot change, so rotating one needs a restart. Carried over by a reload: the health of endpoints that did not change, budget counters, and the session state of unchanged routes.

Operating notes

  • GET /healthz answers 200 while the process is up; GET /v1/health (with a key when the gateway has keys) shows which provider endpoints are being skipped (routing.md).
  • Ledger and budgets live in the SQLite file; back it up by copying the .db together with its -wal file, or stop the service first.
  • Upgrading: replace the binary (or pull the new tag) and restart. The ledger schema is created on open; before 1.0, read the release notes for schema changes.

Releases (maintainers)

Releases are made only by pushing a version tag (ADR 0010); nothing publishes from branches or pull requests.

  1. In a PR, bump version in Cargo.toml and merge it.
  2. From an up-to-date main: git tag vX.Y.Z && git push origin vX.Y.Z.
  3. The release workflow verifies that the tag equals the Cargo.toml version and that the commit is on main, builds each target on a runner of its own architecture, builds the two image architectures, and only when all of that has succeeded creates the GitHub release (pre-release while the version is 0.x) and the multi-architecture image manifest.

A failed run before the last job publishes nothing public except per-architecture image tags (X.Y.Z-amd64, X.Y.Z-arm64) in the registry. To retry, delete the tag and push it again.