High Availability

Overview

The Controller release package includes docker-compose.ha.yml, which runs two or more api containers behind the admin-ui nginx front end. Requests are balanced across the API instances, and the instances coordinate live terminal sessions and session sharing through Valkey. If one API instance stops, the others keep serving requests.

All instances run on one Docker host and share one PostgreSQL database. This protects against the failure or restart of an individual API process (and lets you add capacity), not against the loss of the host itself.

Architecture

Clients ──► admin-ui (nginx, host port 3000)
                 │  least-connections load balancing
            ┌────┴────┐
          api-1     api-2  ...  api-N
            └────┬────┘
              Valkey          (session registry, pub/sub, instance heartbeats)
              PostgreSQL      (all persistent data)
              crawler-backend + crawler-worker (single instance)

• nginx resolves the api service through Docker DNS and sends each request to the instance with the fewest active connections.
• Each API instance registers itself in Valkey with a heartbeat every 10 seconds; an instance that stops heartbeating drops out after 30 seconds.
• Database migrations and first-boot setup are coordinated so only one instance performs them.
• The discovery (crawler) containers are not scaled.

Install in HA Mode

1. Run ./install.sh (see Installing the Controller).
2. At Deployment mode, choose 2 (High Availability).
3. Enter the Number of API instances (default 2).
4. Answer the remaining prompts as for a single-instance install.

The installer starts the stack with docker compose -f docker-compose.ha.yml up -d --scale api=N and prints the HA status URL along with the admin credentials.

To start an HA stack by hand with an existing .env:

docker compose -f docker-compose.ha.yml up -d --scale api=2

Required Settings

After the first login, set these in Settings and restart the API instances:

• HA / Clustering → Advertise Address — the external URL clients use, for example https://controller.example.com:3000. It is used to build session-share links and is read at startup.
• TLS / SSL → Sans — include the external hostname so the certificate matches (see HTTPS Certificate).

docker compose -f docker-compose.ha.yml restart api admin-ui

The HA compose file connects every API instance to the bundled Valkey (VALKEY_URL=redis://valkey:6379). To use a Valkey Sentinel deployment instead, set VALKEY_SENTINEL_URLS and VALKEY_SENTINEL_MASTER on the api service. The HA / Clustering settings page shows Valkey information but Valkey itself is configured through these environment variables.

Scale API Instances

Change the number of API instances at any time:

docker compose -f docker-compose.ha.yml up -d --scale api=3

New instances register in Valkey and start receiving traffic once healthy.

Check HA Status

• The dashboard's HA / Clustering panel and the header connection-status popover show the mode — Active-Active when more than one instance is registered, otherwise Single — and the active session count.
• Administration → Platform Health lists every api container with its health.
• The API endpoint GET /api/admin/ha-status (requires login) returns the mode and the registered instances.

Upgrade an HA Deployment

Download a backup first (Upgrades & Backups), set NETSTACKS_VERSION in .env, then pull and recreate with the HA file, keeping your instance count:

docker compose -f docker-compose.ha.yml pull
docker compose -f docker-compose.ha.yml up -d --scale api=2

Planning Considerations

• Database: every instance depends on the same PostgreSQL. For database resilience, use an external, highly available PostgreSQL 16 with pgvector (see Using an External PostgreSQL).
• Session limits: Max Concurrent Sessions in Settings → Terminal Sessions applies to each API instance.
• Live sessions during a restart: browser-terminal SSH sessions are held by the instance that opened them. Restarting that instance disconnects those sessions, and users reconnect.
• Host resources: all instances share the host's CPU and memory, and jumpbox and plugin containers run on the same host.