Skip to content
gondolier docs

gondolier — System Architecture

gondolier — System Architecture

Section titled “gondolier — System Architecture”

Status: Authoritative architecture reference.

Cross-references: design.md (product/infrastructure decisions), WEBHOOK_DESIGN.md (webhook event routing), LEASING.md (lease protocol), SECURITY.md (security model), DEPLOYMENT.md (operations).

What gondolier does

Section titled “What gondolier does”

Hosted multi-tenant batch merge queue for Forgejo/Gitea/Codeberg. Reuses shunt’s batch-then-bisect engine as the single source of truth and provides the hosted layer: tenant onboarding, encrypted credential management, scheduling, and queue execution.

The merge queue algorithm is unchanged from shunt:

  1. Batching — eligible PRs with auto-merge scheduled are grouped.
  2. Staging — all PRs are cherry-picked onto a mq/<branch>/staging branch.
  3. Gate — CI runs on the staging branch.
  4. Resolution — if CI passes, all PRs merge in batch order; if CI fails, the batch bisects in log₂(N) runs to isolate the bad PR(s).

Branch protection (required merge-queue status + push allow-list for mq/*/staging) is the actual enforcement point. Gondolier writes no merge-queue status — that belongs to the shunt gate workflow on the tenant’s forge instance.

Infrastructure stack

Section titled “Infrastructure stack”
Layer Choice Rationale
Compute Cloudflare Workers (JS/Go WASM) Free tier: 10M req/day. Serverless, scales to zero.
Database Supabase PostgreSQL Managed, RLS, 500MB free.
Secrets Cloudflare Secrets Serverless KMS. Master key injected at deploy.
Encryption Envelope AES-GCM Master key (Secrets) → AES-GCM per token. Legacy base64 supported.
Queue state Cloudflare KV Transient state, fast reads.
Lease manager Cloudflare Durable Object Per-(tenant, repo) concurrency control.
Rate limiter Cloudflare KV (sliding window) Per-(tenant, forge_instance) throttling.

High-level diagram

Section titled “High-level diagram”
                    ┌──────────────────────────────────────┐
                    │         gondolierview.dev             │
                    │   (sell site + dashboard · Pages)     │
                    └──────────────────────┬───────────────┘
                                           │
                    ┌──────────────────────▼───────────────┐
                    │          Cloudflare Workers            │
                    │   index.mjs  ·  internal/api/         │
                    │   · internal/scheduler/  · internal/  │
                    │      cron/     engine/     forge/     │
                    │      ┌──────────────────────────┐     │
                    │      │  LeaseDO (Durable Object)│     │
                    │      │  per-(tenant,repo) state  │     │
                    │      └──────────────────────────┘     │
                    └──────┬──────────────┬───────────────┘
                           │              │
              ┌────────────▼──┐  ┌────────▼──────────────┐
              │ Supabase       │  │ Cloudflare KV          │
              │ PostgreSQL     │  │ RATE_LIMIT_KV          │
              │ · tenants      │  │ · sliding window       │
              │ · connections  │  │ · transient state      │
              │ · repos        │  │                        │
              │ · audit_log    │  │                        │
              └────────────────┘  └────────────────────────┘


                           │
              ┌────────────▼────────────────────────────────┐
              │         Forgejo / Gitea / Codeberg           │
              │   · PR events (webhook push)                 │
              │   · Branch protection enforcement            │
              │   · Staging branch cherry-pick               │
              │   · CI via GitHub Actions / forge Actions    │
              └─────────────────────────────────────────────┘

Data flow: webhook → LeaseDO → engine → forge API

Section titled “Data flow: webhook → LeaseDO → engine → forge API”
Webhook event
    │
    ▼
POST /api/v1/webhooks/forgejo
    │  1. HMAC signature verification
    │  2. Extract tenant/repo/action from payload
    ▼
LeaseDO.processEvent(event)
    │  1. acquire() — claim per-(tenant, repo) lease
    │  2. Route event → engine action
    │  3. _queueJoin / _queueLeave / _queueReevaluate / _queueGateResult
    │  4. Write to LeaseDO storage (queue, audit, events)
    │  5. release() — free lease
    ▼
Supabase write (audit_log, encrypted credentials lookup)
    ▼
Forge API call (if action requires it)
    · stage PR → gitops.Stager
    · set commit status → forge.ForgeClient.SetCommitStatus
    · schedule automerge → forge.ForgeClient.AutomergeScheduled
    · delete branch → forge.ForgeClient.DeleteBranch

Two execution paths:

  1. Webhook path (primary): event-driven, sub-second latency. The webhook endpoint verifies the HMAC signature, dispatches to LeaseDO, which acquires a per-(tenant, repo) lease, routes the event to the appropriate handler, and releases the lease.

  2. Cron path (safety net): runs every 5 minutes, iterates managed repos, reconciles LeaseDO state against forge API reality. Fixes stale leases, detects missed webhooks, and corrects inconsistent queue state.

Tenancy model

Section titled “Tenancy model”

Unit of isolation: (tenant_id, forge_instance). A tenant may connect to multiple forge instances. A forge instance is never shared across tenants even if two tenants point at the same URL — each gets independent credentials and state.

tenant
 ├─ forge_connection (instance URL, bot identity, encrypted token)
 │   ├─ managed_repo (owner/repo, base branch, status context, merge style)
 │   └─ managed_repo ...
 └─ forge_connection ...

Isolation boundaries:

  • API scope: every request carries a tenant_id extracted from the API key hash. Database queries filter by tenant_id.
  • Worker scope: a worker tick processes one tenant at a time. Decrypted tokens exist in memory only during the tick that needs them.
  • Lease scope: Durable Object instances are keyed by (tenant, repo) — one DO per managed repo. No cross-tenant leakage.
  • Rate limiter scope: KV-based sliding window keyed by (tenant, forge). One tenant’s API noise cannot degrade another tenant.

Encryption model

Section titled “Encryption model”

See SECURITY.md for full details.

Cloudflare Secret (master key, 256-bit)  ← CREDENTIAL_MASTER_KEY
  ↓ AES-GCM (random 96-bit IV per token)
Forge token (PAT string or OAuth token, encrypted, stored in Supabase)


Stored format: enc:v1:<iv_base64url>:<ct_base64url>
Legacy format (transparently accepted): raw base64 (btoa)

Flow:

  1. Encrypt token: AES-GCM with master key + random 96-bit IV
  2. Store: enc:v1:<iv>:<ct> in Supabase forge_connections.encrypted_token
  3. Decrypt at tick: master key → token (in-memory only, never logged)
  4. Legacy compat: raw base64 values decode transparently

Master key is injected via Cloudflare Secrets (CREDENTIAL_MASTER_KEY). Decrypted tokens are never returned by API, never written to logs, never persisted beyond the engine tick.

Key rotation: changing the master key invalidates all old credentials. Existing encrypted tokens cannot be decrypted with the new key. The rotation process:

  1. Generate new master key.
  2. Provision as CREDENTIAL_MASTER_KEY.
  3. Force all tenants to re-enroll (re-submit tokens) — old tokens become invalid.
  4. Old key is removed after all tenants have re-enrolled.

Rollback: if the new key fails, restore the previous key value. Legacy raw-base64 tokens (if any remain) continue to work. All new tokens encrypted with the previous key decrypt normally.

Database schema

Section titled “Database schema”

See supabase/migrations/ for full DDL. Key tables:

tenants

Section titled “tenants”
Column Type Notes
id UUID Primary key
name TEXT Display name
api_key_hash TEXT SHA-256(salt + key) — never store plaintext
api_key_salt TEXT Base64, 16 bytes
settings JSONB Per-tenant config
created_at TIMESTAMPTZ
updated_at TIMESTAMPTZ

forge_connections

Section titled “forge_connections”
Column Type Notes
id UUID Primary key
tenant_id UUID FK → tenants (CASCADE DELETE)
provider TEXT “forgejo”, “gitea”, “codeberg”
instance_url TEXT e.g. https://git.example.com
encrypted_token TEXT AES-GCM encrypted
token_nonce BYTEA AES-GCM nonce
token_type TEXT “pat” or “oauth”
webhook_secret TEXT HMAC secret for webhook verification

managed_repos

Section titled “managed_repos”
Column Type Notes
id UUID Primary key
tenant_id UUID FK → tenants
forge_conn_id UUID FK → forge_connections
name TEXT e.g. “owner/repo”
base_branch TEXT Default: “main”
status_context TEXT Default: “merge-queue”
merge_style TEXT “merge”, “squash”, “rebase”
branch_pattern TEXT Default: “mq/*”
settings JSONB Per-repo overrides

audit_log

Section titled “audit_log”
Column Type Notes
id UUID Primary key
tenant_id UUID FK → tenants (SET NULL)
repo_id UUID FK → managed_repos (SET NULL)
event TEXT “join”, “leave”, “bounce”, “merge”, “stage”
details JSONB Context for the event
created_at TIMESTAMPTZ

The shunt engine integration

Section titled “The shunt engine integration”

gondolier imports github.com/rbtr/shunt/internal/engine and wraps it with multi-tenant I/O. The engine’s Config accepts:

  • ForgeClient — interface satisfied by the standard forge client and a Workers-compatible implementation (uses fetch API, no net/http).
  • Stager — interface for creating staging branches.
  • Metrics — optional observability (nil-safe).

gondolier’s engine package (internal/engine/) is ~120 lines of tenant wiring: fetches the decrypted token, constructs the ForgeClient, calls shunt’s NewEngine, and runs the tick. The algorithm itself is shunt’s — unchanged.

Rate limiting strategy

Section titled “Rate limiting strategy”

Per-(tenant, forge_instance) sliding window via Cloudflare KV:

  • Default: 100 requests per 60-second window.
  • Keys: rl:{tenant}:{forge_host}.
  • On 429/5xx from forge API: exponential backoff scoped to the failing connection. One tenant’s rate limit noise cannot impact another.

The KV-based limiter is used by the engine tick and by the forge client wrapper. The API layer has its own rate limiting (configurable per-tenant).

Failure modes and recovery

Section titled “Failure modes and recovery”
Failure Degradation Recovery
Forge API 5xx Skip tenant, log, retry on next tick Exponential backoff; next cron tick catches up
Forge API 429 Pause tenant, log Backoff window expires; next tick proceeds
Supabase unavailable Cannot read/write tenant data Queue state is re-derivable from forge API; cron reconciles
Cloudflare Secrets unavailable Cannot decrypt tokens Engine tick fails gracefully; does not crash
Durable Object unavailable Lease acquisition fails Cron reconciliation detects stale state and corrects
KV unavailable Rate limiting disabled (fail-open) Engine proceeds; rate limit re-established when KV recovers

Principle: the service never crashes on external dependency failure. A failed tick for one tenant never blocks other tenants. Queue state is largely re-derivable from the forge API — in-flight batches survive restarts.

File layout

Section titled “File layout”
index.mjs                 ← Worker entry point, routing, LeaseDO export
wrangler.toml             ← Workers config (DO, KV, cron, bindings)


migrations/               ← Supabase migration SQL
  001_base_tables.sql     ← tenants, forge_connections, managed_repos, audit_log
  002_user_accounts.sql   ← user accounts, organizations
  003_billing.sql         ← billing, subscriptions


pkg/crypto/               ← Envelope encryption (AES-GCM)
  crypto.go               ← Decryptor, EncryptDEK/DEK, EncryptToken/Token


internal/api/             ← REST API
  handler.go              ← Tenant CRUD, connection CRUD, repo CRUD, auth middleware
  api.go                  ← Router setup, middleware chain


internal/tenant/          ← Tenant data models
  tenant.go               ← db.Tenant, db.ForgeConnection, db.ManagedRepo


internal/db/              ← Database access
  db.go                   ← DB interface, Supabase client


internal/scheduler/       ← Queue scheduler
  worker.go               ← Single-tenant engine tick


internal/lease/           ← Lease management
  lease.go                ← LeaseDO client wrapper


internal/cron/            ← Cron reconciliation
  cron.go                 ← scheduled() handler, stale lease fixup


internal/engine/          ← Engine integration
  engine.go               ← shunt engine wiring (~120 lines)


internal/forge/           ← Workers-compatible forge client
  forge_client.go         ← Implements mq.ForgeClient via fetch()


internal/gitops/          ← Staging
  stager.go               ← API-based branch creation via mq.MergedRef


site/                     ← Sell site (Cloudflare Pages)
  index.html


templates/                ← Dashboard (Go templates / Pages)
  dashboard.html