Skip to content

Data plane overview

The data plane is four bounded-context services plus a projector, all under app/. Each service owns its own MySQL schema on HeatWave (ADR 0005), announces every committed change as an event on NATS (ADR 0003), and the projector folds those events into the Valkey settings projection that the hot path reads.

This page is the component view. The rest of the section goes deeper:

flowchart LR
    subgraph services["Data services (app/)"]
        users["users"]
        commands["commands"]
        modules["modules"]
        transactions["transactions"]
    end

    subgraph record["State of record"]
        mysql[("MySQL HeatWave<br/>one schema per service")]
    end

    nats(["NATS JetStream"])
    projector["projector"]
    valkey[("Valkey<br/>settings projection")]
    hot["Hot path readers<br/>(ingress workers)"]

    users --> mysql
    commands --> mysql
    modules --> mysql
    transactions --> mysql

    users -- "change events" --> nats
    commands -- "change events" --> nats
    modules -- "change events" --> nats
    transactions -- "change events" --> nats

    nats -- "invalidation (broadcast)" --> services
    nats -- "durable group" --> projector
    projector --> valkey
    hot --> valkey
Service Schema Owns Write path
app/users bagel_users User records (Twitch ID, username, email, active flag, tier status) and OAuth tokens as Tink AEAD ciphertext Direct, always
app/commands bagel_commands Custom chat commands Write-behind (deletes direct)
app/modules bagel_modules Module toggles and JSON configurations Write-behind
app/transactions bagel_transactions Tebex transaction ID and owning user, nothing else Direct, always, idempotent on retry
app/projector none The Valkey projection (disposable, rebuildable) Event-driven overwrites

Cross-service references are a plain indexed Twitch user ID column. There are no foreign keys across schemas, by construction.

The subjects and payload DTOs live in internal/domain/event/data and are a public contract: renaming a subject or narrowing a payload is a breaking change. Every payload carries the full new state (event-carried state transfer), so consumers never read another service’s schema and redelivery is harmless.

Subject Payload Published when
data.users.changed full user view (ID, username, active, status) Registration, rename, tier change
data.users.deleted user ID User deletion
data.modules.changed user ID, module name, enabled, config JSON Each module row landed by a flush
data.commands.changed user ID, name, response, active, stream-online-only, permissions, cooldown, allowed user, deleted flag Each command row landed by a flush, and deletions
data.transactions.recorded transaction ID, user ID First successful record of a transaction
data.reproject.request empty Projector cold start; owners replay their state as ordinary change events

Two subscription shapes, on purpose:

  • Broadcast (no queue group): cache invalidation. Every instance of a service drops its cached keys when any instance writes.
  • Durable queue group: the projector, and the reproject responders. Exactly one consumer per group handles each event, and the group keeps its position across restarts.

Consumers validate every payload and drop (log and ack) what fails to decode or validate. Nacking a poison message would redeliver it forever.

Every service reads its configuration from the environment. Common variables:

Variable Default Used by
APP_ENV development all (logger profile)
NATS_URL nats://127.0.0.1:4222 all
DB_ADDR 127.0.0.1:3306 data services
DB_USER, DB_PASS required data services
DB_SCHEMA bagel_<service> data services
DB_CA_CERT required data services; dedicated HeatWave endpoint CA PEM used to authenticate the server chain without hostname/SAN verification; startup fails closed when absent
DB_ADMIN_CA_CERT required for DB credential operations admin console; dedicated HeatWave endpoint CA PEM for the privileged schema-admin connection (DB_CA_CERT is the compatibility fallback)
TINK_KEYSET_PATH required users
VALKEY_ADDR 127.0.0.1:6379 projector
VALKEY_PASSWORD empty projector
NEW_RELIC_LICENSE_KEY empty (monitoring disabled) all

Monitoring is a no-op without a license key, so local development needs none of the NEW_RELIC_* variables.