ADR 0004 — Events are the job queue

ADR 0004 — Events are the job queue An architecture diagram generated by Archify. Events are the job queue · in the transaction that caused it · no second queue · Architecture component Events are the job queue in the transaction that caused it · no second queue events.Publish · one INSERT into platformkit_outbox in the caller's tx · the mechanism events.Publish one INSERT into platformkit_outbox in the caller's tx events.Relay · events.JetStream · FOR UPDATE SKIP LOCKED · one durable per subscription · the mechanism events.Relay · events.JetStream FOR UPDATE SKIP LOCKED · one durable per subscription jobs.Scheduler · db.TryLock · Every or Cron · an advisory lock named after the job · the mechanism jobs.Scheduler · db.TryLock Every or Cron · an advisory lock named after the job go test ./kit/events ./kit/jobs · publish in tx · handled once · validateEvents · kept true by go test ./kit/events ./kit/jobs publish in tx · handled once · validateEvents Rejected: a river job queue beside the outbox · two sets of tables · two answers to why it did not run · Architecture component Rejected: a river job queue beside the outbox two sets of tables · two answers to why it did not run every module's Subscriptions and Jobs · Handler in the event's tenant tx · Name is the lock · Architecture component every module's Subscriptions and Jobs Handler in the event's tenant tx · Name is the lock app.Worker · app.All · outbox-relay every second · outbox-purge · Consume · Architecture component app.Worker · app.All outbox-relay every second · outbox-purge · Consume written relayed by timed by checked by instead of relays what Publish wrote Jobs run under its lock the mechanism kept true by Legend Backend Security Message bus External

Context

  • • Before: a transactional outbox, a river job queue with its own tables, workers and migrations, and ad-hoc goroutines
  • • The outbox and the queue solved the same problem: two sets of tables to migrate and two answers to why did that not run

Consequences

  • • Enqueueing cannot fail apart from the write it belongs to; delivery is at-least-once and handling exactly-once through a claim in platformkit_handled
  • • Ordering is per stream, not per aggregate; a periodic job that hangs holds its lock; a row in platformkit_dead_letters is an alert nothing redelivers

Evidence

  • • go test ./kit/events ./kit/jobs: TestPublishIsPartOfTheTransaction, and the runs-once-however-often-delivered, concurrent-relay and one-runner cases beside it
  • • validateEvents at boot refuses a route that would publish an undeclared event; the claim closes the redelivery hole, not every hole