alla-allaos-fullstack/docs/implementation/task-ep1.4.1-transactional-outbox-activation-worker-runtime-2026-07-13.md

# Task EP.1.4.1 Transactional Outbox Activation & Worker Runtime - 2026-07-13

## Scope

- activated Activity Business Event publication through transaction-aware outbox enqueue
- added lease-based outbox claiming fields to `business_event_outbox`
- added projection outbox worker runtime
- added manual projection operations service contracts
- added PostgreSQL `FOR UPDATE SKIP LOCKED` claim strategy in Drizzle store
- preserved existing in-process Business Event publisher for compatibility and tests
- preserved current Activity API response contracts, follow-up APIs, dashboard/report datasets, notifications, approvals, and recent-activity behavior

## Review Summary

Reviewed before and during implementation:

- `AGENTS.md`
- `plans/task-ep.1.4.1.md`
- `docs/implementation/task-ep1.3-business-event-foundation-2026-07-13.md`
- `docs/implementation/task-ep1.4-projection-foundation-delivery-reliability-2026-07-13.md`
- `docs/standards/engineering-constitution.md`
- `docs/standards/project-foundations.md`
- `docs/standards/architecture-rules.md`
- `docs/standards/task-review-checklist.md`
- `docs/security/crm-authorization-boundaries.md`
- existing Activity service mutation paths
- existing Business Event publisher and projection runtime

## Implementation Summary

### Activity Transactional Outbox Adoption

Activity mutation paths now enqueue Business Events to the outbox inside the same `db.transaction()` as the Activity row mutation:

- create
- update
- assign/reassign
- reschedule
- complete
- cancel
- delete

The Activity API still returns hydrated Activity records after commit. Projection consumers are not run synchronously from Activity source mutations.

### Transaction-Aware Publisher

Added:

- `src/features/foundation/projections/server/transactional-publisher.ts`

The publisher validates registry and payload contracts, then inserts into `business_event_outbox` using the supplied transaction/client. This prevents accidental double-publication through the in-process dispatcher.

### Worker Runtime

Added:

- `src/features/foundation/projections/worker.ts`

Worker behavior:

- claims available events in batches
- applies a lease with `claimedBy`, `claimedAt`, and `claimExpiresAt`
- dispatches claimed committed events through `ProjectionConsumerRuntime`
- does not auto-start unless `PROJECTION_WORKER_ENABLED=true`
- supports graceful stop semantics for the in-process loop

Default operational config:

- polling interval: `3000ms`
- batch size: `25`
- max concurrent events: `1`
- max concurrent consumers: `4`
- claim lease: `60000ms`
- stale claim: `90000ms`
- graceful shutdown: `30000ms`

### Multi-Instance Claiming

Added schema fields:

- `business_event_outbox.claimed_by`
- `business_event_outbox.claimed_at`
- `business_event_outbox.claim_expires_at`

Generated migration:

- `drizzle/0004_sharp_mercury.sql`

Database store uses PostgreSQL-safe claiming with `FOR UPDATE SKIP LOCKED`. Expired `claimed` or `processing` leases are recoverable.

### Manual Operations Contracts

Added:

- `src/features/foundation/projections/operations.ts`

Contracts:

- `drainOutbox`
- `retryEvent`
- `retryConsumer`
- `retryDeadLetter`
- `resolveDeadLetter`

Manual operations require `systemRole: "super_admin"` at the service-contract boundary. No public UI/API route was added in this task.

### Delivery State Machine

Outbox lifecycle:

```text
pending -> claimed -> processing -> completed
pending -> claimed -> processing -> retry_scheduled
pending -> claimed -> processing -> dead_letter
claimed / processing -> lease expired -> claimable again
```

Checkpoint lifecycle remains:

```text
pending -> processing -> completed
pending -> processing -> retry_scheduled
pending -> processing -> skipped_unsupported_version
pending -> processing -> dead_letter
```

Completed checkpoints are not rerun when another consumer fails.

### Worker Startup Strategy

Current strategy:

- no automatic worker startup in Next.js request/build/test paths
- dedicated process/service should instantiate `ProjectionOutboxWorker` and call `start()`
- scheduled CLI drain can call `drainOnce()` for UAT or operational repair

This avoids duplicate unmanaged loops during hot reload, build, migrations, tests, or serverless request execution.

### SLA Baseline

Initial non-binding targets:

- event available after commit: immediate
- polling interval: 1-5 seconds, default 3 seconds
- normal Activity projection delivery: under 10 seconds
- retry scheduling accuracy: within one polling interval
- stale claim recovery: lease duration plus one polling interval
- worker heartbeat/health integration: future operational surface

## Compatibility Notes

- Activity events are not emitted through both outbox and in-process publisher in production service paths.
- Existing in-process publisher remains available for tests and explicitly approved compatibility paths.
- Existing Dashboard, Report, Notification, Approval, Follow-up, and recent-activity behavior remains unchanged.
- No final Timeline, Calendar, Notification expansion, My Day, Manager Workspace, Executive Workspace, Forecast, or Relationship Health feature was implemented.

## Verification

- `node --disable-warning=ExperimentalWarning --disable-warning=MODULE_TYPELESS_PACKAGE_JSON --experimental-strip-types --test src/features/foundation/business-events/*.test.ts src/features/foundation/projections/*.test.ts src/features/crm/activities/server/business-events.test.ts`
- `npx oxlint src/features/foundation/projections src/features/crm/activities/server/business-events.ts src/features/crm/activities/server/repository.ts src/features/crm/activities/server/service.ts src/db/schema.ts`
- `npm run db:generate`

## Typecheck Note

`npm run typecheck` is currently blocked by generated `.next/dev/types/routes.d.ts` syntax errors unrelated to the EP.1.4.1 source changes. Targeted runtime tests and scoped lint passed for the changed files.

## Residual Risks / Follow-up

- Worker process entrypoint/CLI command is not yet wired into `package.json`; runtime modules are ready for a dedicated process or scheduled drain task.
- Manual retry/dead-letter APIs are intentionally not exposed publicly yet.
- Health snapshots are recorded by runtime; a user-facing operational health page remains future work.
- Future source domains must adopt the same transaction-aware publisher through dedicated cutover tasks.