task ep.1.4.1

2026-07-13 13:08:00 +07:00
parent 06af36156f
commit bba4c8d97d
18 changed files with 9388 additions and 192 deletions
--- a/docs/implementation/task-ep1.4.1-transactional-outbox-activation-worker-runtime-2026-07-13.md
+++ b/docs/implementation/task-ep1.4.1-transactional-outbox-activation-worker-runtime-2026-07-13.md
@@ -0,0 +1,172 @@
+# Task EP.1.4.1 Transactional Outbox Activation & Worker Runtime - 2026-07-13
+
+## Scope
+
+- activated Activity Business Event publication through transaction-aware outbox enqueue
+- added lease-based outbox claiming fields to `business_event_outbox`
+- added projection outbox worker runtime
+- added manual projection operations service contracts
+- added PostgreSQL `FOR UPDATE SKIP LOCKED` claim strategy in Drizzle store
+- preserved existing in-process Business Event publisher for compatibility and tests
+- preserved current Activity API response contracts, follow-up APIs, dashboard/report datasets, notifications, approvals, and recent-activity behavior
+
+## Review Summary
+
+Reviewed before and during implementation:
+
+- `AGENTS.md`
+- `plans/task-ep.1.4.1.md`
+- `docs/implementation/task-ep1.3-business-event-foundation-2026-07-13.md`
+- `docs/implementation/task-ep1.4-projection-foundation-delivery-reliability-2026-07-13.md`
+- `docs/standards/engineering-constitution.md`
+- `docs/standards/project-foundations.md`
+- `docs/standards/architecture-rules.md`
+- `docs/standards/task-review-checklist.md`
+- `docs/security/crm-authorization-boundaries.md`
+- existing Activity service mutation paths
+- existing Business Event publisher and projection runtime
+
+## Implementation Summary
+
+### Activity Transactional Outbox Adoption
+
+Activity mutation paths now enqueue Business Events to the outbox inside the same `db.transaction()` as the Activity row mutation:
+
+- create
+- update
+- assign/reassign
+- reschedule
+- complete
+- cancel
+- delete
+
+The Activity API still returns hydrated Activity records after commit. Projection consumers are not run synchronously from Activity source mutations.
+
+### Transaction-Aware Publisher
+
+Added:
+
+- `src/features/foundation/projections/server/transactional-publisher.ts`
+
+The publisher validates registry and payload contracts, then inserts into `business_event_outbox` using the supplied transaction/client. This prevents accidental double-publication through the in-process dispatcher.
+
+### Worker Runtime
+
+Added:
+
+- `src/features/foundation/projections/worker.ts`
+
+Worker behavior:
+
+- claims available events in batches
+- applies a lease with `claimedBy`, `claimedAt`, and `claimExpiresAt`
+- dispatches claimed committed events through `ProjectionConsumerRuntime`
+- does not auto-start unless `PROJECTION_WORKER_ENABLED=true`
+- supports graceful stop semantics for the in-process loop
+
+Default operational config:
+
+- polling interval: `3000ms`
+- batch size: `25`
+- max concurrent events: `1`
+- max concurrent consumers: `4`
+- claim lease: `60000ms`
+- stale claim: `90000ms`
+- graceful shutdown: `30000ms`
+
+### Multi-Instance Claiming
+
+Added schema fields:
+
+- `business_event_outbox.claimed_by`
+- `business_event_outbox.claimed_at`
+- `business_event_outbox.claim_expires_at`
+
+Generated migration:
+
+- `drizzle/0004_sharp_mercury.sql`
+
+Database store uses PostgreSQL-safe claiming with `FOR UPDATE SKIP LOCKED`. Expired `claimed` or `processing` leases are recoverable.
+
+### Manual Operations Contracts
+
+Added:
+
+- `src/features/foundation/projections/operations.ts`
+
+Contracts:
+
+- `drainOutbox`
+- `retryEvent`
+- `retryConsumer`
+- `retryDeadLetter`
+- `resolveDeadLetter`
+
+Manual operations require `systemRole: "super_admin"` at the service-contract boundary. No public UI/API route was added in this task.
+
+### Delivery State Machine
+
+Outbox lifecycle:
+
+```text
+pending -> claimed -> processing -> completed
+pending -> claimed -> processing -> retry_scheduled
+pending -> claimed -> processing -> dead_letter
+claimed / processing -> lease expired -> claimable again
+```
+
+Checkpoint lifecycle remains:
+
+```text
+pending -> processing -> completed
+pending -> processing -> retry_scheduled
+pending -> processing -> skipped_unsupported_version
+pending -> processing -> dead_letter
+```
+
+Completed checkpoints are not rerun when another consumer fails.
+
+### Worker Startup Strategy
+
+Current strategy:
+
+- no automatic worker startup in Next.js request/build/test paths
+- dedicated process/service should instantiate `ProjectionOutboxWorker` and call `start()`
+- scheduled CLI drain can call `drainOnce()` for UAT or operational repair
+
+This avoids duplicate unmanaged loops during hot reload, build, migrations, tests, or serverless request execution.
+
+### SLA Baseline
+
+Initial non-binding targets:
+
+- event available after commit: immediate
+- polling interval: 1-5 seconds, default 3 seconds
+- normal Activity projection delivery: under 10 seconds
+- retry scheduling accuracy: within one polling interval
+- stale claim recovery: lease duration plus one polling interval
+- worker heartbeat/health integration: future operational surface
+
+## Compatibility Notes
+
+- Activity events are not emitted through both outbox and in-process publisher in production service paths.
+- Existing in-process publisher remains available for tests and explicitly approved compatibility paths.
+- Existing Dashboard, Report, Notification, Approval, Follow-up, and recent-activity behavior remains unchanged.
+- No final Timeline, Calendar, Notification expansion, My Day, Manager Workspace, Executive Workspace, Forecast, or Relationship Health feature was implemented.
+
+## Verification
+
+- `node --disable-warning=ExperimentalWarning --disable-warning=MODULE_TYPELESS_PACKAGE_JSON --experimental-strip-types --test src/features/foundation/business-events/*.test.ts src/features/foundation/projections/*.test.ts src/features/crm/activities/server/business-events.test.ts`
+- `npx oxlint src/features/foundation/projections src/features/crm/activities/server/business-events.ts src/features/crm/activities/server/repository.ts src/features/crm/activities/server/service.ts src/db/schema.ts`
+- `npm run db:generate`
+
+## Typecheck Note
+
+`npm run typecheck` is currently blocked by generated `.next/dev/types/routes.d.ts` syntax errors unrelated to the EP.1.4.1 source changes. Targeted runtime tests and scoped lint passed for the changed files.
+
+## Residual Risks / Follow-up
+
+- Worker process entrypoint/CLI command is not yet wired into `package.json`; runtime modules are ready for a dedicated process or scheduled drain task.
+- Manual retry/dead-letter APIs are intentionally not exposed publicly yet.
+- Health snapshots are recorded by runtime; a user-facing operational health page remains future work.
+- Future source domains must adopt the same transaction-aware publisher through dedicated cutover tasks.