alla-allaos-fullstack/docs/implementation/task-ep1.4-projection-foundation-delivery-reliability-2026-07-13.md

Task EP.1.4 Projection Foundation & Delivery Reliability - 2026-07-13

Scope

• introduced persistent Business Event outbox schema
• introduced persistent projection consumer checkpoint, dead-letter, rebuild-run, and health snapshot schema
• added shared projection runtime contracts and processing runtime
• added outbox publisher contract that persists events without running projection consumers synchronously
• added governed retry policy and sanitized failure model
• added machine-readable Projection Registry for Timeline, Calendar, Dashboard, Notification, My Day, Manager Workspace, Executive Workspace, Relationship Health, and Forecast
• added no-op skeleton consumers for Timeline, Calendar, Dashboard, and Notification
• added hybrid rebuild contract and projection delivery, failure, and security matrices
• preserved current Dashboard, Report, Notification, Approval, Follow-up, Activity, and recent-activity behavior

Review Summary

Reviewed before and during implementation:

• AGENTS.md
• plans/task-ep.1.4.md
• docs/standards/engineering-constitution.md
• docs/standards/project-foundations.md
• docs/standards/architecture-rules.md
• docs/standards/task-review-checklist.md
• docs/standards/task-catalog.md
• docs/security/crm-authorization-boundaries.md
• docs/business/relationship-sales-workspace-blueprint-v1.md
• docs/adr/0014-crm-multi-role-user-assignment.md
• docs/adr/0017-report-foundation.md
• docs/implementation/task-ep1.2-activity-integration-legacy-followup-adapter-2026-07-13.md
• docs/implementation/task-ep1.3-business-event-foundation-2026-07-13.md
• existing src/features/foundation/business-events/**
• existing Activity business event adapter
• existing notification event and delivery schema

Delivery Semantics Decision

Selected strategy: Transactional Outbox foundation.

Source-domain services should persist the source mutation and Business Event outbox record in the same database transaction. Projection consumers process only committed outbox events. Projection failures are recorded in checkpoints, retries, and dead letters; they must not cause a successful source mutation to be reported as failed.

Event persistence failure inside the source transaction remains mutation-failing and rollback-worthy. Consumer failure after commit is projection technical state, not source-domain lifecycle state.

Implementation Summary

Persistent Delivery Model

Added Drizzle schema and migration for:

• business_event_outbox
• projection_consumer_checkpoints
• projection_dead_letters
• projection_rebuild_runs
• projection_health_snapshots

Generated migration:

• drizzle/0003_green_squadron_supreme.sql

Projection Runtime

Added src/features/foundation/projections/**.

Runtime behavior:

• persists Business Events as delivery records
• supports outbox publisher source path where publish means durable enqueue, not synchronous projection execution
• resolves registered consumers by event type
• validates supported schema versions
• records persistent checkpoint state by consumerName + eventId
• skips duplicate completed checkpoints
• isolates consumer failures
• schedules retry for retryable failures
• creates dead letters for non-retryable or max-attempt failures
• records projection health snapshots without storing unrestricted business payloads

Retry Policy

Frozen default policy:

• attempt 1: immediate
• attempt 2: +1 minute
• attempt 3: +5 minutes
• attempt 4: +30 minutes
• attempt 5: dead letter

Failures are sanitized to code, bounded message, and retryable.

Projection Registry

Machine-readable registry added for:

• Timeline
• Calendar
• Dashboard
• Notification
• My Day
• Manager Workspace
• Executive Workspace
• Relationship Health
• Forecast

Timeline, Calendar, Dashboard, and Notification have no-op skeleton consumers for runtime validation only. No final business-facing projection tables or UI were introduced.

Rebuild Contract

Added ProjectionRebuildRequest and plan creation helper.

Frozen hybrid strategy:

• Initial build: governed source queries plus legacy adapters
• Incremental updates: Business Events through projection runtime checkpoints
• Recovery: source rebuild plus event checkpoint reconciliation
• Reset: explicit resetExisting only and organization-scoped

Matrices

Added code-owned matrices:

• PROJECTION_DELIVERY_MATRIX
• PROJECTION_FAILURE_MATRIX
• PROJECTION_SECURITY_MATRIX

These freeze delivery reliability, failure behavior, and security expectations for future projection tasks.

Compatibility Notes

• Existing in-process Business Event publisher remains available.
• Existing notification inbox and approval notification behavior remain unchanged.
• Existing Dashboard and Report datasets remain unchanged.
• Existing follow-up APIs and Activity APIs remain unchanged.
• No final Timeline, Calendar, My Day, Manager Workspace, Executive Workspace, Relationship Health, Forecast, or dashboard replacement was introduced.

Security Notes

• Projection persistence is organization-scoped.
• Dead-letter records store sanitized failure metadata only.
• Projection consumers must re-query source services or security-aware modules when full detail is needed.
• Internal-only and pricing-sensitive payload handling remains governed by source event payload minimization plus future projection-specific security enforcement.
• Manager, Executive, Forecast, Dashboard, and Relationship Health projections must preserve resolved CRM access, branch/product scope, ownership, and pricing visibility boundaries.

Verification

• npm run typecheck
• node --disable-warning=ExperimentalWarning --disable-warning=MODULE_TYPELESS_PACKAGE_JSON --experimental-strip-types --test src/features/foundation/projections/*.test.ts
• npm run db:generate
• npx oxlint src/features/foundation/projections src/db/schema.ts

Residual Risks / Follow-up

• Source services are not fully refactored to call the outbox publisher inside source transactions in this slice. Future source-domain cutovers must use the outbox transaction pattern before relying on production replay guarantees.
• Drizzle store is foundation-ready, but no polling worker or admin retry route is exposed yet.
• Projection health snapshots are written as append-only snapshots in this slice; operational dashboards can later compact or upsert by consumer.
• EP.1.5 Timeline Projection Foundation can now focus on timeline read-model rules rather than re-solving delivery reliability.