task ep.1.4
This commit is contained in:
phaichayon
@@ -0,0 +1,154 @@
|
||||
# Task EP.1.4 Projection Foundation & Delivery Reliability - 2026-07-13
|
||||
|
||||
## Scope
|
||||
|
||||
- introduced persistent Business Event outbox schema
|
||||
- introduced persistent projection consumer checkpoint, dead-letter, rebuild-run, and health snapshot schema
|
||||
- added shared projection runtime contracts and processing runtime
|
||||
- added outbox publisher contract that persists events without running projection consumers synchronously
|
||||
- added governed retry policy and sanitized failure model
|
||||
- added machine-readable Projection Registry for Timeline, Calendar, Dashboard, Notification, My Day, Manager Workspace, Executive Workspace, Relationship Health, and Forecast
|
||||
- added no-op skeleton consumers for Timeline, Calendar, Dashboard, and Notification
|
||||
- added hybrid rebuild contract and projection delivery, failure, and security matrices
|
||||
- preserved current Dashboard, Report, Notification, Approval, Follow-up, Activity, and recent-activity behavior
|
||||
|
||||
## Review Summary
|
||||
|
||||
Reviewed before and during implementation:
|
||||
|
||||
- `AGENTS.md`
|
||||
- `plans/task-ep.1.4.md`
|
||||
- `docs/standards/engineering-constitution.md`
|
||||
- `docs/standards/project-foundations.md`
|
||||
- `docs/standards/architecture-rules.md`
|
||||
- `docs/standards/task-review-checklist.md`
|
||||
- `docs/standards/task-catalog.md`
|
||||
- `docs/security/crm-authorization-boundaries.md`
|
||||
- `docs/business/relationship-sales-workspace-blueprint-v1.md`
|
||||
- `docs/adr/0014-crm-multi-role-user-assignment.md`
|
||||
- `docs/adr/0017-report-foundation.md`
|
||||
- `docs/implementation/task-ep1.2-activity-integration-legacy-followup-adapter-2026-07-13.md`
|
||||
- `docs/implementation/task-ep1.3-business-event-foundation-2026-07-13.md`
|
||||
- existing `src/features/foundation/business-events/**`
|
||||
- existing Activity business event adapter
|
||||
- existing notification event and delivery schema
|
||||
|
||||
## Delivery Semantics Decision
|
||||
|
||||
Selected strategy: Transactional Outbox foundation.
|
||||
|
||||
Source-domain services should persist the source mutation and Business Event outbox record in the same database transaction. Projection consumers process only committed outbox events. Projection failures are recorded in checkpoints, retries, and dead letters; they must not cause a successful source mutation to be reported as failed.
|
||||
|
||||
Event persistence failure inside the source transaction remains mutation-failing and rollback-worthy. Consumer failure after commit is projection technical state, not source-domain lifecycle state.
|
||||
|
||||
## Implementation Summary
|
||||
|
||||
### Persistent Delivery Model
|
||||
|
||||
Added Drizzle schema and migration for:
|
||||
|
||||
- `business_event_outbox`
|
||||
- `projection_consumer_checkpoints`
|
||||
- `projection_dead_letters`
|
||||
- `projection_rebuild_runs`
|
||||
- `projection_health_snapshots`
|
||||
|
||||
Generated migration:
|
||||
|
||||
- `drizzle/0003_green_squadron_supreme.sql`
|
||||
|
||||
### Projection Runtime
|
||||
|
||||
Added `src/features/foundation/projections/**`.
|
||||
|
||||
Runtime behavior:
|
||||
|
||||
- persists Business Events as delivery records
|
||||
- supports outbox publisher source path where publish means durable enqueue, not synchronous projection execution
|
||||
- resolves registered consumers by event type
|
||||
- validates supported schema versions
|
||||
- records persistent checkpoint state by `consumerName + eventId`
|
||||
- skips duplicate completed checkpoints
|
||||
- isolates consumer failures
|
||||
- schedules retry for retryable failures
|
||||
- creates dead letters for non-retryable or max-attempt failures
|
||||
- records projection health snapshots without storing unrestricted business payloads
|
||||
|
||||
### Retry Policy
|
||||
|
||||
Frozen default policy:
|
||||
|
||||
- attempt 1: immediate
|
||||
- attempt 2: +1 minute
|
||||
- attempt 3: +5 minutes
|
||||
- attempt 4: +30 minutes
|
||||
- attempt 5: dead letter
|
||||
|
||||
Failures are sanitized to `code`, bounded `message`, and `retryable`.
|
||||
|
||||
### Projection Registry
|
||||
|
||||
Machine-readable registry added for:
|
||||
|
||||
- Timeline
|
||||
- Calendar
|
||||
- Dashboard
|
||||
- Notification
|
||||
- My Day
|
||||
- Manager Workspace
|
||||
- Executive Workspace
|
||||
- Relationship Health
|
||||
- Forecast
|
||||
|
||||
Timeline, Calendar, Dashboard, and Notification have no-op skeleton consumers for runtime validation only. No final business-facing projection tables or UI were introduced.
|
||||
|
||||
### Rebuild Contract
|
||||
|
||||
Added `ProjectionRebuildRequest` and plan creation helper.
|
||||
|
||||
Frozen hybrid strategy:
|
||||
|
||||
- Initial build: governed source queries plus legacy adapters
|
||||
- Incremental updates: Business Events through projection runtime checkpoints
|
||||
- Recovery: source rebuild plus event checkpoint reconciliation
|
||||
- Reset: explicit `resetExisting` only and organization-scoped
|
||||
|
||||
### Matrices
|
||||
|
||||
Added code-owned matrices:
|
||||
|
||||
- `PROJECTION_DELIVERY_MATRIX`
|
||||
- `PROJECTION_FAILURE_MATRIX`
|
||||
- `PROJECTION_SECURITY_MATRIX`
|
||||
|
||||
These freeze delivery reliability, failure behavior, and security expectations for future projection tasks.
|
||||
|
||||
## Compatibility Notes
|
||||
|
||||
- Existing in-process Business Event publisher remains available.
|
||||
- Existing notification inbox and approval notification behavior remain unchanged.
|
||||
- Existing Dashboard and Report datasets remain unchanged.
|
||||
- Existing follow-up APIs and Activity APIs remain unchanged.
|
||||
- No final Timeline, Calendar, My Day, Manager Workspace, Executive Workspace, Relationship Health, Forecast, or dashboard replacement was introduced.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- Projection persistence is organization-scoped.
|
||||
- Dead-letter records store sanitized failure metadata only.
|
||||
- Projection consumers must re-query source services or security-aware modules when full detail is needed.
|
||||
- Internal-only and pricing-sensitive payload handling remains governed by source event payload minimization plus future projection-specific security enforcement.
|
||||
- Manager, Executive, Forecast, Dashboard, and Relationship Health projections must preserve resolved CRM access, branch/product scope, ownership, and pricing visibility boundaries.
|
||||
|
||||
## Verification
|
||||
|
||||
- `npm run typecheck`
|
||||
- `node --disable-warning=ExperimentalWarning --disable-warning=MODULE_TYPELESS_PACKAGE_JSON --experimental-strip-types --test src/features/foundation/projections/*.test.ts`
|
||||
- `npm run db:generate`
|
||||
- `npx oxlint src/features/foundation/projections src/db/schema.ts`
|
||||
|
||||
## Residual Risks / Follow-up
|
||||
|
||||
- Source services are not fully refactored to call the outbox publisher inside source transactions in this slice. Future source-domain cutovers must use the outbox transaction pattern before relying on production replay guarantees.
|
||||
- Drizzle store is foundation-ready, but no polling worker or admin retry route is exposed yet.
|
||||
- Projection health snapshots are written as append-only snapshots in this slice; operational dashboards can later compact or upsert by consumer.
|
||||
- EP.1.5 Timeline Projection Foundation can now focus on timeline read-model rules rather than re-solving delivery reliability.
|
||||
117
drizzle/0003_green_squadron_supreme.sql
Normal file
117
drizzle/0003_green_squadron_supreme.sql
Normal file
@@ -0,0 +1,117 @@
|
||||
CREATE TABLE "business_event_outbox" (
|
||||
"id" text PRIMARY KEY NOT NULL,
|
||||
"organization_id" text NOT NULL,
|
||||
"event_type" text NOT NULL,
|
||||
"schema_version" integer NOT NULL,
|
||||
"entity_type" text NOT NULL,
|
||||
"entity_id" text NOT NULL,
|
||||
"correlation_id" text NOT NULL,
|
||||
"causation_id" text,
|
||||
"event_envelope" jsonb NOT NULL,
|
||||
"delivery_status" text DEFAULT 'pending' NOT NULL,
|
||||
"attempt_count" integer DEFAULT 0 NOT NULL,
|
||||
"available_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"processing_started_at" timestamp with time zone,
|
||||
"dispatched_at" timestamp with time zone,
|
||||
"failed_at" timestamp with time zone,
|
||||
"dead_lettered_at" timestamp with time zone,
|
||||
"last_error_code" text,
|
||||
"last_error_message" text,
|
||||
"occurred_at" timestamp with time zone NOT NULL,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE "projection_consumer_checkpoints" (
|
||||
"id" text PRIMARY KEY NOT NULL,
|
||||
"organization_id" text NOT NULL,
|
||||
"consumer_name" text NOT NULL,
|
||||
"projection_name" text NOT NULL,
|
||||
"event_id" text NOT NULL,
|
||||
"event_type" text NOT NULL,
|
||||
"schema_version" integer NOT NULL,
|
||||
"processing_status" text DEFAULT 'pending' NOT NULL,
|
||||
"attempt_count" integer DEFAULT 0 NOT NULL,
|
||||
"first_attempted_at" timestamp with time zone,
|
||||
"last_attempted_at" timestamp with time zone,
|
||||
"completed_at" timestamp with time zone,
|
||||
"next_retry_at" timestamp with time zone,
|
||||
"error_code" text,
|
||||
"error_message" text,
|
||||
"processing_duration_ms" integer,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE "projection_dead_letters" (
|
||||
"id" text PRIMARY KEY NOT NULL,
|
||||
"organization_id" text NOT NULL,
|
||||
"consumer_name" text NOT NULL,
|
||||
"projection_name" text NOT NULL,
|
||||
"event_id" text NOT NULL,
|
||||
"event_type" text NOT NULL,
|
||||
"schema_version" integer NOT NULL,
|
||||
"attempt_count" integer NOT NULL,
|
||||
"error_code" text NOT NULL,
|
||||
"error_message" text NOT NULL,
|
||||
"failure_metadata" jsonb,
|
||||
"status" text DEFAULT 'open' NOT NULL,
|
||||
"resolved_at" timestamp with time zone,
|
||||
"resolved_by" text,
|
||||
"resolution_reason" text,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE "projection_health_snapshots" (
|
||||
"id" text PRIMARY KEY NOT NULL,
|
||||
"organization_id" text NOT NULL,
|
||||
"projection_name" text NOT NULL,
|
||||
"consumer_name" text NOT NULL,
|
||||
"status" text DEFAULT 'healthy' NOT NULL,
|
||||
"last_processed_event_id" text,
|
||||
"last_successful_processing_at" timestamp with time zone,
|
||||
"pending_count" integer DEFAULT 0 NOT NULL,
|
||||
"retry_count" integer DEFAULT 0 NOT NULL,
|
||||
"dead_letter_count" integer DEFAULT 0 NOT NULL,
|
||||
"unsupported_version_count" integer DEFAULT 0 NOT NULL,
|
||||
"processing_lag_ms" integer DEFAULT 0 NOT NULL,
|
||||
"average_processing_duration_ms" integer DEFAULT 0 NOT NULL,
|
||||
"last_rebuild_at" timestamp with time zone,
|
||||
"rebuild_source" text,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE "projection_rebuild_runs" (
|
||||
"id" text PRIMARY KEY NOT NULL,
|
||||
"organization_id" text NOT NULL,
|
||||
"projection_name" text NOT NULL,
|
||||
"entity_type" text,
|
||||
"entity_id" text,
|
||||
"occurred_from" timestamp with time zone,
|
||||
"occurred_to" timestamp with time zone,
|
||||
"dry_run" boolean DEFAULT true NOT NULL,
|
||||
"reset_existing" boolean DEFAULT false NOT NULL,
|
||||
"source_strategy" text NOT NULL,
|
||||
"status" text DEFAULT 'pending' NOT NULL,
|
||||
"processed_count" integer DEFAULT 0 NOT NULL,
|
||||
"skipped_count" integer DEFAULT 0 NOT NULL,
|
||||
"failed_count" integer DEFAULT 0 NOT NULL,
|
||||
"error_code" text,
|
||||
"error_message" text,
|
||||
"requested_by" text NOT NULL,
|
||||
"started_at" timestamp with time zone,
|
||||
"completed_at" timestamp with time zone,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE INDEX "business_event_outbox_status_available_idx" ON "business_event_outbox" USING btree ("delivery_status","available_at");--> statement-breakpoint
|
||||
CREATE INDEX "business_event_outbox_org_event_idx" ON "business_event_outbox" USING btree ("organization_id","event_type");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "projection_checkpoints_consumer_event_idx" ON "projection_consumer_checkpoints" USING btree ("consumer_name","event_id");--> statement-breakpoint
|
||||
CREATE INDEX "projection_checkpoints_status_retry_idx" ON "projection_consumer_checkpoints" USING btree ("processing_status","next_retry_at");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "projection_dead_letters_consumer_event_idx" ON "projection_dead_letters" USING btree ("consumer_name","event_id");--> statement-breakpoint
|
||||
CREATE INDEX "projection_dead_letters_org_status_idx" ON "projection_dead_letters" USING btree ("organization_id","status");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "projection_health_org_consumer_idx" ON "projection_health_snapshots" USING btree ("organization_id","consumer_name");--> statement-breakpoint
|
||||
CREATE INDEX "projection_rebuild_runs_org_projection_idx" ON "projection_rebuild_runs" USING btree ("organization_id","projection_name");
|
||||
7493
drizzle/meta/0003_snapshot.json
Normal file
7493
drizzle/meta/0003_snapshot.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -22,6 +22,13 @@
|
||||
"when": 1783647090217,
|
||||
"tag": "0002_empty_jean_grey",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 3,
|
||||
"version": "7",
|
||||
"when": 1783917011479,
|
||||
"tag": "0003_green_squadron_supreme",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
971
plans/task-ep.1.4.md
Normal file
971
plans/task-ep.1.4.md
Normal file
@@ -0,0 +1,971 @@
|
||||
# Task EP.1.4 – Projection Foundation & Delivery Reliability
|
||||
|
||||
Status: Completed
|
||||
Priority: Critical
|
||||
Type: Platform Foundation / Projection Runtime / Delivery Reliability
|
||||
|
||||
## Depends On
|
||||
|
||||
* EP.1.1 Activity Domain Foundation
|
||||
* EP.1.2 Activity Integration & Legacy Follow-up Adapter
|
||||
* EP.1.3 Business Event Foundation
|
||||
* BU-R.0 Business Blueprint Freeze
|
||||
* BU-R.0.1 Workspace & Activity Business Blueprint Completion
|
||||
* BU-R.1 Business Capability Audit
|
||||
* AR.1 Architecture Transition Plan
|
||||
* AR.2 Epic & Technical Design
|
||||
* ENG.0 Engineering Constitution
|
||||
|
||||
---
|
||||
|
||||
# Objective
|
||||
|
||||
Introduce the shared Projection Foundation and reliable Business Event delivery model for ALLA OS.
|
||||
|
||||
This task establishes the runtime required for future Timeline, Calendar, Dashboard, Notification, My Day, Manager Workspace, Executive Workspace, Forecast, and Relationship Health projections.
|
||||
|
||||
The task must ensure that:
|
||||
|
||||
* successful source-domain mutations are not reported as failed merely because a projection consumer fails
|
||||
* projection consumers process events idempotently
|
||||
* failed projection processing is observable and retryable
|
||||
* projections can be rebuilt from governed source data
|
||||
* projection state remains derived and non-authoritative
|
||||
* current production CRM behavior remains backward compatible
|
||||
|
||||
This task does not implement final Timeline, Calendar, My Day, Manager Workspace, Executive Workspace, or dashboard replacement features.
|
||||
|
||||
---
|
||||
|
||||
# Background
|
||||
|
||||
EP.1.3 introduced:
|
||||
|
||||
* canonical `BusinessEvent`
|
||||
* machine-readable Event Registry
|
||||
* event ownership rules
|
||||
* in-process publisher and dispatcher
|
||||
* Activity lifecycle event publishing
|
||||
* Event Sequence Matrix
|
||||
* Event Consumer Matrix
|
||||
* replay contracts
|
||||
* versioning and idempotency contracts
|
||||
|
||||
The remaining reliability gaps are:
|
||||
|
||||
1. subscriber failure may currently cause publisher dispatch to fail after the source mutation has already succeeded
|
||||
2. idempotency is in-memory and does not survive process restart or multi-instance deployment
|
||||
3. no persistent event delivery or projection processing state exists
|
||||
4. replay is contract-only because there is no persistent event source
|
||||
5. projections do not yet have a shared runtime, checkpoint, rebuild, retry, or health model
|
||||
|
||||
EP.1.4 resolves these gaps before business-facing projection features are implemented.
|
||||
|
||||
---
|
||||
|
||||
# Architecture Principles
|
||||
|
||||
## 1. Source Mutation Independence
|
||||
|
||||
A committed source-domain mutation must not be rolled back or reported as unsuccessful solely because a non-owning projection consumer failed.
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
Activity completed successfully
|
||||
↓
|
||||
activity.completed emitted
|
||||
↓
|
||||
Timeline projection fails
|
||||
↓
|
||||
Activity remains completed
|
||||
Projection failure is recorded for retry
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Eventual Consistency
|
||||
|
||||
Timeline, Calendar, Dashboard, Notification, My Day, Forecast, and other projections are eventually consistent read models.
|
||||
|
||||
They must not become authoritative owners of source-domain lifecycle state.
|
||||
|
||||
---
|
||||
|
||||
## 3. Persistent Idempotency
|
||||
|
||||
Projection consumers must prevent duplicate side effects across:
|
||||
|
||||
* retries
|
||||
* process restarts
|
||||
* multi-instance deployment
|
||||
* repeated event delivery
|
||||
* manual rebuild operations
|
||||
|
||||
In-memory duplicate detection is insufficient for production projection consumers.
|
||||
|
||||
---
|
||||
|
||||
## 4. Rebuildable Projections
|
||||
|
||||
Every projection must be reproducible from governed source data, persisted business events, or an approved hybrid strategy.
|
||||
|
||||
Deleting or rebuilding a projection must not delete operational business records.
|
||||
|
||||
---
|
||||
|
||||
## 5. Preserve Before Replace
|
||||
|
||||
Current dashboard, report, follow-up, approval, notification, and detail-page behavior remains operational.
|
||||
|
||||
Projection consumers are introduced additively before any existing consumer is replaced.
|
||||
|
||||
---
|
||||
|
||||
## 6. Security at Projection Boundaries
|
||||
|
||||
Projection reads and writes must preserve:
|
||||
|
||||
* organization isolation
|
||||
* branch and product scope
|
||||
* CRM record visibility
|
||||
* internal-only visibility
|
||||
* pricing-sensitive redaction
|
||||
* source-domain authorization boundaries
|
||||
|
||||
Projection storage must not become a route around source security rules.
|
||||
|
||||
---
|
||||
|
||||
# Review Required
|
||||
|
||||
## Governance
|
||||
|
||||
* `AGENTS.md`
|
||||
* `docs/standards/engineering-constitution.md`
|
||||
* `docs/standards/project-foundations.md`
|
||||
* `docs/standards/architecture-rules.md`
|
||||
* `docs/standards/task-review-checklist.md`
|
||||
* `docs/security/crm-authorization-boundaries.md`
|
||||
|
||||
## Business and Architecture
|
||||
|
||||
* Relationship & Sales Workspace Blueprint
|
||||
* BU-R.1 Business Capability Audit
|
||||
* AR.1 Architecture Transition Plan
|
||||
* AR.2 Epic & Technical Design
|
||||
|
||||
## Existing Implementation
|
||||
|
||||
* EP.1.1 Activity Domain Foundation
|
||||
* EP.1.2 Activity Integration & Legacy Follow-up Adapter
|
||||
* EP.1.3 Business Event Foundation
|
||||
* Business Event Registry
|
||||
* Event Sequence Matrix
|
||||
* Event Consumer Matrix
|
||||
* in-process dispatcher
|
||||
* Activity event publishers
|
||||
* current Audit Log foundation
|
||||
* current Notification foundation
|
||||
* current Dashboard and Report services
|
||||
* current Approval event behavior
|
||||
* existing database transaction patterns
|
||||
* current background-job or scheduled-task foundations, if any
|
||||
|
||||
---
|
||||
|
||||
# Scope
|
||||
|
||||
## Part 1 — Delivery Semantics
|
||||
|
||||
Freeze the official event delivery semantics.
|
||||
|
||||
### Required Rules
|
||||
|
||||
* Business mutation commits first.
|
||||
* Non-owning consumer failure must not undo a committed source mutation.
|
||||
* Event delivery status must be distinguishable from mutation status.
|
||||
* Consumers may be synchronous internally, but projection processing must follow post-commit semantics.
|
||||
* Source services must not return a misleading mutation failure after successful commit.
|
||||
* Critical synchronous side effects, if any, must be explicitly approved and documented.
|
||||
|
||||
### Required Decision
|
||||
|
||||
Select and document the initial delivery strategy:
|
||||
|
||||
* Transactional Outbox
|
||||
* Post-commit persistent delivery record
|
||||
* Hybrid source-query plus event checkpoint strategy
|
||||
* another approach explicitly justified against AR.1 and ENG.0
|
||||
|
||||
Preferred direction:
|
||||
|
||||
> Transactional Outbox or an equivalent persistent post-commit delivery mechanism.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Persistent Event Delivery Model
|
||||
|
||||
Introduce persistent delivery records for Business Events.
|
||||
|
||||
Recommended responsibilities:
|
||||
|
||||
* persist canonical event envelope
|
||||
* record publication state
|
||||
* record availability for dispatch
|
||||
* record dispatch attempts
|
||||
* record successful completion
|
||||
* record dead-letter or terminal failure
|
||||
* preserve correlation and causation metadata
|
||||
* support organization-scoped querying
|
||||
|
||||
Recommended conceptual states:
|
||||
|
||||
```text
|
||||
pending
|
||||
processing
|
||||
completed
|
||||
failed
|
||||
dead_letter
|
||||
```
|
||||
|
||||
### Rules
|
||||
|
||||
* Event records are immutable business facts after publication.
|
||||
* Delivery metadata may change independently.
|
||||
* Event payload must continue respecting data minimization and pricing sensitivity.
|
||||
* A persistent delivery record does not make Event storage the owner of source lifecycle state.
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Transactional Outbox Decision and Foundation
|
||||
|
||||
Assess and, if approved, implement the Outbox foundation.
|
||||
|
||||
### Outbox Requirements
|
||||
|
||||
* source mutation and event persistence occur in the same database transaction
|
||||
* dispatcher processes only committed events
|
||||
* retries are safe
|
||||
* duplicate delivery is expected and handled
|
||||
* outbox cleanup or retention is governed
|
||||
* events are organization scoped
|
||||
* no external broker is required
|
||||
|
||||
### Deliverables
|
||||
|
||||
* Outbox architecture decision
|
||||
* schema or storage contract
|
||||
* transaction integration pattern
|
||||
* dispatcher polling or draining strategy
|
||||
* failure recovery behavior
|
||||
* retention recommendation
|
||||
|
||||
If Outbox implementation is deferred, an equivalent persistent reliability mechanism must be approved and documented.
|
||||
|
||||
---
|
||||
|
||||
## Part 4 — Projection Consumer Runtime
|
||||
|
||||
Introduce the shared Projection Consumer Runtime.
|
||||
|
||||
Minimum contract:
|
||||
|
||||
```ts
|
||||
export interface ProjectionConsumer<TEvent extends BusinessEvent = BusinessEvent> {
|
||||
readonly consumerName: string;
|
||||
readonly supportedEventTypes: string[];
|
||||
readonly supportedVersions: Record<string, number[]>;
|
||||
|
||||
handle(event: TEvent): Promise<ProjectionHandleResult>;
|
||||
}
|
||||
```
|
||||
|
||||
Runtime responsibilities:
|
||||
|
||||
* resolve registered consumers
|
||||
* validate supported event version
|
||||
* acquire idempotency guard
|
||||
* execute consumer
|
||||
* record result
|
||||
* release or finalize checkpoint
|
||||
* record duration and failure details
|
||||
* schedule retry when appropriate
|
||||
|
||||
---
|
||||
|
||||
## Part 5 — Persistent Consumer Checkpoints
|
||||
|
||||
Introduce persistent consumer processing state.
|
||||
|
||||
Minimum uniqueness:
|
||||
|
||||
```text
|
||||
consumerName + eventId
|
||||
```
|
||||
|
||||
Recommended data:
|
||||
|
||||
* consumer name
|
||||
* event id
|
||||
* event type
|
||||
* schema version
|
||||
* organization id
|
||||
* processing status
|
||||
* attempt count
|
||||
* first attempted at
|
||||
* last attempted at
|
||||
* completed at
|
||||
* next retry at
|
||||
* error code
|
||||
* sanitized error message
|
||||
* processing duration
|
||||
|
||||
### Required States
|
||||
|
||||
```text
|
||||
pending
|
||||
processing
|
||||
completed
|
||||
retry_scheduled
|
||||
failed
|
||||
dead_letter
|
||||
skipped_unsupported_version
|
||||
```
|
||||
|
||||
### Rules
|
||||
|
||||
* completed checkpoints prevent duplicate projection effects
|
||||
* failed consumers may retry independently
|
||||
* one consumer failure must not block unrelated consumers permanently
|
||||
* checkpoints remain technical delivery state, not business lifecycle state
|
||||
|
||||
---
|
||||
|
||||
## Part 6 — Retry Policy
|
||||
|
||||
Freeze retry behavior.
|
||||
|
||||
Determine:
|
||||
|
||||
* retryable error categories
|
||||
* non-retryable error categories
|
||||
* maximum attempts
|
||||
* exponential or fixed backoff
|
||||
* jitter
|
||||
* retry scheduling
|
||||
* dead-letter threshold
|
||||
* manual retry support
|
||||
* organization-level safety limits
|
||||
|
||||
Recommended defaults must be documented rather than hidden in code.
|
||||
|
||||
### Example
|
||||
|
||||
```text
|
||||
attempt 1: immediate
|
||||
attempt 2: +1 minute
|
||||
attempt 3: +5 minutes
|
||||
attempt 4: +30 minutes
|
||||
attempt 5: dead letter
|
||||
```
|
||||
|
||||
Exact values must align with existing operational standards.
|
||||
|
||||
---
|
||||
|
||||
## Part 7 — Dead-Letter and Failure Records
|
||||
|
||||
Introduce a governed dead-letter model for projection failures that cannot be processed automatically.
|
||||
|
||||
Required capabilities:
|
||||
|
||||
* retain failed event reference
|
||||
* retain consumer name
|
||||
* retain sanitized failure information
|
||||
* retain attempt history
|
||||
* support manual review
|
||||
* support manual retry
|
||||
* support mark-resolved with reason
|
||||
* prevent sensitive payload exposure
|
||||
|
||||
No user-facing operations UI is required in this task unless explicitly approved.
|
||||
|
||||
An admin/service interface and documentation are sufficient.
|
||||
|
||||
---
|
||||
|
||||
## Part 8 — Projection Registry
|
||||
|
||||
Create the machine-readable Projection Registry.
|
||||
|
||||
Every projection definition must document:
|
||||
|
||||
* projection name
|
||||
* owning feature
|
||||
* consumed event types
|
||||
* supported versions
|
||||
* rebuild source
|
||||
* persistence strategy
|
||||
* security classification
|
||||
* idempotency key
|
||||
* retry policy
|
||||
* retention policy
|
||||
* implementation status
|
||||
|
||||
Minimum planned projections:
|
||||
|
||||
* Timeline
|
||||
* Calendar
|
||||
* Dashboard
|
||||
* Notification
|
||||
* My Day
|
||||
* Manager Workspace
|
||||
* Executive Workspace
|
||||
* Relationship Health
|
||||
* Forecast
|
||||
|
||||
### Example
|
||||
|
||||
```ts
|
||||
export type ProjectionDefinition = {
|
||||
projectionName: string;
|
||||
consumerName: string;
|
||||
consumedEventTypes: string[];
|
||||
supportedVersions: Record<string, number[]>;
|
||||
rebuildStrategy: "source-query" | "event-replay" | "hybrid";
|
||||
persistence: "query-time" | "cached-read-model" | "materialized";
|
||||
securityClassification: string;
|
||||
implementationStatus: "contract-only" | "active" | "deprecated";
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 9 — Projection Storage Standards
|
||||
|
||||
Freeze the allowed projection storage patterns.
|
||||
|
||||
Supported patterns:
|
||||
|
||||
### Query-Time Projection
|
||||
|
||||
* calculated from source domains when requested
|
||||
* no persisted read-model table
|
||||
* suitable for low-volume or early rollout
|
||||
|
||||
### Cached Read Model
|
||||
|
||||
* persisted derivative data
|
||||
* invalidated or updated by events
|
||||
* fully rebuildable
|
||||
|
||||
### Materialized Projection
|
||||
|
||||
* optimized projection table or materialized view
|
||||
* suitable for high-volume Timeline, Calendar, Dashboard, or reporting use cases
|
||||
* never authoritative
|
||||
|
||||
### Rules
|
||||
|
||||
* projection tables must include organization scope
|
||||
* projection data must be rebuildable
|
||||
* source IDs and event provenance must be retained
|
||||
* unique constraints must protect against duplicate projection rows
|
||||
* deletion of projection data must not delete source records
|
||||
* projection schema changes require rebuild compatibility analysis
|
||||
|
||||
---
|
||||
|
||||
## Part 10 — Initial Projection Skeletons
|
||||
|
||||
Create contracts and no-op or test projection consumers for:
|
||||
|
||||
* Timeline
|
||||
* Calendar
|
||||
* Notification
|
||||
* Dashboard
|
||||
|
||||
These consumers must validate:
|
||||
|
||||
* registration
|
||||
* version support
|
||||
* idempotency
|
||||
* checkpointing
|
||||
* retry behavior
|
||||
* consumer isolation
|
||||
|
||||
They must not yet implement business-facing projection behavior.
|
||||
|
||||
No final projection tables or UI are required.
|
||||
|
||||
---
|
||||
|
||||
## Part 11 — Hybrid Rebuild Strategy
|
||||
|
||||
Freeze the initial rebuild strategy.
|
||||
|
||||
Recommended strategy:
|
||||
|
||||
```text
|
||||
Initial build
|
||||
↓
|
||||
Governed source queries + legacy adapters
|
||||
|
||||
Incremental updates
|
||||
↓
|
||||
Business Events
|
||||
|
||||
Recovery
|
||||
↓
|
||||
Source rebuild + event checkpoint reconciliation
|
||||
```
|
||||
|
||||
This hybrid strategy is preferred because:
|
||||
|
||||
* legacy follow-up events were not historically persisted
|
||||
* current source tables remain authoritative
|
||||
* Activity and new domain events can support incremental projection updates
|
||||
* migration risk remains controlled
|
||||
|
||||
### Required Outputs
|
||||
|
||||
* source-query rebuild contract
|
||||
* event-based incremental contract
|
||||
* legacy adapter role
|
||||
* reconciliation rules
|
||||
* cutover criteria
|
||||
* rollback behavior
|
||||
|
||||
---
|
||||
|
||||
## Part 12 — Projection Rebuild Contract
|
||||
|
||||
Implement or define a shared rebuild contract.
|
||||
|
||||
Suggested interface:
|
||||
|
||||
```ts
|
||||
export type ProjectionRebuildRequest = {
|
||||
projectionName: string;
|
||||
organizationId: string;
|
||||
entityType?: string;
|
||||
entityId?: string;
|
||||
occurredFrom?: string;
|
||||
occurredTo?: string;
|
||||
dryRun?: boolean;
|
||||
resetExisting?: boolean;
|
||||
};
|
||||
```
|
||||
|
||||
Required behavior:
|
||||
|
||||
* organization scoped
|
||||
* permission protected
|
||||
* dry-run supported
|
||||
* progress report supported
|
||||
* idempotent
|
||||
* restartable
|
||||
* source strategy recorded
|
||||
* existing projection reset explicitly controlled
|
||||
|
||||
No public UI is required.
|
||||
|
||||
---
|
||||
|
||||
## Part 13 — Projection Health Model
|
||||
|
||||
Define projection operational health.
|
||||
|
||||
Suggested statuses:
|
||||
|
||||
```text
|
||||
healthy
|
||||
lagging
|
||||
degraded
|
||||
failed
|
||||
rebuilding
|
||||
paused
|
||||
```
|
||||
|
||||
Track:
|
||||
|
||||
* last processed event
|
||||
* last successful processing time
|
||||
* pending count
|
||||
* retry count
|
||||
* dead-letter count
|
||||
* unsupported-version count
|
||||
* processing lag
|
||||
* average processing duration
|
||||
* last rebuild time
|
||||
* rebuild source
|
||||
|
||||
Projection health must be observable without exposing sensitive business payloads.
|
||||
|
||||
---
|
||||
|
||||
## Part 14 — Delivery Observability
|
||||
|
||||
Add structured observability for:
|
||||
|
||||
* event persisted
|
||||
* event ready
|
||||
* dispatch started
|
||||
* dispatch completed
|
||||
* consumer started
|
||||
* consumer completed
|
||||
* consumer retry scheduled
|
||||
* consumer failed
|
||||
* dead-letter created
|
||||
* duplicate skipped
|
||||
* unsupported version skipped
|
||||
* projection rebuild started
|
||||
* projection rebuild completed
|
||||
* projection rebuild failed
|
||||
|
||||
Required metadata:
|
||||
|
||||
* event id
|
||||
* event type
|
||||
* schema version
|
||||
* organization id
|
||||
* correlation id
|
||||
* causation id
|
||||
* consumer name
|
||||
* projection name
|
||||
* attempt
|
||||
* duration
|
||||
* status
|
||||
* sanitized error code
|
||||
|
||||
---
|
||||
|
||||
## Part 15 — Security and Data Redaction
|
||||
|
||||
Freeze projection security rules.
|
||||
|
||||
### Required Rules
|
||||
|
||||
* Projection persistence must not widen record visibility.
|
||||
* Pricing-sensitive event payloads must remain redacted.
|
||||
* Consumers needing full details must re-query through governed source services or security-aware query modules.
|
||||
* Internal-only Activities must not leak into general projections.
|
||||
* Organization scope must be mandatory.
|
||||
* Manager and Executive consumers must still obey branch, product, role, and pricing boundaries.
|
||||
* Dead-letter and failure records must not persist unrestricted sensitive payloads.
|
||||
|
||||
Produce a security matrix for every planned projection.
|
||||
|
||||
---
|
||||
|
||||
## Part 16 — Existing Consumer Compatibility
|
||||
|
||||
Document compatibility with:
|
||||
|
||||
* current approval notification events
|
||||
* current notification inbox
|
||||
* current Dashboard datasets
|
||||
* current Report datasets
|
||||
* legacy lead follow-ups
|
||||
* legacy opportunity follow-ups
|
||||
* legacy quotation follow-ups
|
||||
* recent-activity compatibility reads
|
||||
|
||||
### Frozen Rule
|
||||
|
||||
No current production consumer is replaced in EP.1.4.
|
||||
|
||||
The new runtime operates beside existing consumers until later projection-specific tasks approve cutover.
|
||||
|
||||
---
|
||||
|
||||
## Part 17 — Projection Delivery Matrix
|
||||
|
||||
Create the official Projection Delivery Matrix.
|
||||
|
||||
Minimum columns:
|
||||
|
||||
| Projection | Event Source | Initial Build Source | Incremental Strategy | Persistence | Idempotency | Retry | Cutover Phase |
|
||||
| ------------------- | --------------------------------------------- | ------------------------- | -------------------- | ---------------------------- | ----------- | -------- | ------------------------ |
|
||||
| Timeline | Activity + CRM events | source queries + adapters | Business Events | query-time or cached | required | required | Timeline epic |
|
||||
| Calendar | Activity + milestones + approval | source queries | Business Events | cached/query-time | required | required | Calendar epic |
|
||||
| Dashboard | Opportunity + Quotation + Approval + Activity | existing datasets | hybrid | existing + future read model | required | required | Dashboard enhancement |
|
||||
| Notification | Activity + Approval + CRM events | not applicable | Business Events | notification records | required | required | Notification expansion |
|
||||
| My Day | Activity + approvals + Hot Project | source composition | hybrid | query-time/cached | required | required | My Day epic |
|
||||
| Relationship Health | Customer + Activity + Opportunity | source rebuild | Business Events | cached score | required | required | Relationship Health epic |
|
||||
| Forecast | Opportunity events | source opportunity query | hybrid | report/read model | required | required | Forecast enhancement |
|
||||
|
||||
The final matrix must reflect repository-specific decisions.
|
||||
|
||||
---
|
||||
|
||||
## Part 18 — Delivery Failure Matrix
|
||||
|
||||
Create the official Delivery Failure Matrix.
|
||||
|
||||
Minimum scenarios:
|
||||
|
||||
| Failure Scenario | Source Mutation Result | Event State | Consumer State | Retry | User/API Result |
|
||||
| ------------------------------------------------- | ---------------------- | ----------- | -------------------------- | -------------- | ----------------- |
|
||||
| Event persistence fails inside source transaction | rollback | none | none | no | mutation fails |
|
||||
| Source mutation commits, consumer fails | committed | persisted | retry scheduled | yes | mutation succeeds |
|
||||
| One of multiple consumers fails | committed | persisted | isolated failure | yes | mutation succeeds |
|
||||
| Unsupported event version | committed | persisted | skipped/failed by consumer | governed | mutation succeeds |
|
||||
| Duplicate event delivered | committed | persisted | duplicate skipped | no side effect | unchanged |
|
||||
| Projection storage unavailable | committed | persisted | retry scheduled | yes | mutation succeeds |
|
||||
| Dead-letter threshold reached | committed | persisted | dead letter | manual | mutation succeeds |
|
||||
| Rebuild fails | unchanged | unchanged | rebuild failed | restartable | no source impact |
|
||||
|
||||
This matrix becomes the reliability baseline for future projection tasks.
|
||||
|
||||
---
|
||||
|
||||
# Deliverables
|
||||
|
||||
## 1. Delivery Semantics Decision
|
||||
|
||||
Official post-commit and eventual-consistency behavior.
|
||||
|
||||
## 2. Persistent Event Delivery Foundation
|
||||
|
||||
Event persistence and delivery state.
|
||||
|
||||
## 3. Transactional Outbox or Equivalent Reliability Mechanism
|
||||
|
||||
Approved and implemented foundation.
|
||||
|
||||
## 4. Projection Consumer Runtime
|
||||
|
||||
Shared processing runtime.
|
||||
|
||||
## 5. Persistent Consumer Checkpoints
|
||||
|
||||
Production-grade idempotency and processing state.
|
||||
|
||||
## 6. Retry Policy
|
||||
|
||||
Governed retry and backoff behavior.
|
||||
|
||||
## 7. Dead-Letter Foundation
|
||||
|
||||
Terminal failure handling and manual recovery contract.
|
||||
|
||||
## 8. Projection Registry
|
||||
|
||||
Machine-readable projection catalog.
|
||||
|
||||
## 9. Projection Storage Standards
|
||||
|
||||
Query-time, cached, and materialized projection rules.
|
||||
|
||||
## 10. Initial Projection Consumer Skeletons
|
||||
|
||||
Timeline, Calendar, Dashboard, and Notification runtime validation consumers.
|
||||
|
||||
## 11. Hybrid Rebuild Strategy
|
||||
|
||||
Source-query initial build plus event-driven incremental update.
|
||||
|
||||
## 12. Projection Rebuild Contract
|
||||
|
||||
Dry-run, scoped, restartable rebuild foundation.
|
||||
|
||||
## 13. Projection Health Model
|
||||
|
||||
Lag, failure, retry, dead-letter, and rebuild visibility.
|
||||
|
||||
## 14. Delivery Observability
|
||||
|
||||
Structured event and consumer processing telemetry.
|
||||
|
||||
## 15. Projection Security Matrix
|
||||
|
||||
Visibility, pricing, internal-only, and organization-boundary rules.
|
||||
|
||||
## 16. Existing Consumer Compatibility Report
|
||||
|
||||
No-regression and cutover readiness documentation.
|
||||
|
||||
## 17. Projection Delivery Matrix
|
||||
|
||||
Official source, persistence, retry, and cutover mapping.
|
||||
|
||||
## 18. Delivery Failure Matrix
|
||||
|
||||
Official behavior for mutation, dispatch, consumer, and rebuild failures.
|
||||
|
||||
## 19. Projection Readiness Report
|
||||
|
||||
Readiness for Timeline, Calendar, Notification, Dashboard, My Day, Manager, Executive, Forecast, and Relationship Health tasks.
|
||||
|
||||
---
|
||||
|
||||
# Constraints
|
||||
|
||||
Must preserve:
|
||||
|
||||
* Customer
|
||||
* Contact
|
||||
* Lead
|
||||
* Opportunity
|
||||
* Quotation
|
||||
* Approval
|
||||
* Activity
|
||||
* current Activity APIs
|
||||
* legacy follow-up APIs and storage
|
||||
* recent-activity compatibility reads
|
||||
* current Dashboard and Report datasets
|
||||
* current Notification behavior
|
||||
* current Approval notification publishing
|
||||
* current Audit Log behavior
|
||||
* current security and pricing visibility boundaries
|
||||
|
||||
Do not implement:
|
||||
|
||||
* final Timeline projection
|
||||
* Timeline UI
|
||||
* final Calendar projection
|
||||
* Calendar UI
|
||||
* Dashboard source replacement
|
||||
* Notification business-event expansion
|
||||
* My Day
|
||||
* Manager Workspace
|
||||
* Executive Workspace
|
||||
* Relationship Health calculation
|
||||
* Forecast replacement
|
||||
* Automation rules
|
||||
* external Kafka, RabbitMQ, or Redis Streams infrastructure
|
||||
* public replay UI
|
||||
* legacy follow-up migration
|
||||
* dual-write from follow-up to Activity
|
||||
* source-domain replacement
|
||||
|
||||
No breaking API changes.
|
||||
|
||||
---
|
||||
|
||||
# Compatibility Strategy
|
||||
|
||||
The Projection Foundation is additive.
|
||||
|
||||
Current production consumers remain active.
|
||||
|
||||
Business Events may be persisted and processed through the new delivery runtime without changing existing API responses.
|
||||
|
||||
Projection skeleton consumers must not produce user-visible duplicate records.
|
||||
|
||||
Any future cutover from legacy datasets to a projection requires:
|
||||
|
||||
* parity validation
|
||||
* regression tests
|
||||
* reconciliation report
|
||||
* feature flag or controlled rollout
|
||||
* rollback strategy
|
||||
* explicit implementation task approval
|
||||
|
||||
---
|
||||
|
||||
# Testing Requirements
|
||||
|
||||
## Delivery Tests
|
||||
|
||||
* source transaction plus event persistence succeeds atomically
|
||||
* failed event persistence rolls back source transaction
|
||||
* committed source mutation survives consumer failure
|
||||
* multiple consumers process independently
|
||||
* no-subscriber event completes safely
|
||||
* persistent event resumes after process restart simulation
|
||||
|
||||
## Checkpoint Tests
|
||||
|
||||
* first processing succeeds
|
||||
* duplicate event is skipped
|
||||
* failed attempt schedules retry
|
||||
* completed checkpoint prevents duplicate side effect
|
||||
* unsupported version is recorded
|
||||
* concurrent processing does not execute twice
|
||||
|
||||
## Retry Tests
|
||||
|
||||
* retryable failure
|
||||
* non-retryable failure
|
||||
* backoff scheduling
|
||||
* maximum-attempt behavior
|
||||
* dead-letter creation
|
||||
* manual retry contract
|
||||
|
||||
## Projection Runtime Tests
|
||||
|
||||
* consumer registration
|
||||
* consumer version support
|
||||
* consumer isolation
|
||||
* projection registry lookup
|
||||
* projection health updates
|
||||
* sanitized failure recording
|
||||
|
||||
## Rebuild Tests
|
||||
|
||||
* organization-scoped dry run
|
||||
* source-query rebuild
|
||||
* restartable rebuild
|
||||
* reset-existing protection
|
||||
* duplicate-safe rebuild
|
||||
* failed rebuild does not affect source data
|
||||
|
||||
## Security Tests
|
||||
|
||||
* organization isolation
|
||||
* branch/product scope enforcement
|
||||
* pricing-sensitive redaction
|
||||
* internal-only activity exclusion
|
||||
* unauthorized rebuild rejection
|
||||
* sensitive payload not exposed in dead-letter records
|
||||
|
||||
## Compatibility Tests
|
||||
|
||||
* existing Activity API behavior unchanged
|
||||
* current follow-up APIs unchanged
|
||||
* existing Dashboard datasets unchanged
|
||||
* existing Reports unchanged
|
||||
* existing approval notifications unchanged
|
||||
* existing recent-activity reads unchanged
|
||||
|
||||
---
|
||||
|
||||
# Acceptance Criteria
|
||||
|
||||
* Source mutations and non-owning projection failures are operationally separated.
|
||||
* A committed source mutation is not reported as failed because a projection consumer failed.
|
||||
* Business Events have a persistent delivery or outbox mechanism.
|
||||
* Projection consumers use persistent checkpoints.
|
||||
* Duplicate event processing does not create duplicate side effects.
|
||||
* Retry and dead-letter behavior is implemented and tested.
|
||||
* Projection consumers remain independently recoverable.
|
||||
* Projection Registry is machine-readable.
|
||||
* Projection storage rules are frozen.
|
||||
* Timeline, Calendar, Dashboard, and Notification skeleton consumers validate the runtime without creating final business-facing projections.
|
||||
* Hybrid source rebuild and event-driven incremental strategy is documented.
|
||||
* Projection rebuild contract supports dry-run and organization scope.
|
||||
* Projection health is measurable.
|
||||
* Delivery observability captures processing and failure state.
|
||||
* Projection security rules preserve all current CRM authorization boundaries.
|
||||
* Existing Dashboard, Report, Notification, Approval, Follow-up, and Activity behavior remains unchanged.
|
||||
* No final workspace or projection UI is introduced.
|
||||
* TypeScript, migration generation, targeted lint, and projection/runtime tests pass.
|
||||
|
||||
---
|
||||
|
||||
# Success Criteria
|
||||
|
||||
ALLA OS has a reliable, production-ready Projection Foundation.
|
||||
|
||||
Business Events can be delivered and processed without coupling projection failure to source-domain mutation success.
|
||||
|
||||
Projection consumers can process events idempotently, retry safely, expose health, recover from failure, and rebuild from governed source data.
|
||||
|
||||
Future Timeline, Calendar, Notification, Dashboard, My Day, Manager Workspace, Executive Workspace, Relationship Health, and Forecast tasks can focus on business read models rather than re-solving event delivery reliability.
|
||||
|
||||
Ready for:
|
||||
|
||||
# EP.1.5 – Timeline Projection Foundation
|
||||
167
src/db/schema.ts
167
src/db/schema.ts
@@ -1,6 +1,7 @@
|
||||
import {
|
||||
boolean,
|
||||
doublePrecision,
|
||||
index,
|
||||
integer,
|
||||
jsonb,
|
||||
pgTable,
|
||||
@@ -451,6 +452,172 @@ export const appNotificationDeliveries = pgTable('app_notification_deliveries',
|
||||
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow().notNull()
|
||||
});
|
||||
|
||||
export const businessEventOutbox = pgTable(
|
||||
'business_event_outbox',
|
||||
{
|
||||
id: text('id').primaryKey(),
|
||||
organizationId: text('organization_id').notNull(),
|
||||
eventType: text('event_type').notNull(),
|
||||
schemaVersion: integer('schema_version').notNull(),
|
||||
entityType: text('entity_type').notNull(),
|
||||
entityId: text('entity_id').notNull(),
|
||||
correlationId: text('correlation_id').notNull(),
|
||||
causationId: text('causation_id'),
|
||||
eventEnvelope: jsonb('event_envelope').notNull(),
|
||||
deliveryStatus: text('delivery_status').default('pending').notNull(),
|
||||
attemptCount: integer('attempt_count').default(0).notNull(),
|
||||
availableAt: timestamp('available_at', { withTimezone: true }).defaultNow().notNull(),
|
||||
processingStartedAt: timestamp('processing_started_at', { withTimezone: true }),
|
||||
dispatchedAt: timestamp('dispatched_at', { withTimezone: true }),
|
||||
failedAt: timestamp('failed_at', { withTimezone: true }),
|
||||
deadLetteredAt: timestamp('dead_lettered_at', { withTimezone: true }),
|
||||
lastErrorCode: text('last_error_code'),
|
||||
lastErrorMessage: text('last_error_message'),
|
||||
occurredAt: timestamp('occurred_at', { withTimezone: true }).notNull(),
|
||||
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow().notNull(),
|
||||
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow().notNull()
|
||||
},
|
||||
(table) => ({
|
||||
statusAvailableIdx: index('business_event_outbox_status_available_idx').on(
|
||||
table.deliveryStatus,
|
||||
table.availableAt
|
||||
),
|
||||
organizationEventIdx: index('business_event_outbox_org_event_idx').on(
|
||||
table.organizationId,
|
||||
table.eventType
|
||||
)
|
||||
})
|
||||
);
|
||||
|
||||
export const projectionConsumerCheckpoints = pgTable(
|
||||
'projection_consumer_checkpoints',
|
||||
{
|
||||
id: text('id').primaryKey(),
|
||||
organizationId: text('organization_id').notNull(),
|
||||
consumerName: text('consumer_name').notNull(),
|
||||
projectionName: text('projection_name').notNull(),
|
||||
eventId: text('event_id').notNull(),
|
||||
eventType: text('event_type').notNull(),
|
||||
schemaVersion: integer('schema_version').notNull(),
|
||||
processingStatus: text('processing_status').default('pending').notNull(),
|
||||
attemptCount: integer('attempt_count').default(0).notNull(),
|
||||
firstAttemptedAt: timestamp('first_attempted_at', { withTimezone: true }),
|
||||
lastAttemptedAt: timestamp('last_attempted_at', { withTimezone: true }),
|
||||
completedAt: timestamp('completed_at', { withTimezone: true }),
|
||||
nextRetryAt: timestamp('next_retry_at', { withTimezone: true }),
|
||||
errorCode: text('error_code'),
|
||||
errorMessage: text('error_message'),
|
||||
processingDurationMs: integer('processing_duration_ms'),
|
||||
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow().notNull(),
|
||||
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow().notNull()
|
||||
},
|
||||
(table) => ({
|
||||
consumerEventIdx: uniqueIndex('projection_checkpoints_consumer_event_idx').on(
|
||||
table.consumerName,
|
||||
table.eventId
|
||||
),
|
||||
statusRetryIdx: index('projection_checkpoints_status_retry_idx').on(
|
||||
table.processingStatus,
|
||||
table.nextRetryAt
|
||||
)
|
||||
})
|
||||
);
|
||||
|
||||
export const projectionDeadLetters = pgTable(
|
||||
'projection_dead_letters',
|
||||
{
|
||||
id: text('id').primaryKey(),
|
||||
organizationId: text('organization_id').notNull(),
|
||||
consumerName: text('consumer_name').notNull(),
|
||||
projectionName: text('projection_name').notNull(),
|
||||
eventId: text('event_id').notNull(),
|
||||
eventType: text('event_type').notNull(),
|
||||
schemaVersion: integer('schema_version').notNull(),
|
||||
attemptCount: integer('attempt_count').notNull(),
|
||||
errorCode: text('error_code').notNull(),
|
||||
errorMessage: text('error_message').notNull(),
|
||||
failureMetadata: jsonb('failure_metadata'),
|
||||
status: text('status').default('open').notNull(),
|
||||
resolvedAt: timestamp('resolved_at', { withTimezone: true }),
|
||||
resolvedBy: text('resolved_by'),
|
||||
resolutionReason: text('resolution_reason'),
|
||||
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow().notNull(),
|
||||
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow().notNull()
|
||||
},
|
||||
(table) => ({
|
||||
consumerEventIdx: uniqueIndex('projection_dead_letters_consumer_event_idx').on(
|
||||
table.consumerName,
|
||||
table.eventId
|
||||
),
|
||||
organizationStatusIdx: index('projection_dead_letters_org_status_idx').on(
|
||||
table.organizationId,
|
||||
table.status
|
||||
)
|
||||
})
|
||||
);
|
||||
|
||||
export const projectionRebuildRuns = pgTable(
|
||||
'projection_rebuild_runs',
|
||||
{
|
||||
id: text('id').primaryKey(),
|
||||
organizationId: text('organization_id').notNull(),
|
||||
projectionName: text('projection_name').notNull(),
|
||||
entityType: text('entity_type'),
|
||||
entityId: text('entity_id'),
|
||||
occurredFrom: timestamp('occurred_from', { withTimezone: true }),
|
||||
occurredTo: timestamp('occurred_to', { withTimezone: true }),
|
||||
dryRun: boolean('dry_run').default(true).notNull(),
|
||||
resetExisting: boolean('reset_existing').default(false).notNull(),
|
||||
sourceStrategy: text('source_strategy').notNull(),
|
||||
status: text('status').default('pending').notNull(),
|
||||
processedCount: integer('processed_count').default(0).notNull(),
|
||||
skippedCount: integer('skipped_count').default(0).notNull(),
|
||||
failedCount: integer('failed_count').default(0).notNull(),
|
||||
errorCode: text('error_code'),
|
||||
errorMessage: text('error_message'),
|
||||
requestedBy: text('requested_by').notNull(),
|
||||
startedAt: timestamp('started_at', { withTimezone: true }),
|
||||
completedAt: timestamp('completed_at', { withTimezone: true }),
|
||||
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow().notNull(),
|
||||
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow().notNull()
|
||||
},
|
||||
(table) => ({
|
||||
organizationProjectionIdx: index('projection_rebuild_runs_org_projection_idx').on(
|
||||
table.organizationId,
|
||||
table.projectionName
|
||||
)
|
||||
})
|
||||
);
|
||||
|
||||
export const projectionHealthSnapshots = pgTable(
|
||||
'projection_health_snapshots',
|
||||
{
|
||||
id: text('id').primaryKey(),
|
||||
organizationId: text('organization_id').notNull(),
|
||||
projectionName: text('projection_name').notNull(),
|
||||
consumerName: text('consumer_name').notNull(),
|
||||
status: text('status').default('healthy').notNull(),
|
||||
lastProcessedEventId: text('last_processed_event_id'),
|
||||
lastSuccessfulProcessingAt: timestamp('last_successful_processing_at', { withTimezone: true }),
|
||||
pendingCount: integer('pending_count').default(0).notNull(),
|
||||
retryCount: integer('retry_count').default(0).notNull(),
|
||||
deadLetterCount: integer('dead_letter_count').default(0).notNull(),
|
||||
unsupportedVersionCount: integer('unsupported_version_count').default(0).notNull(),
|
||||
processingLagMs: integer('processing_lag_ms').default(0).notNull(),
|
||||
averageProcessingDurationMs: integer('average_processing_duration_ms').default(0).notNull(),
|
||||
lastRebuildAt: timestamp('last_rebuild_at', { withTimezone: true }),
|
||||
rebuildSource: text('rebuild_source'),
|
||||
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow().notNull(),
|
||||
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow().notNull()
|
||||
},
|
||||
(table) => ({
|
||||
organizationConsumerIdx: uniqueIndex('projection_health_org_consumer_idx').on(
|
||||
table.organizationId,
|
||||
table.consumerName
|
||||
)
|
||||
})
|
||||
);
|
||||
|
||||
export const crmCustomers = pgTable(
|
||||
'crm_customers',
|
||||
{
|
||||
|
||||
33
src/features/foundation/projections/consumers.ts
Normal file
33
src/features/foundation/projections/consumers.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import type { BusinessEvent } from '../business-events/types.ts';
|
||||
import { getProjectionDefinition } from './registry.ts';
|
||||
import type { ProjectionConsumer, ProjectionHandleResult } from './types.ts';
|
||||
|
||||
function createNoopConsumer(projectionName: string): ProjectionConsumer {
|
||||
const definition = getProjectionDefinition(projectionName);
|
||||
|
||||
return {
|
||||
consumerName: definition.consumerName,
|
||||
projectionName: definition.projectionName,
|
||||
supportedEventTypes: definition.consumedEventTypes,
|
||||
supportedVersions: definition.supportedVersions,
|
||||
async handle(_event: BusinessEvent): Promise<ProjectionHandleResult> {
|
||||
return {
|
||||
status: 'processed',
|
||||
message: `${definition.projectionName} projection skeleton acknowledged event`,
|
||||
sideEffectCount: 0
|
||||
};
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
export const timelineProjectionConsumer = createNoopConsumer('Timeline');
|
||||
export const calendarProjectionConsumer = createNoopConsumer('Calendar');
|
||||
export const dashboardProjectionConsumer = createNoopConsumer('Dashboard');
|
||||
export const notificationProjectionConsumer = createNoopConsumer('Notification');
|
||||
|
||||
export const INITIAL_PROJECTION_CONSUMERS = [
|
||||
timelineProjectionConsumer,
|
||||
calendarProjectionConsumer,
|
||||
dashboardProjectionConsumer,
|
||||
notificationProjectionConsumer
|
||||
];
|
||||
9
src/features/foundation/projections/index.ts
Normal file
9
src/features/foundation/projections/index.ts
Normal file
@@ -0,0 +1,9 @@
|
||||
export * from './consumers.ts';
|
||||
export * from './matrices.ts';
|
||||
export * from './memory-store.ts';
|
||||
export * from './publisher.ts';
|
||||
export * from './rebuild.ts';
|
||||
export * from './registry.ts';
|
||||
export * from './retry-policy.ts';
|
||||
export * from './runtime.ts';
|
||||
export * from './types.ts';
|
||||
178
src/features/foundation/projections/matrices.ts
Normal file
178
src/features/foundation/projections/matrices.ts
Normal file
@@ -0,0 +1,178 @@
|
||||
export const PROJECTION_DELIVERY_MATRIX = [
|
||||
{
|
||||
projection: 'Timeline',
|
||||
eventSource: 'Activity + CRM events',
|
||||
initialBuildSource: 'source queries + adapters',
|
||||
incrementalStrategy: 'Business Events',
|
||||
persistence: 'cached read model',
|
||||
idempotency: 'consumerName + eventId',
|
||||
retry: 'projection-default-v1',
|
||||
cutoverPhase: 'EP.1.5 Timeline epic'
|
||||
},
|
||||
{
|
||||
projection: 'Calendar',
|
||||
eventSource: 'Activity + milestones + approval',
|
||||
initialBuildSource: 'source queries',
|
||||
incrementalStrategy: 'Business Events',
|
||||
persistence: 'cached/query-time',
|
||||
idempotency: 'consumerName + eventId',
|
||||
retry: 'projection-default-v1',
|
||||
cutoverPhase: 'Calendar epic'
|
||||
},
|
||||
{
|
||||
projection: 'Dashboard',
|
||||
eventSource: 'Opportunity + Quotation + Approval + Activity',
|
||||
initialBuildSource: 'existing datasets',
|
||||
incrementalStrategy: 'hybrid',
|
||||
persistence: 'existing datasets + future read model',
|
||||
idempotency: 'consumerName + eventId',
|
||||
retry: 'projection-default-v1',
|
||||
cutoverPhase: 'Dashboard enhancement'
|
||||
},
|
||||
{
|
||||
projection: 'Notification',
|
||||
eventSource: 'Activity + Approval + CRM events',
|
||||
initialBuildSource: 'not applicable',
|
||||
incrementalStrategy: 'Business Events',
|
||||
persistence: 'notification-owned records',
|
||||
idempotency: 'consumerName + eventId + recipient',
|
||||
retry: 'projection-default-v1',
|
||||
cutoverPhase: 'Notification expansion'
|
||||
},
|
||||
{
|
||||
projection: 'My Day',
|
||||
eventSource: 'Activity + approvals + Hot Project',
|
||||
initialBuildSource: 'source composition',
|
||||
incrementalStrategy: 'hybrid',
|
||||
persistence: 'query-time/cached',
|
||||
idempotency: 'consumerName + eventId',
|
||||
retry: 'projection-default-v1',
|
||||
cutoverPhase: 'My Day epic'
|
||||
},
|
||||
{
|
||||
projection: 'Relationship Health',
|
||||
eventSource: 'Customer + Activity + Opportunity',
|
||||
initialBuildSource: 'source rebuild',
|
||||
incrementalStrategy: 'Business Events',
|
||||
persistence: 'cached score',
|
||||
idempotency: 'consumerName + eventId',
|
||||
retry: 'projection-default-v1',
|
||||
cutoverPhase: 'Relationship Health epic'
|
||||
},
|
||||
{
|
||||
projection: 'Forecast',
|
||||
eventSource: 'Opportunity events',
|
||||
initialBuildSource: 'source opportunity query',
|
||||
incrementalStrategy: 'hybrid',
|
||||
persistence: 'report/read model',
|
||||
idempotency: 'consumerName + eventId',
|
||||
retry: 'projection-default-v1',
|
||||
cutoverPhase: 'Forecast enhancement'
|
||||
}
|
||||
] as const;
|
||||
|
||||
export const PROJECTION_FAILURE_MATRIX = [
|
||||
{
|
||||
failureScenario: 'Event persistence fails inside source transaction',
|
||||
sourceMutationResult: 'rollback',
|
||||
eventState: 'none',
|
||||
consumerState: 'none',
|
||||
retry: 'no',
|
||||
userApiResult: 'mutation fails'
|
||||
},
|
||||
{
|
||||
failureScenario: 'Source mutation commits, consumer fails',
|
||||
sourceMutationResult: 'committed',
|
||||
eventState: 'persisted',
|
||||
consumerState: 'retry_scheduled',
|
||||
retry: 'yes',
|
||||
userApiResult: 'mutation succeeds'
|
||||
},
|
||||
{
|
||||
failureScenario: 'One of multiple consumers fails',
|
||||
sourceMutationResult: 'committed',
|
||||
eventState: 'persisted',
|
||||
consumerState: 'isolated failure',
|
||||
retry: 'yes',
|
||||
userApiResult: 'mutation succeeds'
|
||||
},
|
||||
{
|
||||
failureScenario: 'Unsupported event version',
|
||||
sourceMutationResult: 'committed',
|
||||
eventState: 'persisted',
|
||||
consumerState: 'skipped_unsupported_version',
|
||||
retry: 'governed by version migration',
|
||||
userApiResult: 'mutation succeeds'
|
||||
},
|
||||
{
|
||||
failureScenario: 'Duplicate event delivered',
|
||||
sourceMutationResult: 'committed',
|
||||
eventState: 'persisted',
|
||||
consumerState: 'duplicate_completed',
|
||||
retry: 'no side effect',
|
||||
userApiResult: 'unchanged'
|
||||
},
|
||||
{
|
||||
failureScenario: 'Projection storage unavailable',
|
||||
sourceMutationResult: 'committed',
|
||||
eventState: 'persisted',
|
||||
consumerState: 'retry_scheduled',
|
||||
retry: 'yes',
|
||||
userApiResult: 'mutation succeeds'
|
||||
},
|
||||
{
|
||||
failureScenario: 'Dead-letter threshold reached',
|
||||
sourceMutationResult: 'committed',
|
||||
eventState: 'persisted',
|
||||
consumerState: 'dead_letter',
|
||||
retry: 'manual retry after review',
|
||||
userApiResult: 'mutation succeeds'
|
||||
},
|
||||
{
|
||||
failureScenario: 'Rebuild fails',
|
||||
sourceMutationResult: 'unchanged',
|
||||
eventState: 'unchanged',
|
||||
consumerState: 'rebuild failed',
|
||||
retry: 'restartable',
|
||||
userApiResult: 'no source impact'
|
||||
}
|
||||
] as const;
|
||||
|
||||
export const PROJECTION_SECURITY_MATRIX = [
|
||||
{
|
||||
projection: 'Timeline',
|
||||
rule: 'Must reapply source record visibility, internal-only filters, and pricing redaction.'
|
||||
},
|
||||
{
|
||||
projection: 'Calendar',
|
||||
rule: 'Must scope by organization and personal/team access before showing scheduled work.'
|
||||
},
|
||||
{
|
||||
projection: 'Dashboard',
|
||||
rule: 'Must preserve dashboard pricing visibility and report context boundaries.'
|
||||
},
|
||||
{
|
||||
projection: 'Notification',
|
||||
rule: 'Must derive recipients through governed source rules and avoid sensitive payload storage.'
|
||||
},
|
||||
{
|
||||
projection: 'My Day',
|
||||
rule: 'Must personalize from source-authorized activities, approvals, and priorities.'
|
||||
},
|
||||
{
|
||||
projection: 'Manager Workspace',
|
||||
rule: 'Must use resolved CRM branch, product, ownership, and approval authority scopes.'
|
||||
},
|
||||
{
|
||||
projection: 'Executive Workspace',
|
||||
rule: 'Must keep pricing-sensitive metrics behind pricing visibility gates.'
|
||||
},
|
||||
{
|
||||
projection: 'Relationship Health',
|
||||
rule: 'Must not expose customer or contact data beyond ownership/contact-sharing boundaries.'
|
||||
},
|
||||
{
|
||||
projection: 'Forecast',
|
||||
rule: 'Must reuse report foundation access context and revenue/pricing governance.'
|
||||
}
|
||||
] as const;
|
||||
158
src/features/foundation/projections/memory-store.ts
Normal file
158
src/features/foundation/projections/memory-store.ts
Normal file
@@ -0,0 +1,158 @@
|
||||
import type { BusinessEvent } from '../business-events/types.ts';
|
||||
import type {
|
||||
BeginProjectionCheckpointInput,
|
||||
BusinessEventDeliveryRecord,
|
||||
CompleteProjectionCheckpointInput,
|
||||
CreateProjectionDeadLetterInput,
|
||||
FailProjectionCheckpointInput,
|
||||
ProjectionCheckpointRecord,
|
||||
ProjectionHealthSnapshotInput,
|
||||
ProjectionRuntimeStore,
|
||||
UnsupportedProjectionCheckpointInput
|
||||
} from './types.ts';
|
||||
|
||||
export class InMemoryProjectionRuntimeStore implements ProjectionRuntimeStore {
|
||||
readonly deliveries = new Map<string, BusinessEventDeliveryRecord>();
|
||||
readonly checkpoints = new Map<string, ProjectionCheckpointRecord>();
|
||||
readonly deadLetters: CreateProjectionDeadLetterInput[] = [];
|
||||
readonly healthSnapshots: ProjectionHealthSnapshotInput[] = [];
|
||||
|
||||
async persistEvent(event: BusinessEvent): Promise<BusinessEventDeliveryRecord> {
|
||||
const existing = this.deliveries.get(event.eventId);
|
||||
if (existing) {
|
||||
return existing;
|
||||
}
|
||||
|
||||
const record: BusinessEventDeliveryRecord = {
|
||||
eventId: event.eventId,
|
||||
organizationId: event.organizationId,
|
||||
eventType: event.eventType,
|
||||
schemaVersion: event.schemaVersion,
|
||||
status: 'pending',
|
||||
attemptCount: 0,
|
||||
availableAt: new Date().toISOString(),
|
||||
event
|
||||
};
|
||||
this.deliveries.set(event.eventId, record);
|
||||
return record;
|
||||
}
|
||||
|
||||
async markDeliveryProcessing(eventId: string): Promise<void> {
|
||||
const delivery = this.requireDelivery(eventId);
|
||||
delivery.status = 'processing';
|
||||
delivery.attemptCount += 1;
|
||||
}
|
||||
|
||||
async markDeliveryCompleted(eventId: string): Promise<void> {
|
||||
this.requireDelivery(eventId).status = 'completed';
|
||||
}
|
||||
|
||||
async markDeliveryFailed(eventId: string): Promise<void> {
|
||||
this.requireDelivery(eventId).status = 'failed';
|
||||
}
|
||||
|
||||
async markDeliveryDeadLetter(eventId: string): Promise<void> {
|
||||
this.requireDelivery(eventId).status = 'dead_letter';
|
||||
}
|
||||
|
||||
async getCheckpoint(
|
||||
consumerName: string,
|
||||
eventId: string
|
||||
): Promise<ProjectionCheckpointRecord | null> {
|
||||
return this.checkpoints.get(checkpointKey(consumerName, eventId)) ?? null;
|
||||
}
|
||||
|
||||
async beginCheckpoint(input: BeginProjectionCheckpointInput): Promise<ProjectionCheckpointRecord> {
|
||||
const key = checkpointKey(input.consumerName, input.eventId);
|
||||
const existing = this.checkpoints.get(key);
|
||||
|
||||
if (existing) {
|
||||
existing.status = 'processing';
|
||||
existing.attemptCount += 1;
|
||||
return existing;
|
||||
}
|
||||
|
||||
const checkpoint: ProjectionCheckpointRecord = {
|
||||
id: crypto.randomUUID(),
|
||||
organizationId: input.organizationId,
|
||||
consumerName: input.consumerName,
|
||||
projectionName: input.projectionName,
|
||||
eventId: input.eventId,
|
||||
eventType: input.eventType,
|
||||
schemaVersion: input.schemaVersion,
|
||||
status: 'processing',
|
||||
attemptCount: 1,
|
||||
nextRetryAt: null,
|
||||
errorCode: null,
|
||||
errorMessage: null
|
||||
};
|
||||
this.checkpoints.set(key, checkpoint);
|
||||
return checkpoint;
|
||||
}
|
||||
|
||||
async completeCheckpoint(input: CompleteProjectionCheckpointInput): Promise<void> {
|
||||
const checkpoint = this.requireCheckpoint(input.consumerName, input.eventId);
|
||||
checkpoint.status = 'completed';
|
||||
checkpoint.nextRetryAt = null;
|
||||
checkpoint.errorCode = null;
|
||||
checkpoint.errorMessage = null;
|
||||
}
|
||||
|
||||
async failCheckpoint(input: FailProjectionCheckpointInput): Promise<void> {
|
||||
const checkpoint =
|
||||
this.checkpoints.get(checkpointKey(input.consumerName, input.eventId)) ??
|
||||
(await this.beginCheckpoint({ ...input, now: input.now }));
|
||||
|
||||
checkpoint.status = input.deadLetter ? 'dead_letter' : 'retry_scheduled';
|
||||
checkpoint.nextRetryAt = input.nextRetryAt?.toISOString() ?? null;
|
||||
checkpoint.errorCode = input.failure.code;
|
||||
checkpoint.errorMessage = input.failure.message;
|
||||
}
|
||||
|
||||
async skipUnsupportedVersion(input: UnsupportedProjectionCheckpointInput): Promise<void> {
|
||||
this.checkpoints.set(checkpointKey(input.consumerName, input.eventId), {
|
||||
id: crypto.randomUUID(),
|
||||
organizationId: input.organizationId,
|
||||
consumerName: input.consumerName,
|
||||
projectionName: input.projectionName,
|
||||
eventId: input.eventId,
|
||||
eventType: input.eventType,
|
||||
schemaVersion: input.schemaVersion,
|
||||
status: 'skipped_unsupported_version',
|
||||
attemptCount: 0,
|
||||
nextRetryAt: null,
|
||||
errorCode: 'unsupported_event_version',
|
||||
errorMessage: `Unsupported schema version ${input.schemaVersion}`
|
||||
});
|
||||
}
|
||||
|
||||
async createDeadLetter(input: CreateProjectionDeadLetterInput): Promise<void> {
|
||||
this.deadLetters.push(input);
|
||||
}
|
||||
|
||||
async recordHealthSnapshot(input: ProjectionHealthSnapshotInput): Promise<void> {
|
||||
this.healthSnapshots.push(input);
|
||||
}
|
||||
|
||||
private requireDelivery(eventId: string): BusinessEventDeliveryRecord {
|
||||
const delivery = this.deliveries.get(eventId);
|
||||
if (!delivery) {
|
||||
throw new Error(`Missing delivery record for event ${eventId}`);
|
||||
}
|
||||
|
||||
return delivery;
|
||||
}
|
||||
|
||||
private requireCheckpoint(consumerName: string, eventId: string): ProjectionCheckpointRecord {
|
||||
const checkpoint = this.checkpoints.get(checkpointKey(consumerName, eventId));
|
||||
if (!checkpoint) {
|
||||
throw new Error(`Missing checkpoint for ${consumerName}:${eventId}`);
|
||||
}
|
||||
|
||||
return checkpoint;
|
||||
}
|
||||
}
|
||||
|
||||
function checkpointKey(consumerName: string, eventId: string): string {
|
||||
return `${consumerName}:${eventId}`;
|
||||
}
|
||||
36
src/features/foundation/projections/publisher.ts
Normal file
36
src/features/foundation/projections/publisher.ts
Normal file
@@ -0,0 +1,36 @@
|
||||
import type {
|
||||
BusinessEvent,
|
||||
BusinessEventDispatchResult,
|
||||
BusinessEventPublisher
|
||||
} from '../business-events/types.ts';
|
||||
import { ProjectionConsumerRuntime } from './runtime.ts';
|
||||
|
||||
export class ProjectionOutboxBusinessEventPublisher implements BusinessEventPublisher {
|
||||
private readonly runtime: ProjectionConsumerRuntime;
|
||||
|
||||
constructor(options: { runtime: ProjectionConsumerRuntime }) {
|
||||
this.runtime = options.runtime;
|
||||
}
|
||||
|
||||
async publish(event: BusinessEvent): Promise<BusinessEventDispatchResult> {
|
||||
await this.runtime.persistEvent(event);
|
||||
|
||||
return {
|
||||
eventId: event.eventId,
|
||||
eventType: event.eventType,
|
||||
status: 'dispatched',
|
||||
durationMs: 0,
|
||||
subscriberResults: []
|
||||
};
|
||||
}
|
||||
|
||||
async publishMany(events: BusinessEvent[]): Promise<BusinessEventDispatchResult[]> {
|
||||
const results: BusinessEventDispatchResult[] = [];
|
||||
|
||||
for (const event of events) {
|
||||
results.push(await this.publish(event));
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
}
|
||||
31
src/features/foundation/projections/rebuild.ts
Normal file
31
src/features/foundation/projections/rebuild.ts
Normal file
@@ -0,0 +1,31 @@
|
||||
import { getProjectionDefinition } from './registry.ts';
|
||||
import type { ProjectionRebuildRequest, ProjectionRebuildResult } from './types.ts';
|
||||
|
||||
export function createProjectionRebuildPlan(
|
||||
request: ProjectionRebuildRequest
|
||||
): ProjectionRebuildResult {
|
||||
const definition = getProjectionDefinition(request.projectionName);
|
||||
|
||||
if (!request.organizationId) {
|
||||
throw new Error('Projection rebuild requires organizationId');
|
||||
}
|
||||
|
||||
return {
|
||||
rebuildRunId: crypto.randomUUID(),
|
||||
projectionName: definition.projectionName,
|
||||
organizationId: request.organizationId,
|
||||
dryRun: request.dryRun ?? true,
|
||||
status: 'pending',
|
||||
processedCount: 0,
|
||||
skippedCount: 0,
|
||||
failedCount: 0
|
||||
};
|
||||
}
|
||||
|
||||
export const PROJECTION_REBUILD_STRATEGY = {
|
||||
initialBuild: 'Governed source queries plus legacy adapters',
|
||||
incrementalUpdates: 'Business Events through projection runtime checkpoints',
|
||||
recovery: 'Source rebuild plus event checkpoint reconciliation',
|
||||
resetRule: 'resetExisting must be explicit and organization-scoped',
|
||||
rollback: 'Disable projection consumer and rebuild from source-owned data'
|
||||
} as const;
|
||||
155
src/features/foundation/projections/registry.ts
Normal file
155
src/features/foundation/projections/registry.ts
Normal file
@@ -0,0 +1,155 @@
|
||||
import type { ProjectionDefinition } from './types.ts';
|
||||
|
||||
const ACTIVITY_EVENTS = [
|
||||
'activity.created',
|
||||
'activity.assigned',
|
||||
'activity.reassigned',
|
||||
'activity.started',
|
||||
'activity.rescheduled',
|
||||
'activity.completed',
|
||||
'activity.cancelled',
|
||||
'activity.deleted'
|
||||
];
|
||||
|
||||
const ACTIVITY_EVENT_VERSIONS = Object.fromEntries(
|
||||
ACTIVITY_EVENTS.map((eventType) => [eventType, [1]])
|
||||
) as Record<string, number[]>;
|
||||
|
||||
export const PROJECTION_REGISTRY: ProjectionDefinition[] = [
|
||||
{
|
||||
projectionName: 'Timeline',
|
||||
consumerName: 'timeline.projection.consumer',
|
||||
owningFeature: 'foundation/projections',
|
||||
consumedEventTypes: ACTIVITY_EVENTS,
|
||||
supportedVersions: ACTIVITY_EVENT_VERSIONS,
|
||||
rebuildStrategy: 'hybrid',
|
||||
persistence: 'cached-read-model',
|
||||
securityClassification: 'organization-scoped; source-visibility-governed',
|
||||
idempotencyKey: 'consumerName + eventId',
|
||||
retryPolicy: 'projection-default-v1',
|
||||
retentionPolicy: 'event checkpoints retained with projection audit horizon',
|
||||
implementationStatus: 'contract-only'
|
||||
},
|
||||
{
|
||||
projectionName: 'Calendar',
|
||||
consumerName: 'calendar.projection.consumer',
|
||||
owningFeature: 'foundation/projections',
|
||||
consumedEventTypes: ACTIVITY_EVENTS,
|
||||
supportedVersions: ACTIVITY_EVENT_VERSIONS,
|
||||
rebuildStrategy: 'hybrid',
|
||||
persistence: 'cached-read-model',
|
||||
securityClassification: 'organization-scoped; assignment and source-visibility-governed',
|
||||
idempotencyKey: 'consumerName + eventId',
|
||||
retryPolicy: 'projection-default-v1',
|
||||
retentionPolicy: 'calendar projection rows rebuildable from source activity and events',
|
||||
implementationStatus: 'contract-only'
|
||||
},
|
||||
{
|
||||
projectionName: 'Dashboard',
|
||||
consumerName: 'dashboard.projection.consumer',
|
||||
owningFeature: 'crm/dashboard',
|
||||
consumedEventTypes: ACTIVITY_EVENTS,
|
||||
supportedVersions: ACTIVITY_EVENT_VERSIONS,
|
||||
rebuildStrategy: 'hybrid',
|
||||
persistence: 'query-time',
|
||||
securityClassification: 'organization-scoped; pricing visibility remains source-governed',
|
||||
idempotencyKey: 'consumerName + eventId',
|
||||
retryPolicy: 'projection-default-v1',
|
||||
retentionPolicy: 'no final dashboard read-model introduced in EP.1.4',
|
||||
implementationStatus: 'contract-only'
|
||||
},
|
||||
{
|
||||
projectionName: 'Notification',
|
||||
consumerName: 'notification.projection.consumer',
|
||||
owningFeature: 'foundation/notifications',
|
||||
consumedEventTypes: ACTIVITY_EVENTS,
|
||||
supportedVersions: ACTIVITY_EVENT_VERSIONS,
|
||||
rebuildStrategy: 'event-replay',
|
||||
persistence: 'cached-read-model',
|
||||
securityClassification: 'organization-scoped; recipient and template governed',
|
||||
idempotencyKey: 'consumerName + eventId + recipientUserId',
|
||||
retryPolicy: 'projection-default-v1',
|
||||
retentionPolicy: 'notification dedupe and delivery retention remains notification-owned',
|
||||
implementationStatus: 'contract-only'
|
||||
},
|
||||
{
|
||||
projectionName: 'My Day',
|
||||
consumerName: 'my-day.projection.consumer',
|
||||
owningFeature: 'future/my-day',
|
||||
consumedEventTypes: ACTIVITY_EVENTS,
|
||||
supportedVersions: ACTIVITY_EVENT_VERSIONS,
|
||||
rebuildStrategy: 'hybrid',
|
||||
persistence: 'query-time',
|
||||
securityClassification: 'personalized organization-scoped source composition',
|
||||
idempotencyKey: 'consumerName + eventId',
|
||||
retryPolicy: 'projection-default-v1',
|
||||
retentionPolicy: 'future workspace task',
|
||||
implementationStatus: 'contract-only'
|
||||
},
|
||||
{
|
||||
projectionName: 'Manager Workspace',
|
||||
consumerName: 'manager-workspace.projection.consumer',
|
||||
owningFeature: 'future/manager-workspace',
|
||||
consumedEventTypes: ACTIVITY_EVENTS,
|
||||
supportedVersions: ACTIVITY_EVENT_VERSIONS,
|
||||
rebuildStrategy: 'hybrid',
|
||||
persistence: 'query-time',
|
||||
securityClassification: 'manager scope governed by resolved CRM access',
|
||||
idempotencyKey: 'consumerName + eventId',
|
||||
retryPolicy: 'projection-default-v1',
|
||||
retentionPolicy: 'future workspace task',
|
||||
implementationStatus: 'contract-only'
|
||||
},
|
||||
{
|
||||
projectionName: 'Executive Workspace',
|
||||
consumerName: 'executive-workspace.projection.consumer',
|
||||
owningFeature: 'future/executive-workspace',
|
||||
consumedEventTypes: ACTIVITY_EVENTS,
|
||||
supportedVersions: ACTIVITY_EVENT_VERSIONS,
|
||||
rebuildStrategy: 'hybrid',
|
||||
persistence: 'query-time',
|
||||
securityClassification: 'executive scope with pricing visibility gates',
|
||||
idempotencyKey: 'consumerName + eventId',
|
||||
retryPolicy: 'projection-default-v1',
|
||||
retentionPolicy: 'future workspace task',
|
||||
implementationStatus: 'contract-only'
|
||||
},
|
||||
{
|
||||
projectionName: 'Relationship Health',
|
||||
consumerName: 'relationship-health.projection.consumer',
|
||||
owningFeature: 'future/relationship-health',
|
||||
consumedEventTypes: ACTIVITY_EVENTS,
|
||||
supportedVersions: ACTIVITY_EVENT_VERSIONS,
|
||||
rebuildStrategy: 'hybrid',
|
||||
persistence: 'cached-read-model',
|
||||
securityClassification: 'customer visibility and internal-only rules required',
|
||||
idempotencyKey: 'consumerName + eventId',
|
||||
retryPolicy: 'projection-default-v1',
|
||||
retentionPolicy: 'derived score rebuildable from customer, opportunity, activity sources',
|
||||
implementationStatus: 'contract-only'
|
||||
},
|
||||
{
|
||||
projectionName: 'Forecast',
|
||||
consumerName: 'forecast.projection.consumer',
|
||||
owningFeature: 'crm/reports',
|
||||
consumedEventTypes: ACTIVITY_EVENTS,
|
||||
supportedVersions: ACTIVITY_EVENT_VERSIONS,
|
||||
rebuildStrategy: 'hybrid',
|
||||
persistence: 'query-time',
|
||||
securityClassification: 'report context and pricing visibility governed',
|
||||
idempotencyKey: 'consumerName + eventId',
|
||||
retryPolicy: 'projection-default-v1',
|
||||
retentionPolicy: 'forecast enhancement task',
|
||||
implementationStatus: 'contract-only'
|
||||
}
|
||||
];
|
||||
|
||||
export function getProjectionDefinition(projectionName: string): ProjectionDefinition {
|
||||
const definition = PROJECTION_REGISTRY.find((projection) => projection.projectionName === projectionName);
|
||||
|
||||
if (!definition) {
|
||||
throw new Error(`Unknown projection: ${projectionName}`);
|
||||
}
|
||||
|
||||
return definition;
|
||||
}
|
||||
79
src/features/foundation/projections/retry-policy.ts
Normal file
79
src/features/foundation/projections/retry-policy.ts
Normal file
@@ -0,0 +1,79 @@
|
||||
import type { ProjectionFailure } from './types.ts';
|
||||
|
||||
export interface ProjectionRetryPolicy {
|
||||
name: string;
|
||||
maxAttempts: number;
|
||||
backoffMs: number[];
|
||||
jitterRatio: number;
|
||||
}
|
||||
|
||||
export const DEFAULT_PROJECTION_RETRY_POLICY: ProjectionRetryPolicy = {
|
||||
name: 'projection-default-v1',
|
||||
maxAttempts: 5,
|
||||
backoffMs: [0, 60_000, 300_000, 1_800_000],
|
||||
jitterRatio: 0.1
|
||||
};
|
||||
|
||||
export class ProjectionConsumerError extends Error {
|
||||
readonly code: string;
|
||||
readonly retryable: boolean;
|
||||
|
||||
constructor(message: string, options: { code?: string; retryable?: boolean } = {}) {
|
||||
super(message);
|
||||
this.name = 'ProjectionConsumerError';
|
||||
this.code = options.code ?? 'projection_consumer_error';
|
||||
this.retryable = options.retryable ?? true;
|
||||
}
|
||||
}
|
||||
|
||||
export function sanitizeProjectionFailure(error: unknown): ProjectionFailure {
|
||||
if (error instanceof ProjectionConsumerError) {
|
||||
return {
|
||||
code: error.code,
|
||||
message: sanitizeErrorMessage(error.message),
|
||||
retryable: error.retryable
|
||||
};
|
||||
}
|
||||
|
||||
if (error instanceof Error) {
|
||||
return {
|
||||
code: 'unexpected_projection_error',
|
||||
message: sanitizeErrorMessage(error.message),
|
||||
retryable: true
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
code: 'unknown_projection_error',
|
||||
message: 'Unknown projection processing error',
|
||||
retryable: true
|
||||
};
|
||||
}
|
||||
|
||||
export function resolveNextRetryAt(
|
||||
attemptCount: number,
|
||||
failure: ProjectionFailure,
|
||||
now: Date,
|
||||
policy: ProjectionRetryPolicy = DEFAULT_PROJECTION_RETRY_POLICY
|
||||
): Date | null {
|
||||
if (!failure.retryable || attemptCount >= policy.maxAttempts) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const delayIndex = Math.min(Math.max(attemptCount - 1, 0), policy.backoffMs.length - 1);
|
||||
const baseDelay = policy.backoffMs[delayIndex] ?? policy.backoffMs.at(-1) ?? 0;
|
||||
const jitter = Math.round(baseDelay * policy.jitterRatio * 0.5);
|
||||
return new Date(now.getTime() + baseDelay + jitter);
|
||||
}
|
||||
|
||||
export function shouldDeadLetter(
|
||||
attemptCount: number,
|
||||
failure: ProjectionFailure,
|
||||
policy: ProjectionRetryPolicy = DEFAULT_PROJECTION_RETRY_POLICY
|
||||
): boolean {
|
||||
return !failure.retryable || attemptCount >= policy.maxAttempts;
|
||||
}
|
||||
|
||||
function sanitizeErrorMessage(message: string): string {
|
||||
return message.replace(/\s+/g, ' ').slice(0, 500);
|
||||
}
|
||||
190
src/features/foundation/projections/runtime.test.ts
Normal file
190
src/features/foundation/projections/runtime.test.ts
Normal file
@@ -0,0 +1,190 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { createBusinessEvent } from '../business-events/publisher.ts';
|
||||
import { INITIAL_PROJECTION_CONSUMERS } from './consumers.ts';
|
||||
import { InMemoryProjectionRuntimeStore } from './memory-store.ts';
|
||||
import { PROJECTION_REGISTRY, getProjectionDefinition } from './registry.ts';
|
||||
import { ProjectionOutboxBusinessEventPublisher } from './publisher.ts';
|
||||
import { ProjectionConsumerError } from './retry-policy.ts';
|
||||
import { ProjectionConsumerRuntime } from './runtime.ts';
|
||||
import type { ProjectionConsumer } from './types.ts';
|
||||
|
||||
function activityEvent(overrides: Record<string, unknown> = {}) {
|
||||
return createBusinessEvent({
|
||||
eventType: 'activity.completed',
|
||||
organizationId: 'org-1',
|
||||
entityType: 'activity',
|
||||
entityId: 'activity-1',
|
||||
primaryRecord: { type: 'activity', id: 'activity-1' },
|
||||
actorUserId: 'user-1',
|
||||
correlationId: 'corr-1',
|
||||
payload: { activityId: 'activity-1', ...overrides }
|
||||
});
|
||||
}
|
||||
|
||||
test('projection registry includes required planned projections', () => {
|
||||
const names = PROJECTION_REGISTRY.map((projection) => projection.projectionName);
|
||||
assert.deepEqual(names, [
|
||||
'Timeline',
|
||||
'Calendar',
|
||||
'Dashboard',
|
||||
'Notification',
|
||||
'My Day',
|
||||
'Manager Workspace',
|
||||
'Executive Workspace',
|
||||
'Relationship Health',
|
||||
'Forecast'
|
||||
]);
|
||||
assert.equal(getProjectionDefinition('Timeline').idempotencyKey, 'consumerName + eventId');
|
||||
});
|
||||
|
||||
test('runtime processes skeleton consumers and records completed checkpoints', async () => {
|
||||
const store = new InMemoryProjectionRuntimeStore();
|
||||
const event = activityEvent();
|
||||
await store.persistEvent(event);
|
||||
|
||||
const runtime = new ProjectionConsumerRuntime({
|
||||
store,
|
||||
consumers: INITIAL_PROJECTION_CONSUMERS
|
||||
});
|
||||
|
||||
const result = await runtime.processEvent(event);
|
||||
|
||||
assert.equal(result.status, 'completed');
|
||||
assert.equal(result.consumerResults.length, 4);
|
||||
assert.equal([...store.checkpoints.values()].every((checkpoint) => checkpoint.status === 'completed'), true);
|
||||
});
|
||||
|
||||
test('outbox publisher persists event without running projection consumers synchronously', async () => {
|
||||
const store = new InMemoryProjectionRuntimeStore();
|
||||
const event = activityEvent();
|
||||
let handled = false;
|
||||
const consumer: ProjectionConsumer = {
|
||||
consumerName: 'sync-guard.consumer',
|
||||
projectionName: 'Sync Guard',
|
||||
supportedEventTypes: ['activity.completed'],
|
||||
supportedVersions: { 'activity.completed': [1] },
|
||||
async handle() {
|
||||
handled = true;
|
||||
return { status: 'processed' };
|
||||
}
|
||||
};
|
||||
const runtime = new ProjectionConsumerRuntime({ store, consumers: [consumer] });
|
||||
const publisher = new ProjectionOutboxBusinessEventPublisher({ runtime });
|
||||
|
||||
const result = await publisher.publish(event);
|
||||
|
||||
assert.equal(result.status, 'dispatched');
|
||||
assert.equal(store.deliveries.get(event.eventId)?.status, 'pending');
|
||||
assert.equal(handled, false);
|
||||
});
|
||||
|
||||
test('completed checkpoint prevents duplicate side effects', async () => {
|
||||
const store = new InMemoryProjectionRuntimeStore();
|
||||
const event = activityEvent();
|
||||
let count = 0;
|
||||
const consumer: ProjectionConsumer = {
|
||||
consumerName: 'counter.consumer',
|
||||
projectionName: 'Counter',
|
||||
supportedEventTypes: ['activity.completed'],
|
||||
supportedVersions: { 'activity.completed': [1] },
|
||||
async handle() {
|
||||
count += 1;
|
||||
return { status: 'processed' };
|
||||
}
|
||||
};
|
||||
|
||||
await store.persistEvent(event);
|
||||
const runtime = new ProjectionConsumerRuntime({ store, consumers: [consumer] });
|
||||
|
||||
await runtime.processEvent(event);
|
||||
const second = await runtime.processEvent(event);
|
||||
|
||||
assert.equal(count, 1);
|
||||
assert.equal(second.consumerResults[0]?.status, 'duplicate_completed');
|
||||
});
|
||||
|
||||
test('consumer failure schedules retry without throwing or blocking unrelated consumers', async () => {
|
||||
const store = new InMemoryProjectionRuntimeStore();
|
||||
const event = activityEvent();
|
||||
const handled: string[] = [];
|
||||
const failing: ProjectionConsumer = {
|
||||
consumerName: 'failing.consumer',
|
||||
projectionName: 'Failing',
|
||||
supportedEventTypes: ['activity.completed'],
|
||||
supportedVersions: { 'activity.completed': [1] },
|
||||
async handle() {
|
||||
throw new ProjectionConsumerError('storage unavailable customer=secret', {
|
||||
code: 'projection_storage_unavailable',
|
||||
retryable: true
|
||||
});
|
||||
}
|
||||
};
|
||||
const healthy: ProjectionConsumer = {
|
||||
consumerName: 'healthy.consumer',
|
||||
projectionName: 'Healthy',
|
||||
supportedEventTypes: ['activity.completed'],
|
||||
supportedVersions: { 'activity.completed': [1] },
|
||||
async handle() {
|
||||
handled.push('healthy');
|
||||
return { status: 'processed' };
|
||||
}
|
||||
};
|
||||
|
||||
await store.persistEvent(event);
|
||||
const runtime = new ProjectionConsumerRuntime({ store, consumers: [failing, healthy] });
|
||||
const result = await runtime.processEvent(event);
|
||||
|
||||
assert.equal(result.status, 'failed');
|
||||
assert.deepEqual(handled, ['healthy']);
|
||||
assert.equal(
|
||||
[...store.checkpoints.values()].some((checkpoint) => checkpoint.status === 'retry_scheduled'),
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
test('unsupported event version is checkpointed as skipped', async () => {
|
||||
const store = new InMemoryProjectionRuntimeStore();
|
||||
const event = { ...activityEvent(), schemaVersion: 99 };
|
||||
const consumer: ProjectionConsumer = {
|
||||
consumerName: 'versioned.consumer',
|
||||
projectionName: 'Versioned',
|
||||
supportedEventTypes: ['activity.completed'],
|
||||
supportedVersions: { 'activity.completed': [1] },
|
||||
async handle() {
|
||||
return { status: 'processed' };
|
||||
}
|
||||
};
|
||||
|
||||
await store.persistEvent(event);
|
||||
const runtime = new ProjectionConsumerRuntime({ store, consumers: [consumer] });
|
||||
const result = await runtime.processEvent(event);
|
||||
|
||||
assert.equal(result.status, 'completed');
|
||||
assert.equal(result.consumerResults[0]?.status, 'skipped_unsupported_version');
|
||||
});
|
||||
|
||||
test('non-retryable failure creates dead letter', async () => {
|
||||
const store = new InMemoryProjectionRuntimeStore();
|
||||
const event = activityEvent();
|
||||
const consumer: ProjectionConsumer = {
|
||||
consumerName: 'dead.consumer',
|
||||
projectionName: 'Dead',
|
||||
supportedEventTypes: ['activity.completed'],
|
||||
supportedVersions: { 'activity.completed': [1] },
|
||||
async handle() {
|
||||
throw new ProjectionConsumerError('unsupported projection invariant', {
|
||||
code: 'projection_invariant_failed',
|
||||
retryable: false
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
await store.persistEvent(event);
|
||||
const runtime = new ProjectionConsumerRuntime({ store, consumers: [consumer] });
|
||||
const result = await runtime.processEvent(event);
|
||||
|
||||
assert.equal(result.status, 'dead_letter');
|
||||
assert.equal(store.deadLetters.length, 1);
|
||||
assert.equal(store.deadLetters[0]?.failure.code, 'projection_invariant_failed');
|
||||
});
|
||||
225
src/features/foundation/projections/runtime.ts
Normal file
225
src/features/foundation/projections/runtime.ts
Normal file
@@ -0,0 +1,225 @@
|
||||
import { performance } from 'node:perf_hooks';
|
||||
import type { BusinessEvent } from '../business-events/types.ts';
|
||||
import {
|
||||
DEFAULT_PROJECTION_RETRY_POLICY,
|
||||
sanitizeProjectionFailure,
|
||||
shouldDeadLetter,
|
||||
resolveNextRetryAt,
|
||||
type ProjectionRetryPolicy
|
||||
} from './retry-policy.ts';
|
||||
import type {
|
||||
ProjectionConsumer,
|
||||
ProjectionConsumerRuntimeResult,
|
||||
ProjectionRuntimeResult,
|
||||
ProjectionRuntimeStore
|
||||
} from './types.ts';
|
||||
|
||||
interface ProjectionRuntimeOptions {
|
||||
store: ProjectionRuntimeStore;
|
||||
consumers: ProjectionConsumer[];
|
||||
retryPolicy?: ProjectionRetryPolicy;
|
||||
now?: () => Date;
|
||||
}
|
||||
|
||||
export class ProjectionConsumerRuntime {
|
||||
private readonly store: ProjectionRuntimeStore;
|
||||
private readonly consumers: ProjectionConsumer[];
|
||||
private readonly retryPolicy: ProjectionRetryPolicy;
|
||||
private readonly now: () => Date;
|
||||
|
||||
constructor(options: ProjectionRuntimeOptions) {
|
||||
this.store = options.store;
|
||||
this.consumers = options.consumers;
|
||||
this.retryPolicy = options.retryPolicy ?? DEFAULT_PROJECTION_RETRY_POLICY;
|
||||
this.now = options.now ?? (() => new Date());
|
||||
}
|
||||
|
||||
async persistEvent(event: BusinessEvent) {
|
||||
return this.store.persistEvent(event);
|
||||
}
|
||||
|
||||
async processEvent(event: BusinessEvent): Promise<ProjectionRuntimeResult> {
|
||||
await this.store.markDeliveryProcessing(event.eventId, this.now());
|
||||
|
||||
const consumers = this.consumers.filter((consumer) =>
|
||||
consumer.supportedEventTypes.includes(event.eventType)
|
||||
);
|
||||
|
||||
if (consumers.length === 0) {
|
||||
await this.store.markDeliveryCompleted(event.eventId, this.now());
|
||||
return {
|
||||
eventId: event.eventId,
|
||||
eventType: event.eventType,
|
||||
status: 'completed',
|
||||
consumerResults: []
|
||||
};
|
||||
}
|
||||
|
||||
const consumerResults: ProjectionConsumerRuntimeResult[] = [];
|
||||
|
||||
for (const consumer of consumers) {
|
||||
consumerResults.push(await this.processConsumer(event, consumer));
|
||||
}
|
||||
|
||||
const hasDeadLetter = consumerResults.some((result) => result.status === 'dead_letter');
|
||||
const hasRetry = consumerResults.some((result) => result.status === 'retry_scheduled');
|
||||
const hasFailure = consumerResults.some((result) => result.status === 'failed');
|
||||
|
||||
if (hasDeadLetter) {
|
||||
await this.store.markDeliveryDeadLetter(
|
||||
event.eventId,
|
||||
consumerResults.find((result) => result.error)?.error ?? {
|
||||
code: 'projection_dead_letter',
|
||||
message: 'At least one projection consumer reached dead-letter state',
|
||||
retryable: false
|
||||
},
|
||||
this.now()
|
||||
);
|
||||
return { eventId: event.eventId, eventType: event.eventType, status: 'dead_letter', consumerResults };
|
||||
}
|
||||
|
||||
if (hasRetry || hasFailure) {
|
||||
await this.store.markDeliveryFailed(
|
||||
event.eventId,
|
||||
consumerResults.find((result) => result.error)?.error ?? {
|
||||
code: 'projection_retry_scheduled',
|
||||
message: 'At least one projection consumer is scheduled for retry',
|
||||
retryable: true
|
||||
},
|
||||
this.now()
|
||||
);
|
||||
return { eventId: event.eventId, eventType: event.eventType, status: 'failed', consumerResults };
|
||||
}
|
||||
|
||||
await this.store.markDeliveryCompleted(event.eventId, this.now());
|
||||
return { eventId: event.eventId, eventType: event.eventType, status: 'completed', consumerResults };
|
||||
}
|
||||
|
||||
private async processConsumer(
|
||||
event: BusinessEvent,
|
||||
consumer: ProjectionConsumer
|
||||
): Promise<ProjectionConsumerRuntimeResult> {
|
||||
const startedAt = performance.now();
|
||||
const supportedVersions = consumer.supportedVersions[event.eventType] ?? [];
|
||||
|
||||
if (!supportedVersions.includes(event.schemaVersion)) {
|
||||
await this.store.skipUnsupportedVersion({
|
||||
organizationId: event.organizationId,
|
||||
consumerName: consumer.consumerName,
|
||||
projectionName: consumer.projectionName,
|
||||
eventId: event.eventId,
|
||||
eventType: event.eventType,
|
||||
schemaVersion: event.schemaVersion,
|
||||
now: this.now()
|
||||
});
|
||||
|
||||
return {
|
||||
consumerName: consumer.consumerName,
|
||||
projectionName: consumer.projectionName,
|
||||
status: 'skipped_unsupported_version',
|
||||
durationMs: performance.now() - startedAt
|
||||
};
|
||||
}
|
||||
|
||||
const existing = await this.store.getCheckpoint(consumer.consumerName, event.eventId);
|
||||
if (existing?.status === 'completed') {
|
||||
return {
|
||||
consumerName: consumer.consumerName,
|
||||
projectionName: consumer.projectionName,
|
||||
status: 'duplicate_completed',
|
||||
durationMs: performance.now() - startedAt
|
||||
};
|
||||
}
|
||||
|
||||
const checkpoint = await this.store.beginCheckpoint({
|
||||
organizationId: event.organizationId,
|
||||
consumerName: consumer.consumerName,
|
||||
projectionName: consumer.projectionName,
|
||||
eventId: event.eventId,
|
||||
eventType: event.eventType,
|
||||
schemaVersion: event.schemaVersion,
|
||||
now: this.now()
|
||||
});
|
||||
|
||||
try {
|
||||
await consumer.handle(event);
|
||||
const durationMs = performance.now() - startedAt;
|
||||
await this.store.completeCheckpoint({
|
||||
consumerName: consumer.consumerName,
|
||||
eventId: event.eventId,
|
||||
durationMs,
|
||||
now: this.now()
|
||||
});
|
||||
await this.store.recordHealthSnapshot({
|
||||
organizationId: event.organizationId,
|
||||
projectionName: consumer.projectionName,
|
||||
consumerName: consumer.consumerName,
|
||||
status: 'healthy',
|
||||
lastProcessedEventId: event.eventId,
|
||||
lastSuccessfulProcessingAt: this.now().toISOString(),
|
||||
averageProcessingDurationMs: Math.round(durationMs)
|
||||
});
|
||||
|
||||
return {
|
||||
consumerName: consumer.consumerName,
|
||||
projectionName: consumer.projectionName,
|
||||
status: 'completed',
|
||||
durationMs
|
||||
};
|
||||
} catch (error) {
|
||||
const durationMs = performance.now() - startedAt;
|
||||
const failure = sanitizeProjectionFailure(error);
|
||||
const attemptCount = checkpoint.attemptCount;
|
||||
const deadLetter = shouldDeadLetter(attemptCount, failure, this.retryPolicy);
|
||||
const nextRetryAt = deadLetter
|
||||
? null
|
||||
: resolveNextRetryAt(attemptCount, failure, this.now(), this.retryPolicy);
|
||||
|
||||
await this.store.failCheckpoint({
|
||||
organizationId: event.organizationId,
|
||||
consumerName: consumer.consumerName,
|
||||
projectionName: consumer.projectionName,
|
||||
eventId: event.eventId,
|
||||
eventType: event.eventType,
|
||||
schemaVersion: event.schemaVersion,
|
||||
failure,
|
||||
nextRetryAt,
|
||||
deadLetter,
|
||||
durationMs,
|
||||
now: this.now()
|
||||
});
|
||||
|
||||
if (deadLetter) {
|
||||
await this.store.createDeadLetter({
|
||||
organizationId: event.organizationId,
|
||||
consumerName: consumer.consumerName,
|
||||
projectionName: consumer.projectionName,
|
||||
eventId: event.eventId,
|
||||
eventType: event.eventType,
|
||||
schemaVersion: event.schemaVersion,
|
||||
attemptCount,
|
||||
failure,
|
||||
now: this.now()
|
||||
});
|
||||
}
|
||||
|
||||
await this.store.recordHealthSnapshot({
|
||||
organizationId: event.organizationId,
|
||||
projectionName: consumer.projectionName,
|
||||
consumerName: consumer.consumerName,
|
||||
status: deadLetter ? 'failed' : 'degraded',
|
||||
retryCount: deadLetter ? 0 : 1,
|
||||
deadLetterCount: deadLetter ? 1 : 0
|
||||
});
|
||||
|
||||
return {
|
||||
consumerName: consumer.consumerName,
|
||||
projectionName: consumer.projectionName,
|
||||
status: deadLetter ? 'dead_letter' : 'retry_scheduled',
|
||||
durationMs,
|
||||
retryAt: nextRetryAt?.toISOString(),
|
||||
error: failure
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
296
src/features/foundation/projections/server/store.ts
Normal file
296
src/features/foundation/projections/server/store.ts
Normal file
@@ -0,0 +1,296 @@
|
||||
import { and, eq } from 'drizzle-orm';
|
||||
import {
|
||||
businessEventOutbox,
|
||||
projectionConsumerCheckpoints,
|
||||
projectionDeadLetters,
|
||||
projectionHealthSnapshots
|
||||
} from '@/db/schema';
|
||||
import { db } from '@/lib/db';
|
||||
import type { BusinessEvent } from '../../business-events/types.ts';
|
||||
import type {
|
||||
BeginProjectionCheckpointInput,
|
||||
BusinessEventDeliveryRecord,
|
||||
CompleteProjectionCheckpointInput,
|
||||
CreateProjectionDeadLetterInput,
|
||||
FailProjectionCheckpointInput,
|
||||
ProjectionCheckpointRecord,
|
||||
ProjectionRuntimeStore,
|
||||
UnsupportedProjectionCheckpointInput,
|
||||
ProjectionHealthSnapshotInput,
|
||||
ProjectionFailure
|
||||
} from '../types.ts';
|
||||
|
||||
export class DrizzleProjectionRuntimeStore implements ProjectionRuntimeStore {
|
||||
async persistEvent(event: BusinessEvent): Promise<BusinessEventDeliveryRecord> {
|
||||
const existing = await this.getDelivery(event.eventId);
|
||||
if (existing) {
|
||||
return existing;
|
||||
}
|
||||
|
||||
const [created] = await db
|
||||
.insert(businessEventOutbox)
|
||||
.values({
|
||||
id: event.eventId,
|
||||
organizationId: event.organizationId,
|
||||
eventType: event.eventType,
|
||||
schemaVersion: event.schemaVersion,
|
||||
entityType: event.entityType,
|
||||
entityId: event.entityId,
|
||||
correlationId: event.correlationId,
|
||||
causationId: event.causationId ?? null,
|
||||
eventEnvelope: event,
|
||||
deliveryStatus: 'pending',
|
||||
occurredAt: new Date(event.occurredAt)
|
||||
})
|
||||
.returning();
|
||||
|
||||
return mapDelivery(created);
|
||||
}
|
||||
|
||||
async markDeliveryProcessing(eventId: string, now: Date): Promise<void> {
|
||||
const existing = await this.getDelivery(eventId);
|
||||
await db
|
||||
.update(businessEventOutbox)
|
||||
.set({
|
||||
deliveryStatus: 'processing',
|
||||
attemptCount: (existing?.attemptCount ?? 0) + 1,
|
||||
processingStartedAt: now,
|
||||
updatedAt: now
|
||||
})
|
||||
.where(eq(businessEventOutbox.id, eventId));
|
||||
}
|
||||
|
||||
async markDeliveryCompleted(eventId: string, now: Date): Promise<void> {
|
||||
await updateDeliveryStatus(eventId, 'completed', now, { dispatchedAt: now });
|
||||
}
|
||||
|
||||
async markDeliveryFailed(eventId: string, error: ProjectionFailure, now: Date): Promise<void> {
|
||||
await updateDeliveryStatus(eventId, 'failed', now, {
|
||||
failedAt: now,
|
||||
lastErrorCode: error.code,
|
||||
lastErrorMessage: error.message
|
||||
});
|
||||
}
|
||||
|
||||
async markDeliveryDeadLetter(eventId: string, error: ProjectionFailure, now: Date): Promise<void> {
|
||||
await updateDeliveryStatus(eventId, 'dead_letter', now, {
|
||||
deadLetteredAt: now,
|
||||
lastErrorCode: error.code,
|
||||
lastErrorMessage: error.message
|
||||
});
|
||||
}
|
||||
|
||||
async getCheckpoint(
|
||||
consumerName: string,
|
||||
eventId: string
|
||||
): Promise<ProjectionCheckpointRecord | null> {
|
||||
const [row] = await db
|
||||
.select()
|
||||
.from(projectionConsumerCheckpoints)
|
||||
.where(
|
||||
and(
|
||||
eq(projectionConsumerCheckpoints.consumerName, consumerName),
|
||||
eq(projectionConsumerCheckpoints.eventId, eventId)
|
||||
)
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
return row ? mapCheckpoint(row) : null;
|
||||
}
|
||||
|
||||
async beginCheckpoint(input: BeginProjectionCheckpointInput): Promise<ProjectionCheckpointRecord> {
|
||||
const existing = await this.getCheckpoint(input.consumerName, input.eventId);
|
||||
|
||||
if (existing) {
|
||||
const [updated] = await db
|
||||
.update(projectionConsumerCheckpoints)
|
||||
.set({
|
||||
processingStatus: 'processing',
|
||||
attemptCount: existing.attemptCount + 1,
|
||||
lastAttemptedAt: input.now,
|
||||
updatedAt: input.now
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(projectionConsumerCheckpoints.consumerName, input.consumerName),
|
||||
eq(projectionConsumerCheckpoints.eventId, input.eventId)
|
||||
)
|
||||
)
|
||||
.returning();
|
||||
return mapCheckpoint(updated);
|
||||
}
|
||||
|
||||
const [created] = await db
|
||||
.insert(projectionConsumerCheckpoints)
|
||||
.values({
|
||||
id: crypto.randomUUID(),
|
||||
organizationId: input.organizationId,
|
||||
consumerName: input.consumerName,
|
||||
projectionName: input.projectionName,
|
||||
eventId: input.eventId,
|
||||
eventType: input.eventType,
|
||||
schemaVersion: input.schemaVersion,
|
||||
processingStatus: 'processing',
|
||||
attemptCount: 1,
|
||||
firstAttemptedAt: input.now,
|
||||
lastAttemptedAt: input.now
|
||||
})
|
||||
.returning();
|
||||
|
||||
return mapCheckpoint(created);
|
||||
}
|
||||
|
||||
async completeCheckpoint(input: CompleteProjectionCheckpointInput): Promise<void> {
|
||||
await db
|
||||
.update(projectionConsumerCheckpoints)
|
||||
.set({
|
||||
processingStatus: 'completed',
|
||||
completedAt: input.now,
|
||||
nextRetryAt: null,
|
||||
errorCode: null,
|
||||
errorMessage: null,
|
||||
processingDurationMs: Math.round(input.durationMs),
|
||||
updatedAt: input.now
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(projectionConsumerCheckpoints.consumerName, input.consumerName),
|
||||
eq(projectionConsumerCheckpoints.eventId, input.eventId)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
async failCheckpoint(input: FailProjectionCheckpointInput): Promise<void> {
|
||||
await db
|
||||
.update(projectionConsumerCheckpoints)
|
||||
.set({
|
||||
processingStatus: input.deadLetter ? 'dead_letter' : 'retry_scheduled',
|
||||
nextRetryAt: input.nextRetryAt,
|
||||
errorCode: input.failure.code,
|
||||
errorMessage: input.failure.message,
|
||||
processingDurationMs: Math.round(input.durationMs),
|
||||
updatedAt: input.now
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(projectionConsumerCheckpoints.consumerName, input.consumerName),
|
||||
eq(projectionConsumerCheckpoints.eventId, input.eventId)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
async skipUnsupportedVersion(input: UnsupportedProjectionCheckpointInput): Promise<void> {
|
||||
const existing = await this.getCheckpoint(input.consumerName, input.eventId);
|
||||
if (existing) {
|
||||
return;
|
||||
}
|
||||
|
||||
await db.insert(projectionConsumerCheckpoints).values({
|
||||
id: crypto.randomUUID(),
|
||||
organizationId: input.organizationId,
|
||||
consumerName: input.consumerName,
|
||||
projectionName: input.projectionName,
|
||||
eventId: input.eventId,
|
||||
eventType: input.eventType,
|
||||
schemaVersion: input.schemaVersion,
|
||||
processingStatus: 'skipped_unsupported_version',
|
||||
attemptCount: 0,
|
||||
errorCode: 'unsupported_event_version',
|
||||
errorMessage: `Unsupported schema version ${input.schemaVersion}`
|
||||
});
|
||||
}
|
||||
|
||||
async createDeadLetter(input: CreateProjectionDeadLetterInput): Promise<void> {
|
||||
await db.insert(projectionDeadLetters).values({
|
||||
id: crypto.randomUUID(),
|
||||
organizationId: input.organizationId,
|
||||
consumerName: input.consumerName,
|
||||
projectionName: input.projectionName,
|
||||
eventId: input.eventId,
|
||||
eventType: input.eventType,
|
||||
schemaVersion: input.schemaVersion,
|
||||
attemptCount: input.attemptCount,
|
||||
errorCode: input.failure.code,
|
||||
errorMessage: input.failure.message,
|
||||
failureMetadata: {
|
||||
retryable: input.failure.retryable
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async recordHealthSnapshot(input: ProjectionHealthSnapshotInput): Promise<void> {
|
||||
await db.insert(projectionHealthSnapshots).values({
|
||||
id: crypto.randomUUID(),
|
||||
organizationId: input.organizationId,
|
||||
projectionName: input.projectionName,
|
||||
consumerName: input.consumerName,
|
||||
status: input.status,
|
||||
lastProcessedEventId: input.lastProcessedEventId ?? null,
|
||||
lastSuccessfulProcessingAt: input.lastSuccessfulProcessingAt
|
||||
? new Date(input.lastSuccessfulProcessingAt)
|
||||
: null,
|
||||
pendingCount: input.pendingCount ?? 0,
|
||||
retryCount: input.retryCount ?? 0,
|
||||
deadLetterCount: input.deadLetterCount ?? 0,
|
||||
unsupportedVersionCount: input.unsupportedVersionCount ?? 0,
|
||||
processingLagMs: input.processingLagMs ?? 0,
|
||||
averageProcessingDurationMs: input.averageProcessingDurationMs ?? 0,
|
||||
lastRebuildAt: input.lastRebuildAt ? new Date(input.lastRebuildAt) : null,
|
||||
rebuildSource: input.rebuildSource ?? null
|
||||
});
|
||||
}
|
||||
|
||||
private async getDelivery(eventId: string): Promise<BusinessEventDeliveryRecord | null> {
|
||||
const [row] = await db
|
||||
.select()
|
||||
.from(businessEventOutbox)
|
||||
.where(eq(businessEventOutbox.id, eventId))
|
||||
.limit(1);
|
||||
|
||||
return row ? mapDelivery(row) : null;
|
||||
}
|
||||
}
|
||||
|
||||
async function updateDeliveryStatus(
|
||||
eventId: string,
|
||||
deliveryStatus: 'completed' | 'failed' | 'dead_letter',
|
||||
now: Date,
|
||||
values: Partial<typeof businessEventOutbox.$inferInsert>
|
||||
): Promise<void> {
|
||||
await db
|
||||
.update(businessEventOutbox)
|
||||
.set({ ...values, deliveryStatus, updatedAt: now })
|
||||
.where(eq(businessEventOutbox.id, eventId));
|
||||
}
|
||||
|
||||
function mapDelivery(row: typeof businessEventOutbox.$inferSelect): BusinessEventDeliveryRecord {
|
||||
return {
|
||||
eventId: row.id,
|
||||
organizationId: row.organizationId,
|
||||
eventType: row.eventType,
|
||||
schemaVersion: row.schemaVersion,
|
||||
status: row.deliveryStatus as BusinessEventDeliveryRecord['status'],
|
||||
attemptCount: row.attemptCount,
|
||||
availableAt: row.availableAt.toISOString(),
|
||||
event: row.eventEnvelope as BusinessEvent
|
||||
};
|
||||
}
|
||||
|
||||
function mapCheckpoint(
|
||||
row: typeof projectionConsumerCheckpoints.$inferSelect
|
||||
): ProjectionCheckpointRecord {
|
||||
return {
|
||||
id: row.id,
|
||||
organizationId: row.organizationId,
|
||||
consumerName: row.consumerName,
|
||||
projectionName: row.projectionName,
|
||||
eventId: row.eventId,
|
||||
eventType: row.eventType,
|
||||
schemaVersion: row.schemaVersion,
|
||||
status: row.processingStatus as ProjectionCheckpointRecord['status'],
|
||||
attemptCount: row.attemptCount,
|
||||
nextRetryAt: row.nextRetryAt?.toISOString() ?? null,
|
||||
errorCode: row.errorCode,
|
||||
errorMessage: row.errorMessage
|
||||
};
|
||||
}
|
||||
214
src/features/foundation/projections/types.ts
Normal file
214
src/features/foundation/projections/types.ts
Normal file
@@ -0,0 +1,214 @@
|
||||
import type { BusinessEvent } from '../business-events/types.ts';
|
||||
|
||||
export type BusinessEventDeliveryStatus =
|
||||
| 'pending'
|
||||
| 'processing'
|
||||
| 'completed'
|
||||
| 'failed'
|
||||
| 'dead_letter';
|
||||
|
||||
export type ProjectionCheckpointStatus =
|
||||
| 'pending'
|
||||
| 'processing'
|
||||
| 'completed'
|
||||
| 'retry_scheduled'
|
||||
| 'failed'
|
||||
| 'dead_letter'
|
||||
| 'skipped_unsupported_version';
|
||||
|
||||
export type ProjectionHealthStatus =
|
||||
| 'healthy'
|
||||
| 'lagging'
|
||||
| 'degraded'
|
||||
| 'failed'
|
||||
| 'rebuilding'
|
||||
| 'paused';
|
||||
|
||||
export type ProjectionRebuildStrategy = 'source-query' | 'event-replay' | 'hybrid';
|
||||
export type ProjectionPersistenceStrategy = 'query-time' | 'cached-read-model' | 'materialized';
|
||||
export type ProjectionImplementationStatus = 'contract-only' | 'active' | 'deprecated';
|
||||
export type ProjectionHandleStatus = 'processed' | 'skipped';
|
||||
|
||||
export interface ProjectionHandleResult {
|
||||
status: ProjectionHandleStatus;
|
||||
message?: string;
|
||||
sideEffectCount?: number;
|
||||
}
|
||||
|
||||
export interface ProjectionConsumer<TEvent extends BusinessEvent = BusinessEvent> {
|
||||
readonly consumerName: string;
|
||||
readonly projectionName: string;
|
||||
readonly supportedEventTypes: string[];
|
||||
readonly supportedVersions: Record<string, number[]>;
|
||||
handle(event: TEvent): Promise<ProjectionHandleResult>;
|
||||
}
|
||||
|
||||
export interface ProjectionDefinition {
|
||||
projectionName: string;
|
||||
consumerName: string;
|
||||
owningFeature: string;
|
||||
consumedEventTypes: string[];
|
||||
supportedVersions: Record<string, number[]>;
|
||||
rebuildStrategy: ProjectionRebuildStrategy;
|
||||
persistence: ProjectionPersistenceStrategy;
|
||||
securityClassification: string;
|
||||
idempotencyKey: string;
|
||||
retryPolicy: string;
|
||||
retentionPolicy: string;
|
||||
implementationStatus: ProjectionImplementationStatus;
|
||||
}
|
||||
|
||||
export interface BusinessEventDeliveryRecord {
|
||||
eventId: string;
|
||||
organizationId: string;
|
||||
eventType: string;
|
||||
schemaVersion: number;
|
||||
status: BusinessEventDeliveryStatus;
|
||||
attemptCount: number;
|
||||
availableAt: string;
|
||||
event: BusinessEvent;
|
||||
}
|
||||
|
||||
export interface ProjectionCheckpointRecord {
|
||||
id: string;
|
||||
organizationId: string;
|
||||
consumerName: string;
|
||||
projectionName: string;
|
||||
eventId: string;
|
||||
eventType: string;
|
||||
schemaVersion: number;
|
||||
status: ProjectionCheckpointStatus;
|
||||
attemptCount: number;
|
||||
nextRetryAt?: string | null;
|
||||
errorCode?: string | null;
|
||||
errorMessage?: string | null;
|
||||
}
|
||||
|
||||
export interface ProjectionRuntimeStore {
|
||||
persistEvent(event: BusinessEvent): Promise<BusinessEventDeliveryRecord>;
|
||||
markDeliveryProcessing(eventId: string, now: Date): Promise<void>;
|
||||
markDeliveryCompleted(eventId: string, now: Date): Promise<void>;
|
||||
markDeliveryFailed(eventId: string, error: ProjectionFailure, now: Date): Promise<void>;
|
||||
markDeliveryDeadLetter(eventId: string, error: ProjectionFailure, now: Date): Promise<void>;
|
||||
getCheckpoint(consumerName: string, eventId: string): Promise<ProjectionCheckpointRecord | null>;
|
||||
beginCheckpoint(input: BeginProjectionCheckpointInput): Promise<ProjectionCheckpointRecord>;
|
||||
completeCheckpoint(input: CompleteProjectionCheckpointInput): Promise<void>;
|
||||
failCheckpoint(input: FailProjectionCheckpointInput): Promise<void>;
|
||||
skipUnsupportedVersion(input: UnsupportedProjectionCheckpointInput): Promise<void>;
|
||||
createDeadLetter(input: CreateProjectionDeadLetterInput): Promise<void>;
|
||||
recordHealthSnapshot(input: ProjectionHealthSnapshotInput): Promise<void>;
|
||||
}
|
||||
|
||||
export interface BeginProjectionCheckpointInput {
|
||||
organizationId: string;
|
||||
consumerName: string;
|
||||
projectionName: string;
|
||||
eventId: string;
|
||||
eventType: string;
|
||||
schemaVersion: number;
|
||||
now: Date;
|
||||
}
|
||||
|
||||
export interface CompleteProjectionCheckpointInput {
|
||||
consumerName: string;
|
||||
eventId: string;
|
||||
durationMs: number;
|
||||
now: Date;
|
||||
}
|
||||
|
||||
export interface FailProjectionCheckpointInput {
|
||||
organizationId: string;
|
||||
consumerName: string;
|
||||
projectionName: string;
|
||||
eventId: string;
|
||||
eventType: string;
|
||||
schemaVersion: number;
|
||||
failure: ProjectionFailure;
|
||||
nextRetryAt: Date | null;
|
||||
deadLetter: boolean;
|
||||
durationMs: number;
|
||||
now: Date;
|
||||
}
|
||||
|
||||
export interface UnsupportedProjectionCheckpointInput {
|
||||
organizationId: string;
|
||||
consumerName: string;
|
||||
projectionName: string;
|
||||
eventId: string;
|
||||
eventType: string;
|
||||
schemaVersion: number;
|
||||
now: Date;
|
||||
}
|
||||
|
||||
export interface ProjectionFailure {
|
||||
code: string;
|
||||
message: string;
|
||||
retryable: boolean;
|
||||
}
|
||||
|
||||
export interface CreateProjectionDeadLetterInput {
|
||||
organizationId: string;
|
||||
consumerName: string;
|
||||
projectionName: string;
|
||||
eventId: string;
|
||||
eventType: string;
|
||||
schemaVersion: number;
|
||||
attemptCount: number;
|
||||
failure: ProjectionFailure;
|
||||
now: Date;
|
||||
}
|
||||
|
||||
export interface ProjectionHealthSnapshotInput {
|
||||
organizationId: string;
|
||||
projectionName: string;
|
||||
consumerName: string;
|
||||
status: ProjectionHealthStatus;
|
||||
lastProcessedEventId?: string | null;
|
||||
lastSuccessfulProcessingAt?: string | null;
|
||||
pendingCount?: number;
|
||||
retryCount?: number;
|
||||
deadLetterCount?: number;
|
||||
unsupportedVersionCount?: number;
|
||||
processingLagMs?: number;
|
||||
averageProcessingDurationMs?: number;
|
||||
lastRebuildAt?: string | null;
|
||||
rebuildSource?: string | null;
|
||||
}
|
||||
|
||||
export interface ProjectionRuntimeResult {
|
||||
eventId: string;
|
||||
eventType: string;
|
||||
status: BusinessEventDeliveryStatus;
|
||||
consumerResults: ProjectionConsumerRuntimeResult[];
|
||||
}
|
||||
|
||||
export interface ProjectionConsumerRuntimeResult {
|
||||
consumerName: string;
|
||||
projectionName: string;
|
||||
status: ProjectionCheckpointStatus | 'duplicate_completed';
|
||||
durationMs: number;
|
||||
retryAt?: string;
|
||||
error?: ProjectionFailure;
|
||||
}
|
||||
|
||||
export interface ProjectionRebuildRequest {
|
||||
projectionName: string;
|
||||
organizationId: string;
|
||||
entityType?: string;
|
||||
entityId?: string;
|
||||
occurredFrom?: string;
|
||||
occurredTo?: string;
|
||||
dryRun?: boolean;
|
||||
resetExisting?: boolean;
|
||||
}
|
||||
|
||||
export interface ProjectionRebuildResult {
|
||||
rebuildRunId: string;
|
||||
projectionName: string;
|
||||
organizationId: string;
|
||||
dryRun: boolean;
|
||||
status: 'pending' | 'processing' | 'completed' | 'failed';
|
||||
processedCount: number;
|
||||
skippedCount: number;
|
||||
failedCount: number;
|
||||
}
|
||||
Reference in New Issue
Block a user