task ep.1.4
This commit is contained in:
971
plans/task-ep.1.4.md
Normal file
971
plans/task-ep.1.4.md
Normal file
@@ -0,0 +1,971 @@
|
||||
# Task EP.1.4 – Projection Foundation & Delivery Reliability
|
||||
|
||||
Status: Completed
|
||||
Priority: Critical
|
||||
Type: Platform Foundation / Projection Runtime / Delivery Reliability
|
||||
|
||||
## Depends On
|
||||
|
||||
* EP.1.1 Activity Domain Foundation
|
||||
* EP.1.2 Activity Integration & Legacy Follow-up Adapter
|
||||
* EP.1.3 Business Event Foundation
|
||||
* BU-R.0 Business Blueprint Freeze
|
||||
* BU-R.0.1 Workspace & Activity Business Blueprint Completion
|
||||
* BU-R.1 Business Capability Audit
|
||||
* AR.1 Architecture Transition Plan
|
||||
* AR.2 Epic & Technical Design
|
||||
* ENG.0 Engineering Constitution
|
||||
|
||||
---
|
||||
|
||||
# Objective
|
||||
|
||||
Introduce the shared Projection Foundation and reliable Business Event delivery model for ALLA OS.
|
||||
|
||||
This task establishes the runtime required for future Timeline, Calendar, Dashboard, Notification, My Day, Manager Workspace, Executive Workspace, Forecast, and Relationship Health projections.
|
||||
|
||||
The task must ensure that:
|
||||
|
||||
* successful source-domain mutations are not reported as failed merely because a projection consumer fails
|
||||
* projection consumers process events idempotently
|
||||
* failed projection processing is observable and retryable
|
||||
* projections can be rebuilt from governed source data
|
||||
* projection state remains derived and non-authoritative
|
||||
* current production CRM behavior remains backward compatible
|
||||
|
||||
This task does not implement final Timeline, Calendar, My Day, Manager Workspace, Executive Workspace, or dashboard replacement features.
|
||||
|
||||
---
|
||||
|
||||
# Background
|
||||
|
||||
EP.1.3 introduced:
|
||||
|
||||
* canonical `BusinessEvent`
|
||||
* machine-readable Event Registry
|
||||
* event ownership rules
|
||||
* in-process publisher and dispatcher
|
||||
* Activity lifecycle event publishing
|
||||
* Event Sequence Matrix
|
||||
* Event Consumer Matrix
|
||||
* replay contracts
|
||||
* versioning and idempotency contracts
|
||||
|
||||
The remaining reliability gaps are:
|
||||
|
||||
1. subscriber failure may currently cause publisher dispatch to fail after the source mutation has already succeeded
|
||||
2. idempotency is in-memory and does not survive process restart or multi-instance deployment
|
||||
3. no persistent event delivery or projection processing state exists
|
||||
4. replay is contract-only because there is no persistent event source
|
||||
5. projections do not yet have a shared runtime, checkpoint, rebuild, retry, or health model
|
||||
|
||||
EP.1.4 resolves these gaps before business-facing projection features are implemented.
|
||||
|
||||
---
|
||||
|
||||
# Architecture Principles
|
||||
|
||||
## 1. Source Mutation Independence
|
||||
|
||||
A committed source-domain mutation must not be rolled back or reported as unsuccessful solely because a non-owning projection consumer failed.
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
Activity completed successfully
|
||||
↓
|
||||
activity.completed emitted
|
||||
↓
|
||||
Timeline projection fails
|
||||
↓
|
||||
Activity remains completed
|
||||
Projection failure is recorded for retry
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Eventual Consistency
|
||||
|
||||
Timeline, Calendar, Dashboard, Notification, My Day, Forecast, and other projections are eventually consistent read models.
|
||||
|
||||
They must not become authoritative owners of source-domain lifecycle state.
|
||||
|
||||
---
|
||||
|
||||
## 3. Persistent Idempotency
|
||||
|
||||
Projection consumers must prevent duplicate side effects across:
|
||||
|
||||
* retries
|
||||
* process restarts
|
||||
* multi-instance deployment
|
||||
* repeated event delivery
|
||||
* manual rebuild operations
|
||||
|
||||
In-memory duplicate detection is insufficient for production projection consumers.
|
||||
|
||||
---
|
||||
|
||||
## 4. Rebuildable Projections
|
||||
|
||||
Every projection must be reproducible from governed source data, persisted business events, or an approved hybrid strategy.
|
||||
|
||||
Deleting or rebuilding a projection must not delete operational business records.
|
||||
|
||||
---
|
||||
|
||||
## 5. Preserve Before Replace
|
||||
|
||||
Current dashboard, report, follow-up, approval, notification, and detail-page behavior remains operational.
|
||||
|
||||
Projection consumers are introduced additively before any existing consumer is replaced.
|
||||
|
||||
---
|
||||
|
||||
## 6. Security at Projection Boundaries
|
||||
|
||||
Projection reads and writes must preserve:
|
||||
|
||||
* organization isolation
|
||||
* branch and product scope
|
||||
* CRM record visibility
|
||||
* internal-only visibility
|
||||
* pricing-sensitive redaction
|
||||
* source-domain authorization boundaries
|
||||
|
||||
Projection storage must not become a route around source security rules.
|
||||
|
||||
---
|
||||
|
||||
# Review Required
|
||||
|
||||
## Governance
|
||||
|
||||
* `AGENTS.md`
|
||||
* `docs/standards/engineering-constitution.md`
|
||||
* `docs/standards/project-foundations.md`
|
||||
* `docs/standards/architecture-rules.md`
|
||||
* `docs/standards/task-review-checklist.md`
|
||||
* `docs/security/crm-authorization-boundaries.md`
|
||||
|
||||
## Business and Architecture
|
||||
|
||||
* Relationship & Sales Workspace Blueprint
|
||||
* BU-R.1 Business Capability Audit
|
||||
* AR.1 Architecture Transition Plan
|
||||
* AR.2 Epic & Technical Design
|
||||
|
||||
## Existing Implementation
|
||||
|
||||
* EP.1.1 Activity Domain Foundation
|
||||
* EP.1.2 Activity Integration & Legacy Follow-up Adapter
|
||||
* EP.1.3 Business Event Foundation
|
||||
* Business Event Registry
|
||||
* Event Sequence Matrix
|
||||
* Event Consumer Matrix
|
||||
* in-process dispatcher
|
||||
* Activity event publishers
|
||||
* current Audit Log foundation
|
||||
* current Notification foundation
|
||||
* current Dashboard and Report services
|
||||
* current Approval event behavior
|
||||
* existing database transaction patterns
|
||||
* current background-job or scheduled-task foundations, if any
|
||||
|
||||
---
|
||||
|
||||
# Scope
|
||||
|
||||
## Part 1 — Delivery Semantics
|
||||
|
||||
Freeze the official event delivery semantics.
|
||||
|
||||
### Required Rules
|
||||
|
||||
* Business mutation commits first.
|
||||
* Non-owning consumer failure must not undo a committed source mutation.
|
||||
* Event delivery status must be distinguishable from mutation status.
|
||||
* Consumers may be synchronous internally, but projection processing must follow post-commit semantics.
|
||||
* Source services must not return a misleading mutation failure after successful commit.
|
||||
* Critical synchronous side effects, if any, must be explicitly approved and documented.
|
||||
|
||||
### Required Decision
|
||||
|
||||
Select and document the initial delivery strategy:
|
||||
|
||||
* Transactional Outbox
|
||||
* Post-commit persistent delivery record
|
||||
* Hybrid source-query plus event checkpoint strategy
|
||||
* another approach explicitly justified against AR.1 and ENG.0
|
||||
|
||||
Preferred direction:
|
||||
|
||||
> Transactional Outbox or an equivalent persistent post-commit delivery mechanism.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Persistent Event Delivery Model
|
||||
|
||||
Introduce persistent delivery records for Business Events.
|
||||
|
||||
Recommended responsibilities:
|
||||
|
||||
* persist canonical event envelope
|
||||
* record publication state
|
||||
* record availability for dispatch
|
||||
* record dispatch attempts
|
||||
* record successful completion
|
||||
* record dead-letter or terminal failure
|
||||
* preserve correlation and causation metadata
|
||||
* support organization-scoped querying
|
||||
|
||||
Recommended conceptual states:
|
||||
|
||||
```text
|
||||
pending
|
||||
processing
|
||||
completed
|
||||
failed
|
||||
dead_letter
|
||||
```
|
||||
|
||||
### Rules
|
||||
|
||||
* Event records are immutable business facts after publication.
|
||||
* Delivery metadata may change independently.
|
||||
* Event payload must continue respecting data minimization and pricing sensitivity.
|
||||
* A persistent delivery record does not make Event storage the owner of source lifecycle state.
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Transactional Outbox Decision and Foundation
|
||||
|
||||
Assess and, if approved, implement the Outbox foundation.
|
||||
|
||||
### Outbox Requirements
|
||||
|
||||
* source mutation and event persistence occur in the same database transaction
|
||||
* dispatcher processes only committed events
|
||||
* retries are safe
|
||||
* duplicate delivery is expected and handled
|
||||
* outbox cleanup or retention is governed
|
||||
* events are organization scoped
|
||||
* no external broker is required
|
||||
|
||||
### Deliverables
|
||||
|
||||
* Outbox architecture decision
|
||||
* schema or storage contract
|
||||
* transaction integration pattern
|
||||
* dispatcher polling or draining strategy
|
||||
* failure recovery behavior
|
||||
* retention recommendation
|
||||
|
||||
If Outbox implementation is deferred, an equivalent persistent reliability mechanism must be approved and documented.
|
||||
|
||||
---
|
||||
|
||||
## Part 4 — Projection Consumer Runtime
|
||||
|
||||
Introduce the shared Projection Consumer Runtime.
|
||||
|
||||
Minimum contract:
|
||||
|
||||
```ts
|
||||
export interface ProjectionConsumer<TEvent extends BusinessEvent = BusinessEvent> {
|
||||
readonly consumerName: string;
|
||||
readonly supportedEventTypes: string[];
|
||||
readonly supportedVersions: Record<string, number[]>;
|
||||
|
||||
handle(event: TEvent): Promise<ProjectionHandleResult>;
|
||||
}
|
||||
```
|
||||
|
||||
Runtime responsibilities:
|
||||
|
||||
* resolve registered consumers
|
||||
* validate supported event version
|
||||
* acquire idempotency guard
|
||||
* execute consumer
|
||||
* record result
|
||||
* release or finalize checkpoint
|
||||
* record duration and failure details
|
||||
* schedule retry when appropriate
|
||||
|
||||
---
|
||||
|
||||
## Part 5 — Persistent Consumer Checkpoints
|
||||
|
||||
Introduce persistent consumer processing state.
|
||||
|
||||
Minimum uniqueness:
|
||||
|
||||
```text
|
||||
consumerName + eventId
|
||||
```
|
||||
|
||||
Recommended data:
|
||||
|
||||
* consumer name
|
||||
* event id
|
||||
* event type
|
||||
* schema version
|
||||
* organization id
|
||||
* processing status
|
||||
* attempt count
|
||||
* first attempted at
|
||||
* last attempted at
|
||||
* completed at
|
||||
* next retry at
|
||||
* error code
|
||||
* sanitized error message
|
||||
* processing duration
|
||||
|
||||
### Required States
|
||||
|
||||
```text
|
||||
pending
|
||||
processing
|
||||
completed
|
||||
retry_scheduled
|
||||
failed
|
||||
dead_letter
|
||||
skipped_unsupported_version
|
||||
```
|
||||
|
||||
### Rules
|
||||
|
||||
* completed checkpoints prevent duplicate projection effects
|
||||
* failed consumers may retry independently
|
||||
* one consumer failure must not block unrelated consumers permanently
|
||||
* checkpoints remain technical delivery state, not business lifecycle state
|
||||
|
||||
---
|
||||
|
||||
## Part 6 — Retry Policy
|
||||
|
||||
Freeze retry behavior.
|
||||
|
||||
Determine:
|
||||
|
||||
* retryable error categories
|
||||
* non-retryable error categories
|
||||
* maximum attempts
|
||||
* exponential or fixed backoff
|
||||
* jitter
|
||||
* retry scheduling
|
||||
* dead-letter threshold
|
||||
* manual retry support
|
||||
* organization-level safety limits
|
||||
|
||||
Recommended defaults must be documented rather than hidden in code.
|
||||
|
||||
### Example
|
||||
|
||||
```text
|
||||
attempt 1: immediate
|
||||
attempt 2: +1 minute
|
||||
attempt 3: +5 minutes
|
||||
attempt 4: +30 minutes
|
||||
attempt 5: dead letter
|
||||
```
|
||||
|
||||
Exact values must align with existing operational standards.
|
||||
|
||||
---
|
||||
|
||||
## Part 7 — Dead-Letter and Failure Records
|
||||
|
||||
Introduce a governed dead-letter model for projection failures that cannot be processed automatically.
|
||||
|
||||
Required capabilities:
|
||||
|
||||
* retain failed event reference
|
||||
* retain consumer name
|
||||
* retain sanitized failure information
|
||||
* retain attempt history
|
||||
* support manual review
|
||||
* support manual retry
|
||||
* support mark-resolved with reason
|
||||
* prevent sensitive payload exposure
|
||||
|
||||
No user-facing operations UI is required in this task unless explicitly approved.
|
||||
|
||||
An admin/service interface and documentation are sufficient.
|
||||
|
||||
---
|
||||
|
||||
## Part 8 — Projection Registry
|
||||
|
||||
Create the machine-readable Projection Registry.
|
||||
|
||||
Every projection definition must document:
|
||||
|
||||
* projection name
|
||||
* owning feature
|
||||
* consumed event types
|
||||
* supported versions
|
||||
* rebuild source
|
||||
* persistence strategy
|
||||
* security classification
|
||||
* idempotency key
|
||||
* retry policy
|
||||
* retention policy
|
||||
* implementation status
|
||||
|
||||
Minimum planned projections:
|
||||
|
||||
* Timeline
|
||||
* Calendar
|
||||
* Dashboard
|
||||
* Notification
|
||||
* My Day
|
||||
* Manager Workspace
|
||||
* Executive Workspace
|
||||
* Relationship Health
|
||||
* Forecast
|
||||
|
||||
### Example
|
||||
|
||||
```ts
|
||||
export type ProjectionDefinition = {
|
||||
projectionName: string;
|
||||
consumerName: string;
|
||||
consumedEventTypes: string[];
|
||||
supportedVersions: Record<string, number[]>;
|
||||
rebuildStrategy: "source-query" | "event-replay" | "hybrid";
|
||||
persistence: "query-time" | "cached-read-model" | "materialized";
|
||||
securityClassification: string;
|
||||
implementationStatus: "contract-only" | "active" | "deprecated";
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 9 — Projection Storage Standards
|
||||
|
||||
Freeze the allowed projection storage patterns.
|
||||
|
||||
Supported patterns:
|
||||
|
||||
### Query-Time Projection
|
||||
|
||||
* calculated from source domains when requested
|
||||
* no persisted read-model table
|
||||
* suitable for low-volume or early rollout
|
||||
|
||||
### Cached Read Model
|
||||
|
||||
* persisted derivative data
|
||||
* invalidated or updated by events
|
||||
* fully rebuildable
|
||||
|
||||
### Materialized Projection
|
||||
|
||||
* optimized projection table or materialized view
|
||||
* suitable for high-volume Timeline, Calendar, Dashboard, or reporting use cases
|
||||
* never authoritative
|
||||
|
||||
### Rules
|
||||
|
||||
* projection tables must include organization scope
|
||||
* projection data must be rebuildable
|
||||
* source IDs and event provenance must be retained
|
||||
* unique constraints must protect against duplicate projection rows
|
||||
* deletion of projection data must not delete source records
|
||||
* projection schema changes require rebuild compatibility analysis
|
||||
|
||||
---
|
||||
|
||||
## Part 10 — Initial Projection Skeletons
|
||||
|
||||
Create contracts and no-op or test projection consumers for:
|
||||
|
||||
* Timeline
|
||||
* Calendar
|
||||
* Notification
|
||||
* Dashboard
|
||||
|
||||
These consumers must validate:
|
||||
|
||||
* registration
|
||||
* version support
|
||||
* idempotency
|
||||
* checkpointing
|
||||
* retry behavior
|
||||
* consumer isolation
|
||||
|
||||
They must not yet implement business-facing projection behavior.
|
||||
|
||||
No final projection tables or UI are required.
|
||||
|
||||
---
|
||||
|
||||
## Part 11 — Hybrid Rebuild Strategy
|
||||
|
||||
Freeze the initial rebuild strategy.
|
||||
|
||||
Recommended strategy:
|
||||
|
||||
```text
|
||||
Initial build
|
||||
↓
|
||||
Governed source queries + legacy adapters
|
||||
|
||||
Incremental updates
|
||||
↓
|
||||
Business Events
|
||||
|
||||
Recovery
|
||||
↓
|
||||
Source rebuild + event checkpoint reconciliation
|
||||
```
|
||||
|
||||
This hybrid strategy is preferred because:
|
||||
|
||||
* legacy follow-up events were not historically persisted
|
||||
* current source tables remain authoritative
|
||||
* Activity and new domain events can support incremental projection updates
|
||||
* migration risk remains controlled
|
||||
|
||||
### Required Outputs
|
||||
|
||||
* source-query rebuild contract
|
||||
* event-based incremental contract
|
||||
* legacy adapter role
|
||||
* reconciliation rules
|
||||
* cutover criteria
|
||||
* rollback behavior
|
||||
|
||||
---
|
||||
|
||||
## Part 12 — Projection Rebuild Contract
|
||||
|
||||
Implement or define a shared rebuild contract.
|
||||
|
||||
Suggested interface:
|
||||
|
||||
```ts
|
||||
export type ProjectionRebuildRequest = {
|
||||
projectionName: string;
|
||||
organizationId: string;
|
||||
entityType?: string;
|
||||
entityId?: string;
|
||||
occurredFrom?: string;
|
||||
occurredTo?: string;
|
||||
dryRun?: boolean;
|
||||
resetExisting?: boolean;
|
||||
};
|
||||
```
|
||||
|
||||
Required behavior:
|
||||
|
||||
* organization scoped
|
||||
* permission protected
|
||||
* dry-run supported
|
||||
* progress report supported
|
||||
* idempotent
|
||||
* restartable
|
||||
* source strategy recorded
|
||||
* existing projection reset explicitly controlled
|
||||
|
||||
No public UI is required.
|
||||
|
||||
---
|
||||
|
||||
## Part 13 — Projection Health Model
|
||||
|
||||
Define projection operational health.
|
||||
|
||||
Suggested statuses:
|
||||
|
||||
```text
|
||||
healthy
|
||||
lagging
|
||||
degraded
|
||||
failed
|
||||
rebuilding
|
||||
paused
|
||||
```
|
||||
|
||||
Track:
|
||||
|
||||
* last processed event
|
||||
* last successful processing time
|
||||
* pending count
|
||||
* retry count
|
||||
* dead-letter count
|
||||
* unsupported-version count
|
||||
* processing lag
|
||||
* average processing duration
|
||||
* last rebuild time
|
||||
* rebuild source
|
||||
|
||||
Projection health must be observable without exposing sensitive business payloads.
|
||||
|
||||
---
|
||||
|
||||
## Part 14 — Delivery Observability
|
||||
|
||||
Add structured observability for:
|
||||
|
||||
* event persisted
|
||||
* event ready
|
||||
* dispatch started
|
||||
* dispatch completed
|
||||
* consumer started
|
||||
* consumer completed
|
||||
* consumer retry scheduled
|
||||
* consumer failed
|
||||
* dead-letter created
|
||||
* duplicate skipped
|
||||
* unsupported version skipped
|
||||
* projection rebuild started
|
||||
* projection rebuild completed
|
||||
* projection rebuild failed
|
||||
|
||||
Required metadata:
|
||||
|
||||
* event id
|
||||
* event type
|
||||
* schema version
|
||||
* organization id
|
||||
* correlation id
|
||||
* causation id
|
||||
* consumer name
|
||||
* projection name
|
||||
* attempt
|
||||
* duration
|
||||
* status
|
||||
* sanitized error code
|
||||
|
||||
---
|
||||
|
||||
## Part 15 — Security and Data Redaction
|
||||
|
||||
Freeze projection security rules.
|
||||
|
||||
### Required Rules
|
||||
|
||||
* Projection persistence must not widen record visibility.
|
||||
* Pricing-sensitive event payloads must remain redacted.
|
||||
* Consumers needing full details must re-query through governed source services or security-aware query modules.
|
||||
* Internal-only Activities must not leak into general projections.
|
||||
* Organization scope must be mandatory.
|
||||
* Manager and Executive consumers must still obey branch, product, role, and pricing boundaries.
|
||||
* Dead-letter and failure records must not persist unrestricted sensitive payloads.
|
||||
|
||||
Produce a security matrix for every planned projection.
|
||||
|
||||
---
|
||||
|
||||
## Part 16 — Existing Consumer Compatibility
|
||||
|
||||
Document compatibility with:
|
||||
|
||||
* current approval notification events
|
||||
* current notification inbox
|
||||
* current Dashboard datasets
|
||||
* current Report datasets
|
||||
* legacy lead follow-ups
|
||||
* legacy opportunity follow-ups
|
||||
* legacy quotation follow-ups
|
||||
* recent-activity compatibility reads
|
||||
|
||||
### Frozen Rule
|
||||
|
||||
No current production consumer is replaced in EP.1.4.
|
||||
|
||||
The new runtime operates beside existing consumers until later projection-specific tasks approve cutover.
|
||||
|
||||
---
|
||||
|
||||
## Part 17 — Projection Delivery Matrix
|
||||
|
||||
Create the official Projection Delivery Matrix.
|
||||
|
||||
Minimum columns:
|
||||
|
||||
| Projection | Event Source | Initial Build Source | Incremental Strategy | Persistence | Idempotency | Retry | Cutover Phase |
|
||||
| ------------------- | --------------------------------------------- | ------------------------- | -------------------- | ---------------------------- | ----------- | -------- | ------------------------ |
|
||||
| Timeline | Activity + CRM events | source queries + adapters | Business Events | query-time or cached | required | required | Timeline epic |
|
||||
| Calendar | Activity + milestones + approval | source queries | Business Events | cached/query-time | required | required | Calendar epic |
|
||||
| Dashboard | Opportunity + Quotation + Approval + Activity | existing datasets | hybrid | existing + future read model | required | required | Dashboard enhancement |
|
||||
| Notification | Activity + Approval + CRM events | not applicable | Business Events | notification records | required | required | Notification expansion |
|
||||
| My Day | Activity + approvals + Hot Project | source composition | hybrid | query-time/cached | required | required | My Day epic |
|
||||
| Relationship Health | Customer + Activity + Opportunity | source rebuild | Business Events | cached score | required | required | Relationship Health epic |
|
||||
| Forecast | Opportunity events | source opportunity query | hybrid | report/read model | required | required | Forecast enhancement |
|
||||
|
||||
The final matrix must reflect repository-specific decisions.
|
||||
|
||||
---
|
||||
|
||||
## Part 18 — Delivery Failure Matrix
|
||||
|
||||
Create the official Delivery Failure Matrix.
|
||||
|
||||
Minimum scenarios:
|
||||
|
||||
| Failure Scenario | Source Mutation Result | Event State | Consumer State | Retry | User/API Result |
|
||||
| ------------------------------------------------- | ---------------------- | ----------- | -------------------------- | -------------- | ----------------- |
|
||||
| Event persistence fails inside source transaction | rollback | none | none | no | mutation fails |
|
||||
| Source mutation commits, consumer fails | committed | persisted | retry scheduled | yes | mutation succeeds |
|
||||
| One of multiple consumers fails | committed | persisted | isolated failure | yes | mutation succeeds |
|
||||
| Unsupported event version | committed | persisted | skipped/failed by consumer | governed | mutation succeeds |
|
||||
| Duplicate event delivered | committed | persisted | duplicate skipped | no side effect | unchanged |
|
||||
| Projection storage unavailable | committed | persisted | retry scheduled | yes | mutation succeeds |
|
||||
| Dead-letter threshold reached | committed | persisted | dead letter | manual | mutation succeeds |
|
||||
| Rebuild fails | unchanged | unchanged | rebuild failed | restartable | no source impact |
|
||||
|
||||
This matrix becomes the reliability baseline for future projection tasks.
|
||||
|
||||
---
|
||||
|
||||
# Deliverables
|
||||
|
||||
## 1. Delivery Semantics Decision
|
||||
|
||||
Official post-commit and eventual-consistency behavior.
|
||||
|
||||
## 2. Persistent Event Delivery Foundation
|
||||
|
||||
Event persistence and delivery state.
|
||||
|
||||
## 3. Transactional Outbox or Equivalent Reliability Mechanism
|
||||
|
||||
Approved and implemented foundation.
|
||||
|
||||
## 4. Projection Consumer Runtime
|
||||
|
||||
Shared processing runtime.
|
||||
|
||||
## 5. Persistent Consumer Checkpoints
|
||||
|
||||
Production-grade idempotency and processing state.
|
||||
|
||||
## 6. Retry Policy
|
||||
|
||||
Governed retry and backoff behavior.
|
||||
|
||||
## 7. Dead-Letter Foundation
|
||||
|
||||
Terminal failure handling and manual recovery contract.
|
||||
|
||||
## 8. Projection Registry
|
||||
|
||||
Machine-readable projection catalog.
|
||||
|
||||
## 9. Projection Storage Standards
|
||||
|
||||
Query-time, cached, and materialized projection rules.
|
||||
|
||||
## 10. Initial Projection Consumer Skeletons
|
||||
|
||||
Timeline, Calendar, Dashboard, and Notification runtime validation consumers.
|
||||
|
||||
## 11. Hybrid Rebuild Strategy
|
||||
|
||||
Source-query initial build plus event-driven incremental update.
|
||||
|
||||
## 12. Projection Rebuild Contract
|
||||
|
||||
Dry-run, scoped, restartable rebuild foundation.
|
||||
|
||||
## 13. Projection Health Model
|
||||
|
||||
Lag, failure, retry, dead-letter, and rebuild visibility.
|
||||
|
||||
## 14. Delivery Observability
|
||||
|
||||
Structured event and consumer processing telemetry.
|
||||
|
||||
## 15. Projection Security Matrix
|
||||
|
||||
Visibility, pricing, internal-only, and organization-boundary rules.
|
||||
|
||||
## 16. Existing Consumer Compatibility Report
|
||||
|
||||
No-regression and cutover readiness documentation.
|
||||
|
||||
## 17. Projection Delivery Matrix
|
||||
|
||||
Official source, persistence, retry, and cutover mapping.
|
||||
|
||||
## 18. Delivery Failure Matrix
|
||||
|
||||
Official behavior for mutation, dispatch, consumer, and rebuild failures.
|
||||
|
||||
## 19. Projection Readiness Report
|
||||
|
||||
Readiness for Timeline, Calendar, Notification, Dashboard, My Day, Manager, Executive, Forecast, and Relationship Health tasks.
|
||||
|
||||
---
|
||||
|
||||
# Constraints
|
||||
|
||||
Must preserve:
|
||||
|
||||
* Customer
|
||||
* Contact
|
||||
* Lead
|
||||
* Opportunity
|
||||
* Quotation
|
||||
* Approval
|
||||
* Activity
|
||||
* current Activity APIs
|
||||
* legacy follow-up APIs and storage
|
||||
* recent-activity compatibility reads
|
||||
* current Dashboard and Report datasets
|
||||
* current Notification behavior
|
||||
* current Approval notification publishing
|
||||
* current Audit Log behavior
|
||||
* current security and pricing visibility boundaries
|
||||
|
||||
Do not implement:
|
||||
|
||||
* final Timeline projection
|
||||
* Timeline UI
|
||||
* final Calendar projection
|
||||
* Calendar UI
|
||||
* Dashboard source replacement
|
||||
* Notification business-event expansion
|
||||
* My Day
|
||||
* Manager Workspace
|
||||
* Executive Workspace
|
||||
* Relationship Health calculation
|
||||
* Forecast replacement
|
||||
* Automation rules
|
||||
* external Kafka, RabbitMQ, or Redis Streams infrastructure
|
||||
* public replay UI
|
||||
* legacy follow-up migration
|
||||
* dual-write from follow-up to Activity
|
||||
* source-domain replacement
|
||||
|
||||
No breaking API changes.
|
||||
|
||||
---
|
||||
|
||||
# Compatibility Strategy
|
||||
|
||||
The Projection Foundation is additive.
|
||||
|
||||
Current production consumers remain active.
|
||||
|
||||
Business Events may be persisted and processed through the new delivery runtime without changing existing API responses.
|
||||
|
||||
Projection skeleton consumers must not produce user-visible duplicate records.
|
||||
|
||||
Any future cutover from legacy datasets to a projection requires:
|
||||
|
||||
* parity validation
|
||||
* regression tests
|
||||
* reconciliation report
|
||||
* feature flag or controlled rollout
|
||||
* rollback strategy
|
||||
* explicit implementation task approval
|
||||
|
||||
---
|
||||
|
||||
# Testing Requirements
|
||||
|
||||
## Delivery Tests
|
||||
|
||||
* source transaction plus event persistence succeeds atomically
|
||||
* failed event persistence rolls back source transaction
|
||||
* committed source mutation survives consumer failure
|
||||
* multiple consumers process independently
|
||||
* no-subscriber event completes safely
|
||||
* persistent event resumes after process restart simulation
|
||||
|
||||
## Checkpoint Tests
|
||||
|
||||
* first processing succeeds
|
||||
* duplicate event is skipped
|
||||
* failed attempt schedules retry
|
||||
* completed checkpoint prevents duplicate side effect
|
||||
* unsupported version is recorded
|
||||
* concurrent processing does not execute twice
|
||||
|
||||
## Retry Tests
|
||||
|
||||
* retryable failure
|
||||
* non-retryable failure
|
||||
* backoff scheduling
|
||||
* maximum-attempt behavior
|
||||
* dead-letter creation
|
||||
* manual retry contract
|
||||
|
||||
## Projection Runtime Tests
|
||||
|
||||
* consumer registration
|
||||
* consumer version support
|
||||
* consumer isolation
|
||||
* projection registry lookup
|
||||
* projection health updates
|
||||
* sanitized failure recording
|
||||
|
||||
## Rebuild Tests
|
||||
|
||||
* organization-scoped dry run
|
||||
* source-query rebuild
|
||||
* restartable rebuild
|
||||
* reset-existing protection
|
||||
* duplicate-safe rebuild
|
||||
* failed rebuild does not affect source data
|
||||
|
||||
## Security Tests
|
||||
|
||||
* organization isolation
|
||||
* branch/product scope enforcement
|
||||
* pricing-sensitive redaction
|
||||
* internal-only activity exclusion
|
||||
* unauthorized rebuild rejection
|
||||
* sensitive payload not exposed in dead-letter records
|
||||
|
||||
## Compatibility Tests
|
||||
|
||||
* existing Activity API behavior unchanged
|
||||
* current follow-up APIs unchanged
|
||||
* existing Dashboard datasets unchanged
|
||||
* existing Reports unchanged
|
||||
* existing approval notifications unchanged
|
||||
* existing recent-activity reads unchanged
|
||||
|
||||
---
|
||||
|
||||
# Acceptance Criteria
|
||||
|
||||
* Source mutations and non-owning projection failures are operationally separated.
|
||||
* A committed source mutation is not reported as failed because a projection consumer failed.
|
||||
* Business Events have a persistent delivery or outbox mechanism.
|
||||
* Projection consumers use persistent checkpoints.
|
||||
* Duplicate event processing does not create duplicate side effects.
|
||||
* Retry and dead-letter behavior is implemented and tested.
|
||||
* Projection consumers remain independently recoverable.
|
||||
* Projection Registry is machine-readable.
|
||||
* Projection storage rules are frozen.
|
||||
* Timeline, Calendar, Dashboard, and Notification skeleton consumers validate the runtime without creating final business-facing projections.
|
||||
* Hybrid source rebuild and event-driven incremental strategy is documented.
|
||||
* Projection rebuild contract supports dry-run and organization scope.
|
||||
* Projection health is measurable.
|
||||
* Delivery observability captures processing and failure state.
|
||||
* Projection security rules preserve all current CRM authorization boundaries.
|
||||
* Existing Dashboard, Report, Notification, Approval, Follow-up, and Activity behavior remains unchanged.
|
||||
* No final workspace or projection UI is introduced.
|
||||
* TypeScript, migration generation, targeted lint, and projection/runtime tests pass.
|
||||
|
||||
---
|
||||
|
||||
# Success Criteria
|
||||
|
||||
ALLA OS has a reliable, production-ready Projection Foundation.
|
||||
|
||||
Business Events can be delivered and processed without coupling projection failure to source-domain mutation success.
|
||||
|
||||
Projection consumers can process events idempotently, retry safely, expose health, recover from failure, and rebuild from governed source data.
|
||||
|
||||
Future Timeline, Calendar, Notification, Dashboard, My Day, Manager Workspace, Executive Workspace, Relationship Health, and Forecast tasks can focus on business read models rather than re-solving event delivery reliability.
|
||||
|
||||
Ready for:
|
||||
|
||||
# EP.1.5 – Timeline Projection Foundation
|
||||
Reference in New Issue
Block a user