task-d.5.1

docs/implementation/task-d51-lead-foundation-stabilization.md

@@ -0,0 +1,140 @@
+# Task D.5.1: Lead Foundation Stabilization & Output Completion
+
+## Objective
+
+Audit the existing D.5 phase-1 lead foundation work, avoid duplicate schema creation, complete missing registration outputs, and confirm current CRM compatibility before D.5.2 lead API work begins.
+
+## Review Completed
+
+- `AGENTS.md`
+- `docs/standards/project-foundations.md`
+- `docs/standards/task-catalog.md`
+- `docs/adr/0018-lead-enquiry-domain-separation.md`
+- `docs/implementation/task-d5-lead-enquiry-domain-separation.md`
+- `src/db/schema.ts`
+- `src/db/seeds/foundation.seed.ts`
+- `drizzle/0020_lead_foundation_schema.sql`
+- `drizzle/0021_worried_nebula.sql`
+- `drizzle/meta/_journal.json`
+- `src/features/foundation/master-options/types.ts`
+- `src/features/foundation/document-sequence/service.ts`
+
+## Existing Foundation
+
+Verified present in the repository:
+
+- `crm_leads` exists in [schema.ts](C:/Users/mtpphtaps/Documents/gitea/alla-allaos-fullstack/src/db/schema.ts)
+- `crm_enquiries.lead_id` exists in [schema.ts](C:/Users/mtpphtaps/Documents/gitea/alla-allaos-fullstack/src/db/schema.ts)
+- ADR-0018 exists at [0018-lead-enquiry-domain-separation.md](C:/Users/mtpphtaps/Documents/gitea/alla-allaos-fullstack/docs/adr/0018-lead-enquiry-domain-separation.md)
+- lead seed categories already exist in [foundation.seed.ts](C:/Users/mtpphtaps/Documents/gitea/alla-allaos-fullstack/src/db/seeds/foundation.seed.ts):
+  - `crm_lead_awareness`
+  - `crm_lead_status`
+  - `crm_lead_followup_status`
+  - `crm_lead_lost_reason`
+- document sequence defaults already include:
+  - `crm_lead`
+  - `crm_enquiry`
+
+## Missing Foundation Found
+
+The missing outputs were not schema objects. They were governance and contract gaps:
+
+- no dedicated Lead Foundation entry in `project-foundations.md`
+- no Task D.5 or Task D.5.1 registration in `task-catalog.md`
+- master-option category type registry did not include the new lead option categories, which made the seed data harder to discover and reuse from foundation settings code
+- no D.5.1 implementation summary existed
+
+## Partial / Compatibility Findings
+
+### Migration history duplication risk
+
+Lead schema work appears twice in migration history:
+
+- [0020_lead_foundation_schema.sql](C:/Users/mtpphtaps/Documents/gitea/alla-allaos-fullstack/drizzle/0020_lead_foundation_schema.sql)
+- [0021_worried_nebula.sql](C:/Users/mtpphtaps/Documents/gitea/alla-allaos-fullstack/drizzle/0021_worried_nebula.sql)
+
+Compatibility status:
+
+- `0020` is idempotent with `IF NOT EXISTS`
+- `0021` is a later non-idempotent generated migration for the same structural slice
+- D.5.1 did not create any additional migration, in line with task rules
+
+This is recorded as a migration-history cleanup risk for future maintenance, not changed by this task.
+
+### Sequence compatibility
+
+The foundation now includes both legacy and future-facing enquiry sequence types:
+
+- legacy active enquiry service still generates `documentType = enquiry`
+- lead split foundation seed also registers `crm_enquiry`
+
+That means the seed layer is ready for split-domain work, but production enquiry creation is still compatible with the older enquiry document-type contract until D.5.2+ refactors the service layer.
+
+### Active production model
+
+Current CRM production behavior is still primarily driven by the old shared enquiry foundation:
+
+- lead and enquiry workspace separation still relies on `crm_enquiries.pipeline_stage`
+- no lead API, lead UI, or lead assignment flow exists yet
+- dashboard and reports still rely on the pre-split data model
+
+This is expected and remains in scope for later D.5 phases.
+
+## Changes Completed In D.5.1
+
+- added Lead Foundation registration to `docs/standards/project-foundations.md`
+- added Task D.5 and Task D.5.1 to `docs/standards/task-catalog.md`
+- extended `CRM_MASTER_OPTION_CATEGORIES` to include the new lead option categories in `src/features/foundation/master-options/types.ts`
+- added this implementation note
+
+## No Duplicate Creation Performed
+
+D.5.1 intentionally did not:
+
+- recreate `crm_leads`
+- recreate `crm_enquiries.lead_id`
+- generate a new migration
+- rename existing schema objects
+- create lead APIs or UI
+
+## Compatibility Verification
+
+Verification executed:
+
+- `npm exec tsc --noEmit`
+
+Result:
+
+- current Customer, Contact, Enquiry, Quotation, Dashboard, and Reports code continues to typecheck after the D.5.1 stabilization changes
+
+## Output Summary
+
+### Existing Foundation
+
+- `crm_leads`
+- `crm_enquiries.lead_id`
+- `ADR-0018`
+
+### Missing Foundation
+
+- Lead Foundation registry entry
+- Task catalog registration for D.5 and D.5.1
+- master-option category contract coverage for lead option seeds
+- D.5.1 implementation summary
+
+### Completed Foundation
+
+- Lead schema and seed presence verified
+- lead option categories registered in foundation type contracts
+- Lead Foundation registered in the standards registry
+- Task D.5 and D.5.1 registered in task history
+- compatibility state documented for D.5.2 planning
+
+## Ready For Next Task
+
+The repo is now better prepared for `Task D.5.2 Lead API Foundation` because:
+
+- the foundation state is discoverable
+- duplicate schema creation was avoided
+- lead option categories are registered in the shared master-option contract
+- current compatibility boundaries are explicitly documented

docs/standards/project-foundations.md

@@ -236,6 +236,19 @@ If ADR coverage is incomplete, this registry marks the gap explicitly.
 | Reuse status | Reusable and active. |
 | Governance note | `docs/implementation/task-l31-customer-contact-approval-visibility.md` contains an outdated limitation stating persisted contact sharing does not exist. Treat ADR-0015 and Task C.1 as the current source of truth. |
 
+### Lead Foundation
+
+| Field | Details |
+| --- | --- |
+| Purpose | Dedicated marketing-owned lead domain, including lead identity, awareness source, marketing follow-up status, lead outcome state, and future `1 lead -> N enquiries` linkage |
+| Location | `src/db/schema.ts` (`crm_leads`), `drizzle/0020_lead_foundation_schema.sql`, `drizzle/0021_worried_nebula.sql`, `src/db/seeds/foundation.seed.ts`, `docs/adr/0018-lead-enquiry-domain-separation.md` |
+| Related tasks | Task D.5, Task D.5.1 |
+| Related ADRs | ADR-0018 |
+| Reusable services | None production-ready yet. D.5.1 only stabilizes schema and seed foundations before D.5.2 API work. |
+| Reusable APIs | None yet. Lead API surface is explicitly deferred to Task D.5.2. |
+| Reuse status | Foundation introduced and partially stabilized. Schema and seed contracts exist, but feature services, APIs, UI, migration flows, and separated datasets are not complete yet. |
+| Governance note | This foundation supersedes the long-term direction of the old single-record lead/enquiry model in ADR-0011, but current production behavior still largely runs on the shared `crm_enquiries` implementation until later D.5 phases are completed. |
+
 ### Lead / Enquiry Foundation
 
 | Field | Details |
@@ -247,6 +260,7 @@ If ADR coverage is incomplete, this registry marks the gap explicitly.
 | Reusable services | `getEnquiryReferenceData()`, `listEnquiries()`, `getEnquiryDetail()`, `createEnquiry()`, `updateEnquiry()`, `softDeleteEnquiry()`, `assignEnquiryToUser()`, `reassignEnquiryToUser()`, `listEnquiryProjectParties()` |
 | Reusable APIs | `/api/crm/enquiries`, `/api/crm/enquiries/[id]`, `/api/crm/enquiries/[id]/assign`, `/api/crm/enquiries/[id]/reassign`, `/api/crm/enquiries/[id]/customers` |
 | Reuse status | Reusable and active. This is the source of truth for lead versus enquiry separation. |
+| Governance note | ADR-0018 introduces the target split-domain direction, but this shared enquiry foundation still describes the active production service layer until D.5 migration phases complete. |
 
 ### Won / Lost Governance Foundation
 
@@ -350,6 +364,7 @@ If ADR coverage is incomplete, this registry marks the gap explicitly.
 
 - Approvals: Approval Foundation, Authorization Foundation, Audit Foundation, PDF Foundation
 - CRM leads and enquiries: Lead / Enquiry Foundation, Won / Lost Governance Foundation, Follow-Up Foundation, Authorization Foundation
+- CRM lead split-domain migration: Lead Foundation, Lead / Enquiry Foundation, Won / Lost Governance Foundation, Authorization Foundation, Reporting Foundation, Dashboard Foundation
 - Customers and contacts: Customer Ownership Foundation, Contact Sharing Foundation, Authorization Foundation
 - Dashboard analytics: Dashboard Foundation, Revenue Attribution Foundation, Follow-Up Foundation, Reporting Foundation
 - Forms and filters: CRM Form Design Foundation, Master Data Foundation

docs/standards/task-catalog.md

@@ -52,6 +52,7 @@ Before changing any feature, foundation, report, PDF flow, or authorization beha
 | Role Management Foundation | Task L | Task L.1, Task L.2 |
 | CRM Role Assignment Foundation | Task L.1 | Task L.2 |
 | Authorization Foundation | Task L | Task L.1, Task L.2, Task L.3, Task L.3.1 |
+| Lead Foundation | Task D.5 | Task D.5.1, Task D.5.2 |
 | Follow-Up Foundation | Task D plus Task E | Task J, Task K.2 |
 | Revenue Attribution Foundation | Task D.2.1 | Task J, Task K.1, Task K.2 |
 
@@ -71,6 +72,7 @@ Before changing any feature, foundation, report, PDF flow, or authorization beha
 | ADR-0015 Customer Ownership and Contact Sharing | Task C.1, Task L.3.1 |
 | ADR-0016 Won / Lost Lifecycle Governance | Task D.4, Task J, Task K.2 |
 | ADR-0017 CRM Report Foundation | Task K.1, Task K.2 |
+| ADR-0018 Lead / Enquiry Domain Separation | Task D.5, Task D.5.1 |
 
 ## Task Dependency Map
 
@@ -85,6 +87,8 @@ Before changing any feature, foundation, report, PDF flow, or authorization beha
 - Task D.3 depends on: Task D, Task D.1
 - Task D.3.1 depends on: Task D.3
 - Task D.4 depends on: Task D, Task D.3
+- Task D.5 depends on: Task D.3, Task D.4, Task J, Task K.2, Task L.3
+- Task D.5.1 depends on: Task D.5
 - Task E depends on: Task D, Task B foundations
 - Task E.1 depends on: Task E
 - Task F depends on: Task E, Task E.1
@@ -269,6 +273,32 @@ Before changing any feature, foundation, report, PDF flow, or authorization beha
 - Key Files: `docs/implementation/task-d4-won-lost-lifecycle-governance.md`, `docs/adr/0016-won-lost-lifecycle-governance.md`, `src/app/api/crm/enquiries/[id]/mark-won/route.ts`
 - Read Before Modify: `0016-won-lost-lifecycle-governance.md`, `task-d4-won-lost-lifecycle-governance.md`, `task-j-crm-dashboard-kpi.md`, `task-k2-pipeline-reports.md`
 
+### Task D.5
+
+- Task ID: `Task D.5`
+- Task Name: `Lead / Enquiry Domain Separation`
+- Status: `In progress`
+- Objective: Replace the single-record lead/enquiry persistence model with a split-domain direction built around `crm_leads` plus lead-linked enquiries.
+- Related ADRs: `ADR-0018`
+- Foundations Created: `Lead Foundation`
+- Foundations Modified: `Lead / Enquiry Foundation` lineage, `Document Sequence Foundation`, `Master Data Foundation`
+- Major Deliverables: ADR-0018 target direction, `crm_leads`, `crm_enquiries.lead_id`, lead option seeds, lead/enquiry split foundation planning.
+- Key Files: `docs/adr/0018-lead-enquiry-domain-separation.md`, `docs/implementation/task-d5-lead-enquiry-domain-separation.md`, `src/db/schema.ts`, `src/db/seeds/foundation.seed.ts`
+- Read Before Modify: `0018-lead-enquiry-domain-separation.md`, `task-d5-lead-enquiry-domain-separation.md`, `0011-lead-enquiry-ownership-model.md`, `task-d31-leads-enquiries-navigation-separation.md`, `task-k2-pipeline-reports.md`
+
+### Task D.5.1
+
+- Task ID: `Task D.5.1`
+- Task Name: `Lead Foundation Stabilization & Output Completion`
+- Status: `Completed`
+- Objective: Audit the existing D.5 phase-1 foundation work, avoid duplicate schema creation, complete missing registrations, and document compatibility status before lead API work begins.
+- Related ADRs: `ADR-0018`
+- Foundations Created: `None`
+- Foundations Modified: `Lead Foundation`, `Master Data Foundation` registration, `Document Sequence Foundation` audit visibility
+- Major Deliverables: lead foundation audit, master-option category contract completion, foundation registry registration, task catalog registration, D.5.1 implementation note.
+- Key Files: `docs/implementation/task-d51-lead-foundation-stabilization.md`, `docs/standards/project-foundations.md`, `docs/standards/task-catalog.md`, `src/features/foundation/master-options/types.ts`
+- Read Before Modify: `task-d51-lead-foundation-stabilization.md`, `0018-lead-enquiry-domain-separation.md`, `task-d5-lead-enquiry-domain-separation.md`, `src/db/schema.ts`, `src/db/seeds/foundation.seed.ts`
+
 ### Task E
 
 - Task ID: `Task E`

drizzle/0021_worried_nebula.sql

@@ -0,0 +1,23 @@
+CREATE TABLE "crm_leads" (
+	"id" text PRIMARY KEY NOT NULL,
+	"organization_id" text NOT NULL,
+	"branch_id" text,
+	"code" text NOT NULL,
+	"customer_id" text,
+	"contact_id" text,
+	"project_name" text,
+	"project_location" text,
+	"awareness_id" text,
+	"status" text DEFAULT 'new_job' NOT NULL,
+	"followup_status" text,
+	"lost_reason" text,
+	"outcome" text DEFAULT 'open' NOT NULL,
+	"owner_marketing_user_id" text,
+	"created_by" text NOT NULL,
+	"created_at" timestamp with time zone DEFAULT now() NOT NULL,
+	"updated_at" timestamp with time zone DEFAULT now() NOT NULL,
+	"deleted_at" timestamp with time zone
+);
+--> statement-breakpoint
+ALTER TABLE "crm_enquiries" ADD COLUMN "lead_id" text;--> statement-breakpoint
+CREATE UNIQUE INDEX "crm_leads_org_code_idx" ON "crm_leads" USING btree ("organization_id","code");

drizzle/meta/_journal.json

@@ -148,6 +148,13 @@
       "when": 1782262800000,
       "tag": "0020_lead_foundation_schema",
       "breakpoints": true
+    },
+    {
+      "idx": 21,
+      "version": "7",
+      "when": 1782279100730,
+      "tag": "0021_worried_nebula",
+      "breakpoints": true
     }
   ]
 }

plans/task-d.5.1.md

@@ -1,4 +1,4 @@
-# Task D.5.1: Lead Foundation Schema & Compatibility Layer
+# Task D.5.1: Lead Foundation Stabilization & Output Completion
 
 ## Status
 
@@ -6,11 +6,18 @@ Planned
 
 ## Objective
 
-Create the foundational schema and compatibility layer required for Lead / Enquiry Domain Separation without breaking the existing CRM workflow.
+Complete and stabilize the Lead Foundation introduced by Task D.5 Phase 1.
 
-This task introduces the Lead domain but does not yet migrate business flows.
+This task must not recreate existing schema objects that already exist.
 
-The objective is to prepare the platform for D.5.2 and later phases while keeping all existing CRM features operational.
+The objective is to:
+
+* verify completed D.5 foundation work
+* document missing outputs
+* complete missing seeds
+* complete foundation registration
+* establish migration verification
+* prepare the platform for D.5.2 Lead API Foundation
 
 ---
 
@@ -21,111 +28,36 @@ Review:
 * AGENTS.md
 * Project Foundations Registry
 * Task Catalog
-* ADR-0011 Lead / Enquiry Ownership Model
-* ADR-0016 Won / Lost Lifecycle Governance
 * ADR-0018 Lead / Enquiry Domain Separation
-* Task D.3
+* Existing D.5 implementation summary
-* Task D.3.1
+* Current schema.ts
-* Task D.4
+* Current migration history
+* Current seed files
 
 ---
 
-# Required Skills
+# Required Historical Verification
 
-## Governance
+Before writing any code verify whether the following already exist:
 
-* AGENTS.md
+## Schema
-* Architecture Rules
-* Task Contract Template
-* Task Review Checklist
-
-## CRM
-
-* Lead / Enquiry Governance
-* Won / Lost Governance
-* Customer Ownership Governance
-
-## Backend
-
-* Drizzle ORM Pattern
-* Service Layer Pattern
-* Route Handler Pattern
-
-## Security
-
-* CRM Authorization Foundation
-
----
-
-# Scope D5.1.1 Create Lead Table
-
-Add:
 
 ```txt
 crm_leads
-```
+crm_enquiries.lead_id
-
-Initial fields:
-
-```txt
-id
-organizationId
-branchId
-
-code
-
-customerId
-contactId
-
-projectName
-projectLocation
-
-awarenessId
-
-status
-followupStatus
-lostReason
-
-outcome
-
-ownerMarketingUserId
-
-createdBy
-createdAt
-updatedAt
-deletedAt
 ```
 
 ---
 
-# Scope D5.1.2 Add Lead Reference to Enquiry
+## ADR
-
-Modify:
 
 ```txt
-crm_enquiries
+ADR-0018
 ```
 
-Add:
-
-```txt
-leadId nullable
-```
-
-Relationship:
-
-```txt
-crm_leads 1
-→ N crm_enquiries
-```
-
-No business flow changes yet.
-
 ---
 
-# Scope D5.1.3 Master Data Foundation
+## Seed Categories
-
-Create categories:
 
 ```txt
 crm_lead_awareness
@@ -134,141 +66,193 @@ crm_lead_followup_status
 crm_lead_lost_reason
 ```
 
-Seed:
-
-## crm_lead_awareness
-
-```txt
-Google Search / Website
-Social Media
-Email Marketing
-Visit
-Exhibition / Event
-Brochure / Catalog
-Word of Mouth
-Corporate Car
-Product Label
-Re-purchasing
-Introducing
-BCI
-```
-
-## crm_lead_status
-
-```txt
-new_job
-follow_up
-closed_lost
-cancel
-```
-
-## crm_lead_followup_status
-
-```txt
-budget_pending
-project_under_review
-spec_clarification
-site_survey_waiting
-drawing_followup
-quotation_preparation
-waiting_po
-```
-
-## crm_lead_lost_reason
-
-```txt
-price
-delivery_time
-sales_person
-service
-customer
-product_quality
-other
-transfer
-```
-
 ---
 
-# Scope D5.1.4 Document Sequence Foundation
+## Document Sequences
-
-Register:
 
 ```txt
 crm_lead
 crm_enquiry
 ```
 
-Document codes:
+---
+
+# Rule
+
+If an item already exists:
 
 ```txt
-Lead:
+DO NOT recreate it
-LD2606-001
+DO NOT generate duplicate migrations
-
+DO NOT rename existing objects
-Enquiry:
-EN2606-001
 ```
 
-Do not switch existing generation logic yet.
+Document the existing implementation instead.
-
-Only prepare configuration.
 
 ---
 
-# Scope D5.1.5 Compatibility Layer
+# Scope D5.1.1 Foundation Audit
 
-Keep existing:
+Create a verification report.
+
+Document:
 
 ```txt
-crm_enquiries.pipeline_stage
+What already exists
+What is missing
+What is partially implemented
 ```
 
-unchanged.
-
-No route behavior changes.
-
-No dashboard changes.
-
-No report changes.
-
-No migration of existing records.
-
-This task prepares schema only.
-
 ---
 
-# Scope D5.1.6 Authorization Preparation
+# Scope D5.1.2 Seed Completion
 
-Add placeholders for future Lead permissions:
+Verify all Lead master options exist.
+
+Missing options only:
 
 ```txt
-crm.lead.read
+create missing records
-crm.lead.create
-crm.lead.update
-crm.lead.assign
-crm.lead.delete
 ```
 
-No enforcement changes yet.
+Existing options:
+
+```txt
+leave unchanged
+```
+
+---
+
+# Scope D5.1.3 Document Sequence Audit
+
+Verify:
+
+```txt
+crm_lead
+crm_enquiry
+```
+
+exist in document sequence configuration.
+
+If missing:
+
+```txt
+add only missing configuration
+```
+
+---
+
+# Scope D5.1.4 Foundation Registration
+
+Update:
+
+```txt
+docs/standards/project-foundations.md
+```
+
+Add:
+
+```txt
+Lead Foundation
+```
+
+Purpose:
+
+Marketing-owned lead domain.
+
+---
+
+# Scope D5.1.5 Task Catalog Registration
+
+Update:
+
+```txt
+docs/standards/task-catalog.md
+```
+
+Register:
+
+```txt
+Task D.5
+Task D.5.1
+```
+
+---
+
+# Scope D5.1.6 Output Documentation
+
+Create:
+
+```txt
+docs/implementation/task-d51-lead-foundation-stabilization.md
+```
+
+Document:
+
+* schema created
+* seeds created
+* sequence configuration
+* compatibility status
+
+---
+
+# Scope D5.1.7 Compatibility Verification
+
+Verify:
+
+Current CRM still works:
+
+```txt
+Customer
+Contact
+Enquiry
+Quotation
+Dashboard
+Reports
+```
+
+No behavior changes allowed.
 
 ---
 
 # Deliverables
 
-## New
+## New Documentation
 
-* crm_leads schema
+* task-d51-lead-foundation-stabilization.md
-* lead master option categories
-* lead document sequence configuration
-* ADR-0018 references
 
 ---
 
-## Modified
+## Updated Documentation
 
-* crm_enquiries
+* project-foundations.md
-* foundation.seed.ts
+* task-catalog.md
-* document sequence seed
+
-* CRM permission registry
+---
+
+## Verification Output
+
+Produce:
+
+### Existing Foundation
+
+```txt
+crm_leads
+crm_enquiries.lead_id
+ADR-0018
+```
+
+### Missing Foundation
+
+```txt
+...
+```
+
+### Completed Foundation
+
+```txt
+...
+```
 
 ---
 
@@ -278,54 +262,13 @@ Do NOT:
 
 * create Lead APIs
 * create Lead UI
-* create Lead assignment workflow
+* create Lead Assignment workflow
-* migrate existing records
+* migrate records
-* change dashboard
+* update dashboard datasets
-* change reports
+* update report datasets
-* change security behavior
 * remove pipeline_stage
 
-Those belong to later D.5 phases.
+Those belong to D.5.2+
-
----
-
-# Verification
-
-Verify:
-
-```txt
-npm exec tsc --noEmit
-```
-
-Verify:
-
-```txt
-crm_leads table exists
-```
-
-Verify:
-
-```txt
-crm_enquiries.leadId exists
-```
-
-Verify:
-
-```txt
-lead master options seeded
-```
-
-Verify:
-
-```txt
-document sequence configuration exists
-```
-
-Verify:
-
-```txt
-existing CRM workflow remains unchanged
-```
 
 ---
 
@@ -333,17 +276,17 @@ existing CRM workflow remains unchanged
 
 Task is complete when:
 
-* Lead schema exists
+* no duplicate schema is created
-* Enquiry can reference Lead
+* existing D.5 work is audited
-* Lead master data exists
+* missing seed/configuration is completed
-* Lead document sequence exists
+* foundation registry is updated
-* Existing CRM remains fully operational
+* task catalog is updated
-* No business workflow regression occurs
+* D.5 foundation state is documented
 
 Result:
 
-Lead Foundation = Established
+Lead Foundation = Stabilized
 
-Compatibility Layer = Established
+Lead Foundation Outputs = Recorded
 
-Ready for D.5.2 Lead APIs & UI
+Ready for D.5.2 Lead API Foundation

src/features/foundation/master-options/types.ts

@@ -3,6 +3,10 @@ export const CRM_MASTER_OPTION_CATEGORIES = [
   'crm_customer_status',
   'crm_customer_type',
   'crm_enquiry_status',
+  'crm_lead_awareness',
+  'crm_lead_status',
+  'crm_lead_followup_status',
+  'crm_lead_lost_reason',
   'crm_quotation_status',
   'crm_quotation_type',
   'crm_discount_type',