chronogrid/docs/ARCHITECTURE/06-master-development-plan.md

460 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# ChronoGrid P2P — Master Development Plan v2
**Document Version:** 2.0
**Created:** 2026-08-22
**Authority:** Architecture Review Board
**Supersedes:** OSAD Section 8 (Phase Roadmap)
**Status:** Active — Drives Automated Agent Execution
---
## 1. Purpose & Scope
This document is the **single authoritative execution plan** for ChronoGrid P2P development. It defines the complete phased roadmap, task sequencing, inter-phase dependencies, automated agent execution protocol, and quality gates for every phase from current state through v1.0.0 release.
This plan is designed for **automated agent execution** via the `.clinerules` Autonomous Task Execution Protocol. Each task in `docs/TASKS/` is written to be self-contained, verifiable, and executable by an AI agent without human intervention beyond phase boundary approvals.
---
## 2. Current State Assessment (2026-08-22)
### Completed Work
| Component | Status | Tests |
|---|---|---|
| `chronogrid_crypto` — Argon2id KDF + OS Keychain | ✅ Complete | ✅ Passing |
| `chronogrid_core` — SQLCipher DB + Schema | ✅ Complete | ✅ 11 tests |
| `chronogrid_core` — Automerge CRDT (Task + Note) | ✅ Complete | ✅ 45 tests |
| `chronogrid_ffi` — Flutter Rust Bridge v2 FFI | ✅ Complete | ✅ 25 tests |
| `docs/API_CONTRACTS/schema.sql` — Core schema | ✅ Complete | N/A |
| ADRs 0002–0005 | ✅ Complete | N/A |
### Pending Work (Phases 2–5)
| Component | Status | Blocking |
|---|---|---|
| `chronogrid_net` — libp2p P2P mesh | ❌ Stub | Phase 3, 4, 5 |
| Schema extension (6 new tables) | ❌ Missing | Phase 3, 4 |
| `chronogrid_daemon` — Headless binary | ❌ Not scaffolded | Phase 5 |
| `chronogrid_ai` — NLP + LLM pipeline | ❌ Stub | Phase 4 |
| Flutter UI — All views | ❌ Placeholder | Phase 5 |
| FRB codegen output | ❌ Not generated | Phase 3 |
| Integration tests | ❌ None | Phase 5 |
| Build scripts | ❌ Missing | Phase 3, 5 |
---
## 3. Phased Execution Roadmap
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ CHRONOGRID P2P — EXECUTION ROADMAP │
│ │
│ [Phase 1: Core Engine] ──────────────────────────────────── COMPLETE ✅ │
│ Rust, SQLCipher, Automerge CRDT, FFI Bridge │
│ │
│ [Phase 2: Networking & Schema] ──────────────────────── NEXT TO EXECUTE │
│ libp2p, QUIC, Noise XK, Schema Extension, Daemon Scaffold │
│ │ │
│ ▼ │
│ [Phase 3: Flutter UI Shell] ─────────────────────────── Pending Phase 2 │
│ BLoC Architecture, All 4 Views, FRB Codegen, Build Scripts │
│ │ │
│ ▼ │
│ [Phase 4: AI & Automation] ──────────────────────────── Pending Phase 3 │
│ NLP Engine, IMAP Ingestion, File Watchers, CalDAV, Rules Engine │
│ │ │
│ ▼ │
│ [Phase 5: Headless & Release] ───────────────────────── Pending Phase 4 │
│ Daemon Completion, Integration Tests, Packaging, v1.0.0 Release │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## 4. Phase Summary & Task Index
### Phase 1 — Native Storage & CRDT Foundation ✅ COMPLETE
**Task File:** `docs/TASKS/phase_1_storage_and_crdt.md`
| Task | Description | Status |
|---|---|---|
| 1.1 | Rust Cargo Workspace initialization | ✅ |
| 1.2 | `chronogrid_crypto` — Argon2id KDF + OS Keychain | ✅ |
| 1.3 | `chronogrid_core` — SQLCipher DB initializer | ✅ |
| 1.4 | `chronogrid_core` — Automerge CRDT integration | ✅ |
| 1.5 | `chronogrid_ffi` — Flutter Rust Bridge FFI layer | ✅ |
---
### Phase 2 — P2P Networking Mesh & Schema Extension
**Task File:** `docs/TASKS/phase_2_networking_and_schema.md`
**Status:** Ready for Agent Execution
**Estimated Duration:** 8–12 agent cycles
| Task | Description | Priority |
|---|---|---|
| 2.1 | Extend `schema.sql` with 6 new tables | CRITICAL |
| 2.2 | `chronogrid_net` — Node Identity & Swarm Key | HIGH |
| 2.3 | `chronogrid_net` — Transport & Discovery Layer | HIGH |
| 2.4 | `chronogrid_net` — CRDT Delta Sync Protocol | HIGH |
| 2.5 | `chronogrid_net` — P2P Swarm Manager | HIGH |
| 2.6 | Integrate `chronogrid_net` into `chronogrid_core` | HIGH |
| 2.7 | Scaffold `chronogrid_daemon` binary crate | MEDIUM |
| 2.8 | Expand verification pipeline | MEDIUM |
| 2.9 | Update FFI bridge spec & fix type inconsistency | MEDIUM |
**Phase 2 Gate:** `cargo test --workspace` passes; daemon binary compiles; schema has 13 tables.
---
### Phase 3 — Cross-Platform Flutter Presentation Layer
**Task File:** `docs/TASKS/phase_3_flutter_ui_shell.md`
**Status:** Pending Phase 2
**Estimated Duration:** 12–18 agent cycles
| Task | Description | Priority |
|---|---|---|
| 3.1 | FRB codegen + Flutter architecture setup | CRITICAL |
| 3.2 | Engine initialization & onboarding flow | HIGH |
| 3.3 | Task repository & BLoC layer | HIGH |
| 3.4 | Eisenhower Priority Matrix view | HIGH |
| 3.5 | Calendar Time-Blocking Grid view | HIGH |
| 3.6 | Kanban Board view | HIGH |
| 3.7 | Gantt Timeline view | MEDIUM |
| 3.8 | Notes & Knowledge Workspace view | HIGH |
| 3.9 | Navigation shell & settings | HIGH |
| 3.10 | Build scripts & platform configuration | HIGH |
**Phase 3 Gate:** `flutter test` passes; `flutter build windows` succeeds; all 4 primary views render live data.
---
### Phase 4 — Local AI, Automation & Ingestion Pipeline
**Task File:** `docs/TASKS/phase_4_ai_and_automation.md`
**Status:** Pending Phase 3
**Estimated Duration:** 10–14 agent cycles
| Task | Description | Priority |
|---|---|---|
| 4.1 | Eisenhower priority score engine | CRITICAL |
| 4.2 | `chronogrid_ai` — NLP metadata extractor | HIGH |
| 4.3 | `chronogrid_ai` — Ollama LLM integration | MEDIUM |
| 4.4 | IMAP ingestion engine | HIGH |
| 4.5 | Filesystem watcher | HIGH |
| 4.6 | Global hotkey jot capture | HIGH |
| 4.7 | CalDAV embedded server | HIGH |
| 4.8 | Automation rules engine | MEDIUM |
| 4.9 | Wire AI pipeline into FFI & Flutter | HIGH |
**Phase 4 Gate:** IMAP creates tasks within 2s; CalDAV responds to clients; jot capture works end-to-end.
---
### Phase 5 — Headless Node, Integration Testing & Release
**Task File:** `docs/TASKS/phase_5_headless_and_release.md`
**Status:** Pending Phase 4
**Estimated Duration:** 8–12 agent cycles
| Task | Description | Priority |
|---|---|---|
| 5.1 | Complete `chronogrid_daemon` headless binary | CRITICAL |
| 5.2 | Headless keyfile credential store | HIGH |
| 5.3 | Multi-device integration test suite | CRITICAL |
| 5.4 | Performance & stress testing | HIGH |
| 5.5 | Android platform integration | HIGH |
| 5.6 | Linux platform integration & AppImage | HIGH |
| 5.7 | Windows MSI installer | HIGH |
| 5.8 | Systemd service & deployment guide | MEDIUM |
| 5.9 | Final verification & release | CRITICAL |
**Phase 5 Gate:** All acceptance criteria from master user story verified; v1.0.0 tagged and pushed.
---
## 5. Automated Agent Execution Protocol
When an agent receives "Execute next task", it follows the `.clinerules` 7-step protocol:
### Step 1: State Discovery
```
1. Read .clinerules
2. Read docs/ARCHITECTURE/01-OSAD-system-architecture.md
3. Read docs/ARCHITECTURE/06-master-development-plan.md (this file)
4. Scan docs/TASKS/ — find FIRST file with an incomplete [ ] task
5. Read DECISION_LOG.md for latest context
6. State: "Executing Task X.Y — [Task Title]"
```
### Step 2: Feature Branch
```bash
git checkout main && git pull origin main
git checkout -b feature/<task-id>-<short-description>
# Example: git checkout -b feature/2.1-schema-extension
```
### Step 3: Implementation
- Read all referenced spec files before writing code
- Implement ONLY the target task — no scope creep
- Follow all `.clinerules` mandates (no cloud endpoints, correct crate boundaries, etc.)
### Step 4: Testing & Verification
```powershell
# Run the verification pipeline
powershell -File scripts/verify_agent_work.ps1
# Run crate-specific tests
cargo test --package <crate_name>
# Fix all errors before proceeding
```
### Step 5: Documentation Update
- Mark task `[x]` in the task file
- Update `DECISION_LOG.md` with decisions made
- Create ADR if architectural decisions were made
### Step 6: Commit & Merge
```bash
git add .
git commit -m "feat(<scope>): complete Task X.Y — <Task Title>"
git checkout main
git merge feature/<task-id>-<short-description> --no-ff -m "chore: merge feature/<task-id> into main"
```
### Step 7: Push & Cleanup
```bash
git push origin main
git branch -d feature/<task-id>-<short-description>
# Report: completed task, test status, next task
```
---
## 6. Quality Gates & Non-Negotiable Standards
### Code Quality
- All public Rust functions must have `///` doc comments
- All error types must use `thiserror` with meaningful variants
- No `unwrap()` in production code paths — use `?` operator
- Sensitive data types must implement `Zeroize + Drop`
- No `unsafe` blocks without explicit justification in a comment
### Testing Standards
- Every new module must have unit tests covering happy path and error cases
- Minimum test coverage targets:
- `chronogrid_crypto`: 90%+ (security-critical)
- `chronogrid_core`: 85%+
- `chronogrid_net`: 80%+
- `chronogrid_ffi`: 75%+
- `chronogrid_ai`: 75%+
- Integration tests required for all cross-crate data flows
### Architecture Mandates (from `.clinerules`)
- **ZERO** third-party cloud API calls anywhere in the codebase
- All network sync via `chronogrid_net` over libp2p/QUIC/Noise XK only
- Flutter presentation only — no business logic in Dart
- All schema changes must update `docs/API_CONTRACTS/schema.sql` first
- All architectural decisions must be logged in `DECISION_LOG.md` and have an ADR
### Verification Pipeline
Every task must pass `scripts/verify_agent_work.ps1` before commit. The script must:
1. `cargo check` all crates
2. `cargo test` all crates
3. `flutter analyze` the Flutter app
4. Validate schema table count
5. Exit 0 on success, non-zero on any failure
---
## 7. Technology Stack Reference
| Layer | Technology | Version | Purpose |
|---|---|---|---|
| Presentation | Flutter | 3.22.x+ | Cross-platform UI (Win/Linux/Android) |
| State Management | flutter_bloc | 8.1.x | BLoC pattern for reactive UI |
| FFI Bridge | flutter_rust_bridge | 2.7.0 | Zero-copy Dart↔Rust bindings |
| Core Language | Rust | 1.75+ (MSRV) | All business logic and storage |
| Local Storage | SQLCipher | 4.x | AES-256-CBC encrypted SQLite |
| CRDT Engine | Automerge | 0.5.x | Conflict-free replicated data types |
| Key Derivation | Argon2id | — | Password-based master key derivation |
| P2P Transport | libp2p | 0.54.x | QUIC + Noise XK + mDNS + Gossipsub |
| AI Inference | ONNX Runtime / Ollama | — | Local-only NLP and LLM inference |
| Async Runtime | Tokio | 1.x | Async I/O for networking and ingestion |
| CalDAV Server | Axum | 0.7.x | Embedded HTTP server for CalDAV |
| Logging | tracing | — | Structured logging for daemon |
---
## 8. File & Directory Structure (Target State)
```
chronogrid/
├── Cargo.toml # Workspace root
├── Cargo.lock
├── flutter_rust_bridge.yaml # FRB codegen config
├── DECISION_LOG.md # Architectural decision log
├── CHANGELOG.md # Release notes (created in Phase 5)
├── .clinerules # Agent execution protocol
│
├── apps/
│ ├── chronogrid_flutter/ # Flutter UI (Win/Linux/Android)
│ │ ├── pubspec.yaml
│ │ ├── lib/
│ │ │ ├── main.dart
│ │ │ ├── app.dart
│ │ │ └── src/
│ │ │ ├── rust/ # FRB generated bindings
│ │ │ ├── core/ # Engine service, DI, routing
│ │ │ ├── features/ # BLoC + UI per feature
│ │ │ └── shared/ # Widgets, theme, models
│ │ ├── windows/ # Win32 platform config
│ │ ├── linux/ # GTK3 platform config
│ │ └── android/ # NDK platform config
│ │
│ └── chronogrid_daemon/ # Headless peer node binary
│ ├── Cargo.toml
│ ├── config.example.toml
│ ├── src/
│ │ ├── main.rs
│ │ ├── config.rs
│ │ └── web_gui.rs
│ └── deploy/
│ ├── chronogrid.service # systemd unit
│ └── install.sh
│
├── crates/
│ ├── chronogrid_core/ # Storage + CRDT + Ingestion + CalDAV
│ │ └── src/
│ │ ├── lib.rs
│ │ ├── db.rs
│ │ ├── error.rs
│ │ ├── crdt/ # Automerge CRDT (task, note)
│ │ ├── scoring/ # Eisenhower score engine
│ │ ├── ingestion/ # IMAP + file watcher
│ │ ├── caldav/ # Embedded CalDAV server
│ │ ├── automation/ # Rules engine
│ │ └── sync/ # P2P sync bridge
│ │
│ ├── chronogrid_crypto/ # KDF + Keychain + Keyfile
│ │ └── src/
│ │ ├── lib.rs
│ │ ├── kdf.rs
│ │ ├── keychain.rs
│ │ └── keyfile.rs # Added in Phase 5
│ │
│ ├── chronogrid_net/ # libp2p P2P mesh
│ │ └── src/
│ │ ├── lib.rs
│ │ ├── identity.rs
│ │ ├── transport.rs
│ │ ├── discovery.rs
│ │ └── sync.rs
│ │
│ ├── chronogrid_ai/ # NLP + LLM inference
│ │ └── src/
│ │ ├── lib.rs
│ │ ├── nlp/ # Regex NLP extractors
│ │ └── llm/ # Ollama client (optional)
│ │
│ └── chronogrid_ffi/ # Flutter Rust Bridge FFI
│ └── src/
│ ├── lib.rs
│ ├── frb_generated.rs # FRB generated (after codegen)
│ └── api/
│ ├── mod.rs
│ ├── engine.rs
│ ├── database.rs
│ ├── crypto.rs
│ ├── task_crdt.rs
│ ├── note_crdt.rs
│ ├── ingestion.rs # Added in Phase 4
│ ├── state.rs
│ └── types.rs
│
├── docs/
│ ├── API_CONTRACTS/
│ │ ├── schema.sql # Single source of truth for DB schema
│ │ └── ffi_bridge_spec.json # FFI function catalog
│ ├── ARCHITECTURE/
│ │ ├── 01-OSAD-system-architecture.md
│ │ ├── 02-security-and-crypto.md
│ │ ├── 03-crdt-sync-protocol.md
│ │ ├── 04-master-user-story.md
│ │ ├── 05-architecture-review-assessment.md
│ │ ├── 06-master-development-plan.md ← THIS FILE
│ │ └── ADR/
│ │ ├── 0000-adr-template.md
│ │ ├── 0002-crypto-argon2id-keychain.md
│ │ ├── 0003-core-sqlcipher-db-init.md
│ │ ├── 0004-core-automerge-crdt-integration.md
│ │ ├── 0005-ffi-bridge-flutter-rust-bridge.md
│ │ ├── 0006-p2p-libp2p-networking.md # Created in Phase 2
│ │ ├── 0007-flutter-bloc-architecture.md # Created in Phase 3
│ │ └── 0008-local-ai-automation-pipeline.md # Created in Phase 4
│ ├── TASKS/
│ │ ├── phase_1_storage_and_crdt.md ✅ Complete
│ │ ├── phase_2_networking_and_schema.md
│ │ ├── phase_3_flutter_ui_shell.md
│ │ ├── phase_4_ai_and_automation.md
│ │ └── phase_5_headless_and_release.md
│ ├── UI_UX/
│ │ ├── 01-eisenhower-matrix.md
│ │ ├── 02-calendar-timeblocking.md
│ │ └── 03-kanban-gantt-views.md
│ ├── WORKFLOWS/
│ │ ├── 01-master-user-story.md
│ │ └── 02-ingestion-pipeline.md
│ └── DEPLOYMENT/
│ ├── 01-headless-daemon-setup.md # Created in Phase 2
│ ├── 02-linux-installation.md # Created in Phase 5
│ ├── 03-windows-installation.md # Created in Phase 5
│ └── 04-multi-device-pairing.md # Created in Phase 5
│
├── scripts/
│ ├── verify_agent_work.ps1 # CI verification pipeline
│ ├── build_desktop_libs.ps1 # Windows build (Phase 3)
│ ├── build_desktop_libs.sh # Linux build (Phase 3)
│ ├── build_android_libs.sh # Android NDK build (Phase 3)
│ ├── build_windows_installer.ps1 # MSI packaging (Phase 5)
│ └── package_linux.sh # AppImage/.deb packaging (Phase 5)
│
└── tests/
└── integration/ # Cross-crate integration tests (Phase 5)
├── sync_convergence.rs
├── crypto_pipeline.rs
├── ingestion_pipeline.rs
└── caldav_sync.rs
```
---
## 9. Risk Register
| Risk | Probability | Impact | Mitigation |
|---|---|---|---|
| libp2p NAT traversal fails on restrictive networks | MEDIUM | HIGH | Implement Circuit Relay v2 as fallback; document port forwarding requirements |
| Automerge merge conflicts produce unexpected results | LOW | HIGH | Extensive fork/merge tests; HLC ordering as tiebreaker |
| SQLCipher bundled build fails on Android NDK | MEDIUM | HIGH | Test early in Phase 3; have fallback to system SQLite with manual encryption |
| Ollama not available on target device | HIGH | LOW | NLP regex extractor is the primary path; Ollama is opt-in enhancement |
| Flutter Windows build fails due to missing MSVC | LOW | MEDIUM | Document prerequisites clearly; verify in CI |
| IMAP TLS certificate validation issues | MEDIUM | MEDIUM | Support both strict and permissive TLS modes; document configuration |
| CalDAV RFC compliance gaps | MEDIUM | MEDIUM | Test with multiple CalDAV clients (Thunderbird, DAVx⁵, Outlook) |
| Android background sync killed by OS | HIGH | MEDIUM | Use WorkManager with KEEP policy; document battery optimization exemption |
---
## 10. Acceptance Criteria Traceability
All acceptance criteria from `docs/ARCHITECTURE/04-master-user-story.md` are traced to implementing tasks:
| Acceptance Criterion | Implementing Tasks | Verified In |
|---|---|---|
| IMAP polling creates tasks within 2 seconds | 4.4, 4.9 | Phase 4 gate |
| Database unreadable without master key | 1.2, 1.3 (complete) | Phase 1 gate ✅ |
| Offline edits converge via CRDT on reconnection | 2.4, 2.5, 2.6 | Phase 2 gate |
| All 4 UI views render from same CRDT state | 3.3–3.8 | Phase 3 gate |
| Headless daemon builds for Linux/systemd | 2.7, 5.1 | Phase 5 gate |
| Global hotkey jot capture < 2 seconds | 4.6 | Phase 4 gate |
| CalDAV sync with external calendar tools | 4.7 | Phase 4 gate |
| P2P sync via QUIC/Noise XK | 2.2–2.5 | Phase 2 gate |
| Android KeyStore-backed key storage | 5.5 | Phase 5 gate |