215 lines
13 KiB
Markdown
215 lines
13 KiB
Markdown
# Phase 4: Local AI, Automation & Ingestion Pipeline
|
|
|
|
## Status: Pending Phase 3 Completion
|
|
|
|
**Phase Goal:** Implement the full local AI inference pipeline in `chronogrid_ai`, the IMAP ingestion engine, filesystem watchers, the Eisenhower score computation engine, the CalDAV embedded server, and the automation rules engine — all running entirely on-device with no cloud API calls.
|
|
|
|
**Prerequisites:** Phase 3 complete ✅
|
|
**Estimated Complexity:** Very High
|
|
**Blocking:** Phase 5 (final integration and release testing)
|
|
|
|
---
|
|
|
|
## Task 4.1 — Eisenhower Priority Score Engine
|
|
|
|
- [ ] **Task 4.1**: Implement the Eisenhower priority score computation engine in `chronogrid_core`:
|
|
- Create `crates/chronogrid_core/src/scoring/mod.rs`
|
|
- Implement `EisenhowerWeights` struct: `w_importance: f64`, `w_urgency: f64`, `w_age: f64` (must sum to 1.0)
|
|
- Implement `urgency_decay(due_date_ms: u64, now_ms: u64) -> f64`:
|
|
- Exponential decay function: `f(Δt) = e^(-λ * Δt_hours)` where λ = 0.05
|
|
- Returns 1.0 when due_date is now, approaches 0.0 for distant future dates
|
|
- Returns 1.5 (overdue boost) when due_date is in the past
|
|
- Implement `compute_priority_score(task: &TaskSnapshot, weights: &EisenhowerWeights, now_ms: u64) -> f64`
|
|
- `S = (w_i * p_user) + (w_u * urgency_decay(due_date, now)) + (w_a * age_factor)`
|
|
- `age_factor = min(1.0, days_since_created / 30.0)` — tasks age into importance over 30 days
|
|
- Implement `assign_eisenhower_quadrant(score: f64, due_date: Option<u64>, now_ms: u64, threshold_high: f64) -> i64`
|
|
- Q1: score >= threshold AND due within 48 hours
|
|
- Q2: score >= threshold AND due > 48 hours or no due date
|
|
- Q3: score < threshold AND due within 48 hours
|
|
- Q4: score < threshold AND due > 48 hours or no due date
|
|
- Implement `Database::recompute_all_scores(weights: &EisenhowerWeights) -> Result<usize>` — batch update all tasks
|
|
- Add `scoring` module to `chronogrid_core/src/lib.rs` public API
|
|
- Unit tests: decay function values, score computation, quadrant assignment, batch recompute
|
|
- Run `cargo test --package chronogrid_core`
|
|
|
|
---
|
|
|
|
## Task 4.2 — Implement chronogrid_ai: NLP Metadata Extractor
|
|
|
|
- [ ] **Task 4.2**: Implement the NLP metadata extraction engine in `crates/chronogrid_ai/`:
|
|
- Add dependencies: `regex = "1"`, `chrono = { version = "0.4", features = ["serde"] }`, `serde = { version = "1", features = ["derive"] }`, `serde_json = "1"`
|
|
- Implement `nlp::DateTimeExtractor`:
|
|
- Parse natural language date/time expressions using regex patterns:
|
|
- Relative: "tomorrow", "next Monday", "in 3 days", "next week"
|
|
- Absolute: "August 25th", "2026-08-25", "Thursday at 2pm", "by 3 PM Friday"
|
|
- Duration: "for 2 hours", "30 minutes", "half an hour"
|
|
- Returns `ExtractedDateTime { due_date: Option<DateTime<Utc>>, duration_minutes: Option<i64> }`
|
|
- Implement `nlp::NamespaceClassifier`:
|
|
- Rule-based keyword classifier mapping text to namespace IDs
|
|
- Configurable keyword lists per namespace (loaded from `automation_rules` table)
|
|
- Returns `ClassificationResult { namespace_id: String, confidence: f64 }`
|
|
- Implement `nlp::TaskNoteSplitter`:
|
|
- Detects action items: lines starting with `- [ ]`, `TODO:`, `ACTION:`, imperative verbs
|
|
- Separates actionable items (→ tasks) from reference text (→ notes)
|
|
- Returns `SplitResult { tasks: Vec<String>, note_content: String }`
|
|
- Implement `nlp::process_raw_input(text: &str, context: &ProcessingContext) -> ProcessedInput`
|
|
- Runs all three extractors in sequence
|
|
- Returns structured `ProcessedInput` with extracted tasks, note content, dates, namespace
|
|
- Unit tests: date extraction for all pattern types, namespace classification, task/note splitting
|
|
- Run `cargo test --package chronogrid_ai`
|
|
|
|
---
|
|
|
|
## Task 4.3 — Implement chronogrid_ai: Local LLM Integration (Ollama)
|
|
|
|
- [ ] **Task 4.3**: Implement optional local LLM inference via Ollama HTTP socket in `crates/chronogrid_ai/`:
|
|
- Add dependencies: `reqwest = { version = "0.12", features = ["json", "blocking"] }`, `serde_json = "1"`
|
|
- Implement `llm::OllamaClient`:
|
|
- `OllamaClient::new(base_url: &str) -> Self` — defaults to `http://127.0.0.1:11434`
|
|
- `OllamaClient::is_available() -> bool` — health check ping
|
|
- `OllamaClient::complete(model: &str, prompt: &str) -> Result<String>` — blocking inference call
|
|
- Implement `llm::TaskEnrichmentPrompt`:
|
|
- Structured prompt template for task metadata extraction
|
|
- Input: raw text, Output: JSON with `{ title, description, due_date, duration_minutes, namespace, priority_hint }`
|
|
- Implement `llm::enrich_task_with_llm(text: &str, client: &OllamaClient) -> Result<EnrichedTask>`
|
|
- Falls back to regex NLP extractor if Ollama is unavailable
|
|
- Validates LLM JSON output against expected schema
|
|
- Feature flag: `features = ["ollama"]` — Ollama integration is opt-in
|
|
- Unit tests: prompt construction, JSON response parsing, fallback behavior when Ollama unavailable
|
|
- Run `cargo test --package chronogrid_ai`
|
|
|
|
---
|
|
|
|
## Task 4.4 — Implement IMAP Ingestion Engine
|
|
|
|
- [ ] **Task 4.4**: Implement the direct TLS IMAP polling engine in `chronogrid_core`:
|
|
- Add dependencies: `async-imap = "0.9"`, `async-native-tls = "0.5"`, `tokio = { version = "1", features = ["full"] }`, `mail-parser = "0.9"`
|
|
- Create `crates/chronogrid_core/src/ingestion/imap.rs`:
|
|
- Implement `ImapConfig` struct: `host`, `port`, `username`, `password_keychain_key`, `mailbox`, `poll_interval_secs`, `namespace_id`
|
|
- Implement `ImapPoller`:
|
|
- `ImapPoller::new(config: ImapConfig, db: Arc<Mutex<Database>>) -> Self`
|
|
- `ImapPoller::poll_once() -> Result<Vec<IngestionResult>>` — connects, fetches unseen messages, disconnects
|
|
- `ImapPoller::run_background(shutdown: CancellationToken)` — Tokio task that polls on interval
|
|
- Email processing pipeline:
|
|
1. Fetch unseen messages from configured mailbox
|
|
2. Extract: `message_id`, `subject`, `body_text`, `attachments`, `date`
|
|
3. Convert body to Markdown (strip HTML, preserve structure)
|
|
4. Save attachments as encrypted blobs in `attachments` table
|
|
5. Run `chronogrid_ai::nlp::process_raw_input` on subject + body
|
|
6. Create task CRDT document with extracted metadata
|
|
7. Log to `ingestion_logs` table with `source_type = 'imap'`
|
|
8. Mark message as seen / move to processed folder
|
|
- Unit tests: email parsing, Markdown conversion, attachment handling, ingestion log creation
|
|
- Integration test: mock IMAP server (using `greenmail` or in-process mock) → full pipeline → task created in DB
|
|
- Run `cargo test --package chronogrid_core`
|
|
|
|
---
|
|
|
|
## Task 4.5 — Implement Filesystem Watcher
|
|
|
|
- [ ] **Task 4.5**: Implement the local filesystem watcher in `chronogrid_core`:
|
|
- Add dependency: `notify = "6"` (cross-platform filesystem events)
|
|
- Create `crates/chronogrid_core/src/ingestion/file_watcher.rs`:
|
|
- Implement `WatchConfig` struct: `watch_path: PathBuf`, `namespace_id: String`, `file_extensions: Vec<String>`
|
|
- Implement `FileWatcher`:
|
|
- `FileWatcher::new(config: WatchConfig, db: Arc<Mutex<Database>>) -> Result<Self>`
|
|
- `FileWatcher::start(shutdown: CancellationToken)` — Tokio task watching for `Create` and `Modify` events
|
|
- File processing pipeline:
|
|
1. On file event: read file content
|
|
2. Parse Markdown `- [ ]` checkboxes as task candidates
|
|
3. Run `chronogrid_ai::nlp::process_raw_input` on file content
|
|
4. Create/update note CRDT document with file content
|
|
5. Extract tasks from `- [ ]` items and create task CRDT documents
|
|
6. Log to `ingestion_logs` with `source_type = 'file_watcher'`
|
|
- Unit tests: file event handling, Markdown task extraction, note creation
|
|
- Run `cargo test --package chronogrid_core`
|
|
|
|
---
|
|
|
|
## Task 4.6 — Implement Global Hotkey Jot Capture (Desktop)
|
|
|
|
- [ ] **Task 4.6**: Implement the global hotkey quick-jot capture system:
|
|
- Add dependency to `chronogrid_ffi`: `global-hotkey = "0.5"` (cross-platform global hotkeys)
|
|
- Implement `api::jot::register_global_hotkey(hotkey: String) -> anyhow::Result<()>` — registers system-wide hotkey
|
|
- Implement `api::jot::submit_jot(text: String) -> anyhow::Result<FfiTaskData>` — processes raw jot text through AI pipeline and creates task
|
|
- Flutter side: implement `JotOverlay` — lightweight floating window that appears on hotkey press:
|
|
- Single-line text input with submit button
|
|
- Appears as a system overlay (Windows: `flutter_acrylic` for frosted glass effect)
|
|
- On submit: calls `submitJot()` FFI function, shows confirmation toast, dismisses
|
|
- Android: implement `QuickJotWidget` — home screen widget using `home_widget` Flutter package
|
|
- Unit tests: hotkey registration, jot processing pipeline
|
|
- Run `cargo test --package chronogrid_ffi`
|
|
|
|
---
|
|
|
|
## Task 4.7 — Implement CalDAV Embedded Server
|
|
|
|
- [ ] **Task 4.7**: Implement the embedded CalDAV server in `chronogrid_core`:
|
|
- Add dependencies: `axum = "0.7"`, `tokio = { version = "1", features = ["full"] }`, `quick-xml = "0.36"`
|
|
- Create `crates/chronogrid_core/src/caldav/mod.rs`:
|
|
- Implement `CalDavServer`:
|
|
- `CalDavServer::new(db: Arc<Mutex<Database>>, bind_addr: SocketAddr) -> Self`
|
|
- `CalDavServer::start(shutdown: CancellationToken)` — starts Axum HTTP server on `127.0.0.1:4242`
|
|
- Implement CalDAV endpoints:
|
|
- `PROPFIND /` — calendar discovery
|
|
- `PROPFIND /calendars/{namespace_id}/` — calendar collection properties
|
|
- `REPORT /calendars/{namespace_id}/` — calendar query (returns iCalendar events for time-blocked tasks)
|
|
- `PUT /calendars/{namespace_id}/{task_id}.ics` — create/update time block from external calendar
|
|
- `DELETE /calendars/{namespace_id}/{task_id}.ics` — remove time block
|
|
- iCalendar (RFC 5545) serialization: convert `time_blocks` + `tasks` to `VEVENT` format
|
|
- iCalendar parsing: convert incoming `VEVENT` to `time_blocks` entries
|
|
- Unit tests: iCalendar serialization, PROPFIND response XML, REPORT query filtering
|
|
- Integration test: CalDAV client (using `vdirsyncer` or `curl`) syncs with embedded server
|
|
- Run `cargo test --package chronogrid_core`
|
|
|
|
---
|
|
|
|
## Task 4.8 — Implement Automation Rules Engine
|
|
|
|
- [ ] **Task 4.8**: Implement the local automation rules engine in `chronogrid_core`:
|
|
- Create `crates/chronogrid_core/src/automation/mod.rs`:
|
|
- Implement `AutomationRule` struct: `trigger: Trigger`, `conditions: Vec<Condition>`, `actions: Vec<Action>`
|
|
- Implement `Trigger` enum: `OnIngestion { source_type }`, `OnTaskCreated`, `OnTaskStatusChanged`, `OnSchedule { cron_expr }`
|
|
- Implement `Condition` enum: `NamespaceMatches { id }`, `TitleContains { keyword }`, `PriorityAbove { threshold }`, `DueDateWithin { hours }`
|
|
- Implement `Action` enum: `SetNamespace { id }`, `SetProject { id }`, `SetPriority { score }`, `AddTag { tag }`, `SendToCalendar`, `ArchiveTask`
|
|
- Implement `RulesEngine`:
|
|
- `RulesEngine::load_rules(db: &Database) -> Result<Self>` — loads rules from `automation_rules` table
|
|
- `RulesEngine::evaluate(event: &AutomationEvent, db: &mut Database) -> Result<Vec<ActionResult>>`
|
|
- Expose rules management via FFI: `create_automation_rule`, `list_automation_rules`, `delete_automation_rule`
|
|
- Unit tests: rule evaluation, condition matching, action execution, rule persistence
|
|
- Run `cargo test --package chronogrid_core`
|
|
|
|
---
|
|
|
|
## Task 4.9 — Wire AI Pipeline into FFI & Flutter UI
|
|
|
|
- [ ] **Task 4.9**: Expose the AI/automation pipeline through the FFI layer and integrate into Flutter:
|
|
- Add `chronogrid_ai` as a dependency in `chronogrid_ffi/Cargo.toml`
|
|
- Implement FFI functions in `api/ingestion.rs`:
|
|
- `process_jot_text(text: String) -> anyhow::Result<FfiTaskData>` — full AI pipeline on raw text
|
|
- `configure_imap_account(config_json: String) -> anyhow::Result<()>` — stores IMAP config
|
|
- `start_imap_polling() -> anyhow::Result<()>` — starts background IMAP poller
|
|
- `stop_imap_polling() -> anyhow::Result<()>`
|
|
- `configure_file_watcher(path: String, namespace_id: String) -> anyhow::Result<()>`
|
|
- `get_ingestion_logs(limit: u32) -> anyhow::Result<Vec<FfiIngestionLog>>`
|
|
- Re-run `flutter_rust_bridge_codegen generate` to update Dart bindings
|
|
- Implement `IngestionBloc` in Flutter with events: `StartImapPolling`, `StopImapPolling`, `ProcessJot`, `LoadIngestionLogs`
|
|
- Add ingestion status panel to `SettingsScreen`
|
|
- Unit tests: FFI ingestion functions, `IngestionBloc` state transitions
|
|
- Run `cargo test --package chronogrid_ffi` and `flutter test`
|
|
|
|
---
|
|
|
|
## Phase 4 Completion Criteria
|
|
|
|
- [ ] All 9 tasks above marked `[x]`
|
|
- [ ] `cargo test --workspace` passes with zero failures
|
|
- [ ] IMAP polling creates tasks in the database from test email messages
|
|
- [ ] File watcher creates notes and tasks from Markdown files dropped in watch folder
|
|
- [ ] Eisenhower scores recompute correctly for all test task scenarios
|
|
- [ ] CalDAV server responds to `PROPFIND` and `REPORT` requests from a CalDAV client
|
|
- [ ] Global hotkey jot capture creates a task within 2 seconds (acceptance criterion from user story)
|
|
- [ ] `scripts/verify_agent_work.ps1` exits with code 0
|
|
- [ ] ADR created for AI/automation decisions (`docs/ARCHITECTURE/ADR/0008-local-ai-automation-pipeline.md`)
|
|
- [ ] `DECISION_LOG.md` updated with Phase 4 decisions
|