13 KiB

Raw Blame History

Phase 4: Local AI, Automation & Ingestion Pipeline

Status: Pending Phase 3 Completion

Phase Goal: Implement the full local AI inference pipeline in chronogrid_ai, the IMAP ingestion engine, filesystem watchers, the Eisenhower score computation engine, the CalDAV embedded server, and the automation rules engine — all running entirely on-device with no cloud API calls.

Prerequisites: Phase 3 complete ✅
Estimated Complexity: Very High
Blocking: Phase 5 (final integration and release testing)

Task 4.1 — Eisenhower Priority Score Engine

Task 4.1: Implement the Eisenhower priority score computation engine in chronogrid_core:
- Create crates/chronogrid_core/src/scoring/mod.rs
- Implement EisenhowerWeights struct: w_importance: f64, w_urgency: f64, w_age: f64 (must sum to 1.0)
- Implement urgency_decay(due_date_ms: u64, now_ms: u64) -> f64:
  - Exponential decay function: f(Δt) = e^(-λ * Δt_hours) where λ = 0.05
  - Returns 1.0 when due_date is now, approaches 0.0 for distant future dates
  - Returns 1.5 (overdue boost) when due_date is in the past
- Implement compute_priority_score(task: &TaskSnapshot, weights: &EisenhowerWeights, now_ms: u64) -> f64
  - S = (w_i * p_user) + (w_u * urgency_decay(due_date, now)) + (w_a * age_factor)
  - age_factor = min(1.0, days_since_created / 30.0) — tasks age into importance over 30 days
- Implement assign_eisenhower_quadrant(score: f64, due_date: Option<u64>, now_ms: u64, threshold_high: f64) -> i64
  - Q1: score >= threshold AND due within 48 hours
  - Q2: score >= threshold AND due > 48 hours or no due date
  - Q3: score < threshold AND due within 48 hours
  - Q4: score < threshold AND due > 48 hours or no due date
- Implement Database::recompute_all_scores(weights: &EisenhowerWeights) -> Result<usize> — batch update all tasks
- Add scoring module to chronogrid_core/src/lib.rs public API
- Unit tests: decay function values, score computation, quadrant assignment, batch recompute
- Run cargo test --package chronogrid_core

Task 4.2 — Implement chronogrid_ai: NLP Metadata Extractor

Task 4.2: Implement the NLP metadata extraction engine in crates/chronogrid_ai/:
- Add dependencies: regex = "1", chrono = { version = "0.4", features = ["serde"] }, serde = { version = "1", features = ["derive"] }, serde_json = "1"
- Implement nlp::DateTimeExtractor:
  - Parse natural language date/time expressions using regex patterns:
    - Relative: "tomorrow", "next Monday", "in 3 days", "next week"
    - Absolute: "August 25th", "2026-08-25", "Thursday at 2pm", "by 3 PM Friday"
    - Duration: "for 2 hours", "30 minutes", "half an hour"
  - Returns ExtractedDateTime { due_date: Option<DateTime<Utc>>, duration_minutes: Option<i64> }
- Implement nlp::NamespaceClassifier:
  - Rule-based keyword classifier mapping text to namespace IDs
  - Configurable keyword lists per namespace (loaded from automation_rules table)
  - Returns ClassificationResult { namespace_id: String, confidence: f64 }
- Implement nlp::TaskNoteSplitter:
  - Detects action items: lines starting with - [ ], TODO:, ACTION:, imperative verbs
  - Separates actionable items (→ tasks) from reference text (→ notes)
  - Returns SplitResult { tasks: Vec<String>, note_content: String }
- Implement nlp::process_raw_input(text: &str, context: &ProcessingContext) -> ProcessedInput
  - Runs all three extractors in sequence
  - Returns structured ProcessedInput with extracted tasks, note content, dates, namespace
- Unit tests: date extraction for all pattern types, namespace classification, task/note splitting
- Run cargo test --package chronogrid_ai

Task 4.3 — Implement chronogrid_ai: Local LLM Integration (Ollama)

Task 4.3: Implement optional local LLM inference via Ollama HTTP socket in crates/chronogrid_ai/:
- Add dependencies: reqwest = { version = "0.12", features = ["json", "blocking"] }, serde_json = "1"
- Implement llm::OllamaClient:
  - OllamaClient::new(base_url: &str) -> Self — defaults to http://127.0.0.1:11434
  - OllamaClient::is_available() -> bool — health check ping
  - OllamaClient::complete(model: &str, prompt: &str) -> Result<String> — blocking inference call
- Implement llm::TaskEnrichmentPrompt:
  - Structured prompt template for task metadata extraction
  - Input: raw text, Output: JSON with { title, description, due_date, duration_minutes, namespace, priority_hint }
- Implement llm::enrich_task_with_llm(text: &str, client: &OllamaClient) -> Result<EnrichedTask>
  - Falls back to regex NLP extractor if Ollama is unavailable
  - Validates LLM JSON output against expected schema
- Feature flag: features = ["ollama"] — Ollama integration is opt-in
- Unit tests: prompt construction, JSON response parsing, fallback behavior when Ollama unavailable
- Run cargo test --package chronogrid_ai

Task 4.4 — Implement IMAP Ingestion Engine

Task 4.4: Implement the direct TLS IMAP polling engine in chronogrid_core:
- Add dependencies: async-imap = "0.9", async-native-tls = "0.5", tokio = { version = "1", features = ["full"] }, mail-parser = "0.9"
- Create crates/chronogrid_core/src/ingestion/imap.rs:
  - Implement ImapConfig struct: host, port, username, password_keychain_key, mailbox, poll_interval_secs, namespace_id
  - Implement ImapPoller:
    - ImapPoller::new(config: ImapConfig, db: Arc<Mutex<Database>>) -> Self
    - ImapPoller::poll_once() -> Result<Vec<IngestionResult>> — connects, fetches unseen messages, disconnects
    - ImapPoller::run_background(shutdown: CancellationToken) — Tokio task that polls on interval
  - Email processing pipeline:
    1. Fetch unseen messages from configured mailbox
    2. Extract: message_id, subject, body_text, attachments, date
    3. Convert body to Markdown (strip HTML, preserve structure)
    4. Save attachments as encrypted blobs in attachments table
    5. Run chronogrid_ai::nlp::process_raw_input on subject + body
    6. Create task CRDT document with extracted metadata
    7. Log to ingestion_logs table with source_type = 'imap'
    8. Mark message as seen / move to processed folder
- Unit tests: email parsing, Markdown conversion, attachment handling, ingestion log creation
- Integration test: mock IMAP server (using greenmail or in-process mock) → full pipeline → task created in DB
- Run cargo test --package chronogrid_core

Task 4.5 — Implement Filesystem Watcher

Task 4.5: Implement the local filesystem watcher in chronogrid_core:
- Add dependency: notify = "6" (cross-platform filesystem events)
- Create crates/chronogrid_core/src/ingestion/file_watcher.rs:
  - Implement WatchConfig struct: watch_path: PathBuf, namespace_id: String, file_extensions: Vec<String>
  - Implement FileWatcher:
    - FileWatcher::new(config: WatchConfig, db: Arc<Mutex<Database>>) -> Result<Self>
    - FileWatcher::start(shutdown: CancellationToken) — Tokio task watching for Create and Modify events
  - File processing pipeline:
    1. On file event: read file content
    2. Parse Markdown - [ ] checkboxes as task candidates
    3. Run chronogrid_ai::nlp::process_raw_input on file content
    4. Create/update note CRDT document with file content
    5. Extract tasks from - [ ] items and create task CRDT documents
    6. Log to ingestion_logs with source_type = 'file_watcher'
- Unit tests: file event handling, Markdown task extraction, note creation
- Run cargo test --package chronogrid_core

Task 4.6 — Implement Global Hotkey Jot Capture (Desktop)

Task 4.6: Implement the global hotkey quick-jot capture system:
- Add dependency to chronogrid_ffi: global-hotkey = "0.5" (cross-platform global hotkeys)
- Implement api::jot::register_global_hotkey(hotkey: String) -> anyhow::Result<()> — registers system-wide hotkey
- Implement api::jot::submit_jot(text: String) -> anyhow::Result<FfiTaskData> — processes raw jot text through AI pipeline and creates task
- Flutter side: implement JotOverlay — lightweight floating window that appears on hotkey press:
  - Single-line text input with submit button
  - Appears as a system overlay (Windows: flutter_acrylic for frosted glass effect)
  - On submit: calls submitJot() FFI function, shows confirmation toast, dismisses
- Android: implement QuickJotWidget — home screen widget using home_widget Flutter package
- Unit tests: hotkey registration, jot processing pipeline
- Run cargo test --package chronogrid_ffi

Task 4.7 — Implement CalDAV Embedded Server

Task 4.7: Implement the embedded CalDAV server in chronogrid_core:
- Add dependencies: axum = "0.7", tokio = { version = "1", features = ["full"] }, quick-xml = "0.36"
- Create crates/chronogrid_core/src/caldav/mod.rs:
  - Implement CalDavServer:
    - CalDavServer::new(db: Arc<Mutex<Database>>, bind_addr: SocketAddr) -> Self
    - CalDavServer::start(shutdown: CancellationToken) — starts Axum HTTP server on 127.0.0.1:4242
  - Implement CalDAV endpoints:
    - PROPFIND / — calendar discovery
    - PROPFIND /calendars/{namespace_id}/ — calendar collection properties
    - REPORT /calendars/{namespace_id}/ — calendar query (returns iCalendar events for time-blocked tasks)
    - PUT /calendars/{namespace_id}/{task_id}.ics — create/update time block from external calendar
    - DELETE /calendars/{namespace_id}/{task_id}.ics — remove time block
  - iCalendar (RFC 5545) serialization: convert time_blocks + tasks to VEVENT format
  - iCalendar parsing: convert incoming VEVENT to time_blocks entries
- Unit tests: iCalendar serialization, PROPFIND response XML, REPORT query filtering
- Integration test: CalDAV client (using vdirsyncer or curl) syncs with embedded server
- Run cargo test --package chronogrid_core

Task 4.8 — Implement Automation Rules Engine

Task 4.8: Implement the local automation rules engine in chronogrid_core:
- Create crates/chronogrid_core/src/automation/mod.rs:
  - Implement AutomationRule struct: trigger: Trigger, conditions: Vec<Condition>, actions: Vec<Action>
  - Implement Trigger enum: OnIngestion { source_type }, OnTaskCreated, OnTaskStatusChanged, OnSchedule { cron_expr }
  - Implement Condition enum: NamespaceMatches { id }, TitleContains { keyword }, PriorityAbove { threshold }, DueDateWithin { hours }
  - Implement Action enum: SetNamespace { id }, SetProject { id }, SetPriority { score }, AddTag { tag }, SendToCalendar, ArchiveTask
  - Implement RulesEngine:
    - RulesEngine::load_rules(db: &Database) -> Result<Self> — loads rules from automation_rules table
    - RulesEngine::evaluate(event: &AutomationEvent, db: &mut Database) -> Result<Vec<ActionResult>>
- Expose rules management via FFI: create_automation_rule, list_automation_rules, delete_automation_rule
- Unit tests: rule evaluation, condition matching, action execution, rule persistence
- Run cargo test --package chronogrid_core

Task 4.9 — Wire AI Pipeline into FFI & Flutter UI

Task 4.9: Expose the AI/automation pipeline through the FFI layer and integrate into Flutter:
- Add chronogrid_ai as a dependency in chronogrid_ffi/Cargo.toml
- Implement FFI functions in api/ingestion.rs:
  - process_jot_text(text: String) -> anyhow::Result<FfiTaskData> — full AI pipeline on raw text
  - configure_imap_account(config_json: String) -> anyhow::Result<()> — stores IMAP config
  - start_imap_polling() -> anyhow::Result<()> — starts background IMAP poller
  - stop_imap_polling() -> anyhow::Result<()>
  - configure_file_watcher(path: String, namespace_id: String) -> anyhow::Result<()>
  - get_ingestion_logs(limit: u32) -> anyhow::Result<Vec<FfiIngestionLog>>
- Re-run flutter_rust_bridge_codegen generate to update Dart bindings
- Implement IngestionBloc in Flutter with events: StartImapPolling, StopImapPolling, ProcessJot, LoadIngestionLogs
- Add ingestion status panel to SettingsScreen
- Unit tests: FFI ingestion functions, IngestionBloc state transitions
- Run cargo test --package chronogrid_ffi and flutter test

Phase 4 Completion Criteria

All 9 tasks above marked [x]
cargo test --workspace passes with zero failures
IMAP polling creates tasks in the database from test email messages
File watcher creates notes and tasks from Markdown files dropped in watch folder
Eisenhower scores recompute correctly for all test task scenarios
CalDAV server responds to PROPFIND and REPORT requests from a CalDAV client
Global hotkey jot capture creates a task within 2 seconds (acceptance criterion from user story)
scripts/verify_agent_work.ps1 exits with code 0
ADR created for AI/automation decisions (docs/ARCHITECTURE/ADR/0008-local-ai-automation-pipeline.md)
DECISION_LOG.md updated with Phase 4 decisions

13 KiB Raw Blame History