Phase 2: P2P Networking Mesh & Schema Extension

Status: Ready for Agent Execution

Phase Goal: Implement the libp2p P2P networking stack in chronogrid_net, extend the database schema with all missing entities, scaffold the headless daemon binary, and harden the verification pipeline.

Prerequisites: Phase 1 complete ✅
Estimated Complexity: High
Blocking: Phase 3 (Flutter UI requires sync events), Phase 4 (AI requires ingestion schema)

Task 2.1 — Extend Schema with Missing Entities

Task 2.1: Extend docs/API_CONTRACTS/schema.sql with the following new tables:
- time_blocks — Calendar time-blocking entries linked to tasks
- attachments — Binary/file attachments linked to notes or ingestion logs
- calendar_feeds — CalDAV external feed configuration per namespace
- node_peers — P2P peer registry (NodeId, public key, last seen, swarm membership)
- automation_rules — Local trigger/action rule definitions
- ingestion_configs — IMAP account and file-watcher configuration per namespace
- Add crdt_docs entries for time_block entity type
- Add indexes for all new foreign keys and query patterns
- Verify db.rs EXPECTED_TABLES list is updated to include all new tables
- All new tables must include hlc_timestamp TEXT NOT NULL for CRDT compatibility
- Run cargo test --package chronogrid_core — all existing tests must still pass

Task 2.2 — Implement chronogrid_net: Node Identity & Swarm Key

Task 2.2: Implement node identity and swarm key management in crates/chronogrid_net/:
- Add dependencies: libp2p = { version = "0.54", features = ["noise", "quic", "mdns", "gossipsub", "kad", "identify", "relay"] }, tokio = { version = "1", features = ["full"] }, ed25519-dalek = "2", rand = "0.8"
- Implement NodeIdentity struct: Ed25519 keypair generation, serialization, and NodeId (public key hash)
- Implement SwarmKey struct: 256-bit random key generation, serialization, and validation
- Implement identity::generate_node_identity() -> NodeIdentity
- Implement identity::load_or_create_identity(path: &Path) -> Result<NodeIdentity> — persists keypair to encrypted file
- Unit tests: keypair generation, serialization round-trip, NodeId derivation, SwarmKey validation
- Run cargo test --package chronogrid_net

Task 2.3 — Implement chronogrid_net: Transport & Discovery Layer

Task 2.3: Implement the libp2p transport stack and peer discovery in crates/chronogrid_net/:
- Implement transport::build_transport(identity: &NodeIdentity, swarm_key: &SwarmKey) -> Result<Boxed<(PeerId, StreamMuxerBox)>>
  - QUIC over UDP as primary transport
  - Noise XK handshake (Noise_XK_25519_ChaChaPoly_BLAKE2s) for authentication
  - Swarm Key pre-shared authentication layer
- Implement discovery::MdnsDiscovery — zero-config LAN peer discovery
- Implement discovery::KademliaDht — WAN peer discovery via Kademlia DHT
- Implement PeerEvent enum: PeerDiscovered { peer_id, addrs }, PeerExpired { peer_id }, PeerConnected { peer_id }, PeerDisconnected { peer_id }
- Unit tests: transport construction, mDNS event handling, DHT routing table operations
- Run cargo test --package chronogrid_net

Task 2.4 — Implement chronogrid_net: CRDT Delta Sync Protocol

Task 2.4: Implement the CRDT delta exchange protocol over libp2p in crates/chronogrid_net/:
- Implement sync::VectorClock — tracks per-entity HLC head state for delta computation
- Implement sync::DeltaPacket — serializable struct containing compressed CRDT changesets
- Implement sync::SyncProtocol — custom libp2p NetworkBehaviour:
  - On peer connect: exchange vector clocks via request/response protocol
  - Compute missing deltas by comparing remote vs. local vector clock
  - Compress delta bytes (LZ4 or zstd), encrypt with ChaCha20-Poly1305, stream over QUIC channel
  - On receive: decrypt, decompress, and return Vec<DeltaPacket> for core to merge
- Implement sync::GossipsubBroadcast — real-time state change notifications via Gossipsub
- Implement NetEvent enum: DeltasReceived { from_peer, deltas }, SyncComplete { peer_id }, SyncError { peer_id, reason }
- Unit tests: vector clock comparison, delta packet serialization, gossipsub message routing
- Run cargo test --package chronogrid_net

Task 2.5 — Implement chronogrid_net: P2P Swarm Manager

Task 2.5: Implement the top-level SwarmManager in crates/chronogrid_net/:
- Implement SwarmManager struct: owns the libp2p Swarm, runs the event loop in a dedicated Tokio task
- Public API:
  - SwarmManager::start(identity, swarm_key, listen_addrs) -> Result<SwarmHandle>
  - SwarmHandle::trigger_sync() -> Result<()> — initiates delta sync with all connected peers
  - SwarmHandle::broadcast_event(event: GossipEvent) -> Result<()>
  - SwarmHandle::subscribe_events() -> tokio::sync::broadcast::Receiver<NetEvent>
  - SwarmHandle::shutdown() -> Result<()>
- Integrate MdnsDiscovery, KademliaDht, SyncProtocol, and GossipsubBroadcast into a single NetworkBehaviour composite
- Implement NAT hole-punching via libp2p Circuit Relay v2 (relay client mode)
- Unit tests: swarm startup/shutdown, peer connection lifecycle, event subscription
- Integration test: two in-process swarms exchange a CRDT delta and verify convergence
- Run cargo test --package chronogrid_net

Task 2.6 — Integrate chronogrid_net into chronogrid_core

Task 2.6: Wire the P2P sync layer into chronogrid_core:
- Add chronogrid_net as an optional dependency in chronogrid_core/Cargo.toml (feature: "p2p")
- Implement core::sync::SyncEngine — bridges SwarmHandle events to Database CRDT merges:
  - On NetEvent::DeltasReceived: deserialize Automerge changesets, merge into local crdt_docs, update SQL rows
  - On local CRDT mutation: compute delta vs. last-synced state, broadcast via SwarmHandle
- Implement core::sync::SyncEngine::merge_remote_deltas(deltas: Vec<DeltaPacket>) -> Result<Vec<String>> — returns list of affected entity IDs
- Unit tests: delta merge into in-memory database, conflict resolution with concurrent edits
- Run cargo test --package chronogrid_core --features p2p

Task 2.7 — Scaffold chronogrid_daemon Binary

Task 2.7: Create the apps/chronogrid_daemon/ headless binary crate:
- Create apps/chronogrid_daemon/Cargo.toml with [[bin]] target chronogrid-daemon
- Add apps/chronogrid_daemon to workspace Cargo.toml members list
- Dependencies: chronogrid_core, chronogrid_crypto, chronogrid_net, tokio, clap, serde, toml, tracing, tracing-subscriber
- Create apps/chronogrid_daemon/src/main.rs:
  - CLI argument parsing via clap: --config <path>, --db <path>, --log-level <level>
  - Config file loading from TOML (/etc/chronogrid/config.toml or --config path)
  - Initialize chronogrid_crypto key derivation and keychain retrieval
  - Initialize chronogrid_core database engine
  - Start chronogrid_net SwarmManager
  - Structured logging via tracing
  - Graceful shutdown on SIGTERM/SIGINT
- Create apps/chronogrid_daemon/config.example.toml — documented example configuration
- Create docs/DEPLOYMENT/01-headless-daemon-setup.md — systemd service setup guide
- Run cargo build --package chronogrid_daemon

Task 2.8 — Expand Verification Pipeline

Task 2.8: Expand scripts/verify_agent_work.ps1 to cover all crates and run full test suite:
- Add cargo test --package chronogrid_crypto step
- Add cargo test --package chronogrid_core step
- Add cargo test --package chronogrid_ffi step
- Add cargo test --package chronogrid_net step
- Add cargo check --package chronogrid_daemon step
- Add schema table count validation (query SQLite in-memory and count tables)
- Add check that docs/API_CONTRACTS/schema.sql contains all required table names
- Improve error reporting with per-step timing and failure context
- Run the expanded script and confirm all checks pass

Task 2.9 — Update FFI Bridge Spec & Fix Type Inconsistency

Task 2.9: Update API contracts and fix the due_date type mismatch:
- Fix crates/chronogrid_ffi/src/api/types.rs: change due_date: Option<i64> to Option<u64> and completed_at: Option<i64> to Option<u64>
- Update docs/API_CONTRACTS/ffi_bridge_spec.json to document all 38 FFI functions across the 5 API modules (engine, database, crypto, task_crdt, note_crdt)
- Add P2P sync FFI functions to the spec: init_p2p_swarm, trigger_sync, subscribe_sync_events, get_connected_peers
- Run cargo test --package chronogrid_ffi — all tests must pass

Phase 2 Completion Criteria

All 9 tasks above marked [x]
cargo test --workspace passes with zero failures
scripts/verify_agent_work.ps1 exits with code 0
chronogrid_daemon binary compiles for x86_64-pc-windows-msvc and x86_64-unknown-linux-gnu
docs/API_CONTRACTS/schema.sql contains all 13 tables (7 original + 6 new)
ADR created for P2P networking decisions (docs/ARCHITECTURE/ADR/0006-p2p-libp2p-networking.md)
DECISION_LOG.md updated with Phase 2 decisions

9.2 KiB Raw Blame History