155 lines
7.9 KiB
Markdown
155 lines
7.9 KiB
Markdown
# ADR-0010: chronogrid_net — CRDT Delta Sync Protocol (Task 2.4)
|
||
|
||
**Status:** Accepted
|
||
**Date:** 2026-08-22
|
||
**Task:** 2.4 — Implement chronogrid_net: CRDT Delta Sync Protocol
|
||
**Author:** Autonomous Agent
|
||
|
||
---
|
||
|
||
## Context
|
||
|
||
Task 2.4 required implementing the CRDT delta exchange protocol over libp2p in
|
||
`crates/chronogrid_net/`, building on the transport and discovery layers from
|
||
Tasks 2.2 and 2.3.
|
||
|
||
The implementation must satisfy:
|
||
- `VectorClock` — per-entity HLC head-state tracking for delta computation.
|
||
- `DeltaPacket` — serializable, compressed, encrypted CRDT changeset container.
|
||
- `SyncProtocol` — libp2p `NetworkBehaviour` for request/response vector-clock exchange.
|
||
- `GossipsubBroadcast` — real-time state-change notifications via Gossipsub.
|
||
- `NetEvent` — unified event enum for `SwarmManager` (Task 2.5) consumption.
|
||
|
||
---
|
||
|
||
## Decision
|
||
|
||
### New module: `crates/chronogrid_net/src/sync.rs`
|
||
|
||
| Type | Role |
|
||
|-----------------------|-------------------------------------------------------------------------------|
|
||
| `VectorClock` | `HashMap<entity_id, hlc_timestamp>` — lexicographic HLC comparison for delta computation |
|
||
| `DeltaPacket` | Serializable CRDT changeset with entity routing metadata |
|
||
| `SyncCodec` | libp2p `request_response::Codec` — length-prefixed bincode framing |
|
||
| `SyncProtocol` | `#[derive(NetworkBehaviour)]` wrapping `ReqResBehaviour<SyncCodec>` |
|
||
| `GossipEvent` | Application-level gossip message (entity_type, entity_id, hlc_timestamp, origin_peer) |
|
||
| `GossipsubBroadcast` | Wraps `libp2p::gossipsub::Behaviour` with topic subscription and event parsing |
|
||
| `NetEvent` | Unified event enum: `DeltasReceived`, `SyncComplete`, `SyncError`, `GossipReceived` |
|
||
|
||
### VectorClock Design
|
||
|
||
- `HashMap<String, String>` mapping `entity_id → hlc_timestamp`.
|
||
- HLC timestamps use lexicographic ordering (format: `<ms>-<logical>-<node_prefix>`),
|
||
which is equivalent to causal ordering — no numeric parsing required.
|
||
- `update()` returns `bool` indicating whether the entry was actually advanced.
|
||
- `missing_from(other)` computes the set of entity IDs where `other` is ahead of `self`.
|
||
- `merge(other)` takes the maximum timestamp per entity (LWW semantics).
|
||
- Fully serializable via `bincode` for wire transmission in `SyncRequest`/`SyncResponse`.
|
||
|
||
### DeltaPacket Wire Format
|
||
|
||
```text
|
||
[ 12-byte random nonce ]
|
||
[ ChaCha20-Poly1305 ciphertext of:
|
||
LZ4(bincode(DeltaPacket { entity_type, entity_id, hlc_timestamp, changeset_bytes, sequence }))
|
||
]
|
||
```
|
||
|
||
- **Serialization:** `bincode` (compact binary, no schema overhead).
|
||
- **Compression:** `lz4_flex` with size-prepended format — Automerge changesets
|
||
compress well (typically 60–80% reduction for structured JSON-like data).
|
||
- **Encryption:** ChaCha20-Poly1305 AEAD with the 32-byte `SwarmKey` as the
|
||
symmetric key. A fresh 12-byte random nonce per packet ensures ciphertext
|
||
uniqueness even for identical plaintexts.
|
||
- **Authentication:** The AEAD tag authenticates the ciphertext; tampering
|
||
causes decryption to fail with an error (not silent corruption).
|
||
- **AAD:** `b"chronogrid-delta"` as additional authenticated data to bind the
|
||
ciphertext to the ChronoGrid protocol context.
|
||
|
||
### SyncProtocol (NetworkBehaviour)
|
||
|
||
- Wraps `libp2p::request_response::Behaviour<SyncCodec>` via `#[derive(NetworkBehaviour)]`.
|
||
- `SyncCodec` implements `libp2p::request_response::Codec` using `#[async_trait]`
|
||
(required by libp2p-request-response 0.27.0 which uses `async_trait` internally).
|
||
- Frame format: 4-byte big-endian u32 length prefix + bincode payload.
|
||
- Maximum frame size: 64 MiB (generous upper bound for large delta batches).
|
||
- `send_sync_request()` / `send_sync_response()` delegate to the inner behaviour.
|
||
- `encode_deltas()` / `decode_deltas()` are static helpers for batch processing.
|
||
|
||
### GossipsubBroadcast Design
|
||
|
||
- Subscribes to topic `chronogrid/state-changes/1` on construction.
|
||
- Message deduplication: SHA-256 of message data as `MessageId` (content-addressed).
|
||
- `ValidationMode::Strict` — only signed messages from known peers are accepted.
|
||
- `handle_behaviour_event()` maps `GossipBehaviourEvent::Message` → `NetEvent::GossipReceived`.
|
||
- All other Gossipsub events (Subscribed, Unsubscribed, GossipsubNotSupported) are
|
||
logged at DEBUG level and produce no `NetEvent` output.
|
||
|
||
### NetEvent enum
|
||
|
||
```rust
|
||
pub enum NetEvent {
|
||
DeltasReceived { from_peer: PeerId, deltas: Vec<DeltaPacket> },
|
||
SyncComplete { peer_id: PeerId },
|
||
SyncError { peer_id: PeerId, reason: String },
|
||
GossipReceived { entity_type, entity_id, hlc_timestamp, origin_peer: String },
|
||
}
|
||
```
|
||
|
||
`SwarmManager` (Task 2.5) broadcasts these events via a
|
||
`tokio::sync::broadcast::Sender<NetEvent>` channel to all application subscribers.
|
||
|
||
### New dependencies added
|
||
|
||
| Crate | Version | Purpose |
|
||
|--------------------|---------|--------------------------------------------------|
|
||
| `bincode` | 1 | Compact binary serialization for DeltaPacket |
|
||
| `lz4_flex` | 0.11 | LZ4 compression for delta payloads |
|
||
| `chacha20poly1305` | 0.10 | ChaCha20-Poly1305 AEAD encryption |
|
||
| `generic-array` | 0.14 | GenericArray for AEAD key/nonce types |
|
||
| `async-trait` | 0.1 | Required by libp2p request_response Codec trait |
|
||
| `libp2p` feature | — | Added `"request-response"` feature flag |
|
||
|
||
---
|
||
|
||
## Test Coverage
|
||
|
||
**100 unit tests pass + 5 doc-tests pass** across all modules:
|
||
|
||
| Module | Tests | Coverage highlights |
|
||
|---------------|-------|----------------------------------------------------------------------------|
|
||
| `sync` | 35 | VectorClock (11), DeltaPacket (8), SyncProtocol helpers (3), GossipEvent (2), NetEvent (11) |
|
||
| `peer_event` | 10 | (from Task 2.3, unchanged) |
|
||
| `transport` | 12 | (from Task 2.3, unchanged) |
|
||
| `discovery` | 13 | (from Task 2.3, unchanged) |
|
||
| `identity` | 9 | (from Task 2.2, unchanged) |
|
||
| `swarm_key` | 16 | (from Task 2.2, unchanged) |
|
||
| `sync` (integration) | 1 | `test_vector_clock_drives_delta_selection` — full encode/decode/merge cycle |
|
||
|
||
### Key test scenarios
|
||
|
||
- **VectorClock:** insert, update (newer/older/equal), `missing_from` with empty/partial/superset,
|
||
merge, iter, bincode round-trip.
|
||
- **DeltaPacket:** encode/decode round-trip, random nonce uniqueness, wrong-key rejection,
|
||
truncated payload rejection, tampered ciphertext rejection (AEAD authentication),
|
||
1 MiB large changeset, bincode round-trip.
|
||
- **SyncProtocol:** batch encode/decode, empty batch, construction.
|
||
- **GossipEvent:** bincode round-trip, invalid bytes rejection.
|
||
- **NetEvent:** all 4 variants — predicates, Display, Clone.
|
||
- **Integration:** two-peer delta selection driven by VectorClock comparison,
|
||
encode/decode pipeline, clock update after merge.
|
||
|
||
---
|
||
|
||
## Consequences
|
||
|
||
- `SwarmManager` (Task 2.5) can now import `SyncProtocol`, `GossipsubBroadcast`,
|
||
`VectorClock`, `DeltaPacket`, and `NetEvent` directly from `chronogrid_net::sync`.
|
||
- The `SyncProtocol` behaviour must be polled by the libp2p `Swarm` event loop in
|
||
Task 2.5; `send_sync_request()` should be called on `SwarmEvent::ConnectionEstablished`.
|
||
- `GossipsubBroadcast.behaviour` must be included in the composite `NetworkBehaviour`
|
||
derive struct in Task 2.5.
|
||
- `chronogrid_core::sync::SyncEngine` (Task 2.6) receives `NetEvent::DeltasReceived`
|
||
and calls `Database::merge_crdt_doc()` for each `DeltaPacket`.
|
||
- No breaking changes to existing modules (`identity`, `swarm_key`, `transport`, `discovery`).
|