Expand description
coven — end-to-end encrypted, multi-writer, bring-your-own-storage SQLite sync, with an encrypted blob store and a cryptographic membership model.
The host app owns its SQLite schema and domain. coven owns the sync layer:
changesets captured via the SQLite session extension, HLC-stamped
and signed per author, encrypted and pushed/pulled through a pluggable
CloudHome, conflict-resolved per row by _updated_at arbitration with a
column-level premerge so concurrent edits to different columns of a row survive.
An append-only Ed25519-signed membership chain wraps the per-store
symmetric key to each member.
Integration contract for the host:
- coven OWNS the SQLite connection. The host opens it once with
[
Coven::builder(config)], declares synced tables and its synced-schemaMigrationladder, and callsopen: coven installs or verifies its one current internal schema, then runs the host’s migration ladder overPRAGMA user_versionfor the app’s tables and seeds the register clock off the rows on disk. The host runs its SQL through [CovenHandle::sql] or [CovenHandle::write]; coven captures each write into the pending-changeset journal as it commits, inside its own journaled transaction. - Every synced table has an
idtext primary key at column 0 and an_updated_at TEXT NOT NULLcolumn, and is declared as aSyncedTablein the builder’ssynced_tablesset with a requiredRowIdentity.(table, id)is one logical row across every device. Independently created rows use canonical UUIDv4 or UUIDv7 ids; shared-key tables intentionally merge equal application keys. A primary-key change removes the old identity and inserts the new validated identity. A plainsync::session::SyncedTable::newtable syncs unconditionally;sync::session::SyncedTable::remote_rootalso syncs the whole table and makes blobs on the row or its FK-descendants always Remote;sync::session::SyncedTable::gated_bymarks a gated root whose boolean gate column decides, per row, whether that row and its declared FK-descendants are shared. See [sync::gate] for the gating semantics. _updated_atis coven’s last-writer-wins register, an opaque Hybrid Logical Clock stamp the host mints with [SqlContext::stamp] and binds into every synced-row write. The host must not parse or compare the stamp as a wall-clock time; coven advances the clock past pulled rows so a later local write always sorts causally after them, which a wall clock cannot guarantee under skew.- When a cloud provider is connected, the host calls
[
CovenHandle::connect_sync]. Which rows carry blobs is declared per table viasync::session::SyncedTable::carries_blob; the host also supplies an optionalblob::BlobTransitionObserverto the builder. A local-only store that never connects a provider still reads and writes through the handle.
§Blob storage model
A blob is opaque bytes a synced row references — a photo, an audio file, a
cover image. Each blob declares two orthogonal properties and has one runtime
state. [blob] holds the full concept tree; the summary:
- Provenance (
blob::Provenance) — where the bytes live while the blob is Local. User-provided: the user’s own file at a path coven references (the Remote→Local restore writes back to a user path, so it needs one). Host-provided: data the host hands coven, which coven keeps in its own local store atstorage/local/<namespace>/<id>(no path needed to restore). - Cache fill (
blob::CacheFill) — how a device gets the bytes while the blob is Remote.CacheEagerfetches into the cache on pull (cover art, so a grid renders from local bytes);CacheLazyfetches on first read (audio). - Locality — the state: Local (bytes on-device, in the user’s file or
coven’s local store) or Remote (bytes in the cloud, each device’s copy a
cache copy).
make_remoteuploads the bytes and flips a gated root’s gate on;make_localbrings them back to a local file and flips it off (see [blob::transition]). A remote root starts Remote and rejects those transitions.
The cache ([blob::cache]) is a Remote-only mechanism: it holds
re-fetchable copies of Remote blobs under storage/cache/<namespace>/…
(evictable, against a per-namespace size budget — [CovenHandle::set_cache_budget])
and storage/pinned/<namespace>/… (kept). A Local blob is never in the cache,
and CacheEager/CacheLazy/pin/budget describe a blob only while it is Remote.
An asset (a cover, an artist image — sync::session::SyncedTable::asset)
rides its subject’s gate but never keeps the subject alive.
Re-exports§
pub use rusqlite;
Structs§
- Activated
Continuation - Exact durable state required to continue an activated Store device.
- Affected
Row - One table/primary-key identity affected by the shared part of a write.
- Blob
Body - A blob as a sized stream of already-final bytes: sealed chunks for an
encrypted home, plaintext for a browsable one. Encryption-agnostic and
concrete (no
dyn Stream).next_parthands the bytes to a streaming upload in bounded windows so a large blob is never held whole in memory; the onlycollectis the single-request path for blobs at or below a provider’s multipart threshold. - Blob
Decl - Where a blob-bearing table’s blob columns live, declared by the host so coven
can derive every blob a row references without a runtime callback. Resolved
against the live schema into a [
crate::blob::decl::BlobDecls] each cycle. - BlobRef
- A blob a row references: its cloud identity, encryption scope, and the two
declared properties (
provenance+fill). coven derives it from the row’s declared columns (crate::sync::session::BlobDecl) via [decl::BlobDecls]. Where its bytes live depends on its locality and provenance: a user-provided Local blob is the user’s file at its path; a host-provided Local blob is in coven’s local store (storage/local/<namespace>/<id>); a Remote blob’s device-local copy is a cache copy (storage/pinned/<namespace>/<locator-hash>/storage/cache/<namespace>/<locator-hash>, built from the validated namespace + exact locator hash — see [cache]). - Circle
- One Circle as
Circles::listreports it: its id, display name, the local identity’s role when it holds active access, and the derived state. The name is absent for a Circle with no resolved metadata (inactive, conflicted, or deleted); the role is present only when the local identity holds active roster membership. - Circle
Close Participant - One participant in an in-flight epoch close and its slot settlement.
- Circle
Close Status - The read-only status of a Circle’s in-flight epoch close: which participant slots hold responses, exclusions, or nothing.
- Circle
Control Coord - Exact coordinate of one signed circle control entry.
- Circle
Epoch Close Id - The stable identity of one Circle epoch close, derived from the durable
operation that opened it. It names the close a
Circles::close_statusinspects and the reserved response and outcome slots that settle it; a close’s identity is fixed for its lifetime and survives cancellation and retry. - Circle
Id - A self-certifying 128-bit circle identity encoded as canonical lowercase base32.
- Circle
Member Info - Circle
Operation Id - Circle
Operation Info - Cloud
Home Config - The cloud home: which provider backs sync and its per-provider settings. One cohesive unit — connecting picks a provider and fills its fields; disconnecting resets the whole thing to default.
- Cloud
Object Version - Opaque provider revision for an exact cloud object.
- Cloud
Versioned Object - Commit
Frontier - Exact materialized cut across author streams.
- Config
- Sync + storage configuration for one store.
- Device
Activity - Activity summary for a single remote device.
- Device
Join Abandonment - Device
Join Activation - Device
Join Cancellation - Device
Join Cleanup Activation - Device
Join Cleanup Receipt - Device
Join Offer - Device
Join Producer Write Revocation - Device
Join Readiness - Device
Provider Access Request - Device
Provider Admission Approval - Device
Provider Admission Completion - Device
Registration Request - Encryption
Service - Manages encryption keys and provides XChaCha20-Poly1305 encryption/decryption
- Fixed
Clock - Every
now()returns the same instant. - Held
Store Position - Hlc
- Hybrid Logical Clock.
- InMemory
Cloud Home - In-memory CloudHome backed by a HashMap.
Cloneshares one backing store, so clones act as separate devices reading and writing the same cloud bucket, and a test can keep its own handle for direct at-rest assertions while each device owns aBox<dyn CloudHome>clone. - Joined
Store - Joiner
Join Closure - KeyFingerprint
- Stable wire identity of one 32-byte encryption key: its full SHA-256 digest, serialized as exactly 64 lowercase hex digits.
- Master
Keyring - A store’s master key material: every key it holds. This is the value custody
implementations store, unlock, and re-protect — never a cipher. coven builds
the
EncryptionServicecipher from it internally; custody never touches cipher machinery. - Member
Info - Membership
Conflict Choice - Migration
- One ordered step in the host’s synced-schema ladder.
- Object
Hash - Owner
Recovery Authority - Exact Owner grant and recovery-stream authority used to create a replacement device when no activated device continuation survives.
- Pending
Write - Durable write information returned by
CovenHandle::pending_writes. - Provider
Admin Join Closure - Provider
Ready Device Bootstrap - Provisional
Device Bootstrap - Published
Position - Exact position that made a write visible to peers.
- RowBlob
Ref - One exact blob-bearing row version. A reference becomes stale when the live row stamp or any declared blob value changes.
- RowChange
- One row change extracted from a changeset.
- Sequential
IdProvider - Deterministic but unique:
"{prefix}-0","{prefix}-1", … Preserves the per-entity uniqueness invariant while being reproducible across runs. - Stepping
Clock - Advances a fixed delta per
now()call: the first call returnsstart, the nextstart + step, and so on. For tests that assert ordering. - Store
Batch Commit Ref - Exact identity of one signed Store commit candidate.
- Store
Commit Coord - Closed coordinate of one Store commit in its author stream.
- Store
Commit Order - Predecessor and dependency order authenticated by one Store commit.
- Store
Device Id - The stable identity of one device in a Store, derived from the Store root and
the device’s registration origin. It names a device across the protocol — in
membership, commit authorship, and epoch-close participation — and is what
Circles::exclude_close_deviceandCircles::close_statusaddress. - Store
Dir - Typed wrapper for a store directory path.
- Store
Layout - The host’s on-disk layout for stores: where they live and what the
database file is called. One rule shared by create, open, join, and
restore, so a host that wants
libraries/<id>/library.dbinstead of coven’s defaultstores/<id>/store.dbnames it once here rather than each flow hardwiring (or working around) coven’s own choice. - Sync
Loop Alerts - Sync
Loop Success - Synced
Table - A table that participates in changeset sync, declared at startup by the host
and passed to [
crate::CovenBuilder::synced_tables]. - System
Clock - Production clock: real wall time.
- Timestamp
- A parsed HLC timestamp.
- Updated
AtStamper - A cloneable handle that mints
_updated_atregister values from a sharedHlc— the register-stamping capability, sliced off the whole [crate::sync::sync_manager::SyncManager]. - User
Keypair - Ed25519 keypair for signing changesets and membership changes. The same seed can derive an X25519 keypair for key wrapping.
- Uuid
Provider - Production provider: random v4 UUIDs.
- WriteId
- Stable identity of one successfully committed host transaction.
- Write
Receipt - Result of one successful host transaction and its durable publication identity.
- Write
Retraction Witness - Durable proof that a previously published write cannot activate.
Enums§
- Audience
- The one audience a synced row belongs to.
- Blob
Cache Error - Why a blob-cache operation failed.
- Blob
Replacement - A blob’s replacement story: whether the row carrying it may ever be repointed at
a different blob. Orthogonal to
ProvenanceandCacheFill; a blob declares all three. - Blob
Scope - Which key encrypts a blob, as a host names it on a
BlobRef. - Cache
Fill - A blob’s Remote story: how a device gets the bytes once the blob is Remote.
A cache-mechanism setting — it describes a blob only while Remote — that applies
to ANY blob regardless of
Provenance. Orthogonal to provenance; a blob declares both. - Change
Op - The operation type for a changeset entry.
- Circle
Close Settlement - The settlement of one participant device’s create-once epoch-close response slot: it published its own applied frontier, an Owner excluded it, or the slot is still empty.
- Circle
Info - One Circle as the local application sees it. A Circle with a single resolved
control is
Active; a Circle whose control history forked into concurrent valid successors isConflictedand carries no name, role, or key until an Owner resolves it. A conflicted Circle refuses authoring and package publication, so it has no single resolved roster or metadata to report. - Circle
Operation Block - Why a durable Circle operation cannot currently publish. One variant per production block site; each future block site adds its own.
- Circle
Operation Kind - Circle
Operation State - Circle
Role - Circle
State - The public derived state of one Circle. Mapped once from the internal current
state;
Circles::listreports it per Circle. - Cloud
Access Outcome - Cloud
Access State - Cloud
Cipher - How a cloud home protects its objects at rest. An
Encryptedhome seals every object under the store key (the default); aPlaintexthome stores objects in the clear so the bucket is browsable, and drops the.encsuffix. - Cloud
Home Credentials - Credentials for the cloud home, stored as a single JSON keyring entry.
- Cloud
Home Error - Errors from raw cloud storage operations.
- Cloud
Home Join Info - Information needed to join a cloud home from another device.
- Cloud
Provider - Cloud home provider selection.
- Config
Error - Configuration errors.
- Custom
S3Exact Slots - Local operator assertion required before a custom S3 endpoint may provide exact protocol and blob slots. This is local configuration and is never accepted from invite or restore data.
- DbError
- An error from the owned database.
- Device
Join Action - Device
Join Cleanup Progress - Device
Join Error - Device
Join Producer - Device
Join Role - Device
Join Status - Device
Provider Admission - Device
Provider Readiness - Encryption
Error - Held
Store Coordinate - Held
Store Position Reason - Home
Storage - How a cloud home stores its objects: opaque (encrypted, unreadable to anyone who can read the bucket) or browsable (stored in the clear at readable paths). This is not about who can reach the bucket — the storage provider’s own access control applies either way; it is about whether what they store is legible. The host picks it once, when it creates the home; it cannot change later (it determines how every object is written). One choice drives two mechanisms together:
- Join
Code Error - Joiner
Join Terminal - KeyError
- Member
Role - Membership
Conflict Info - Migration
Step - How a migration applies its change to the synced schema.
- Provenance
- A blob’s Local story: where its bytes live while the blob is Local, and
whether bringing it back from Remote needs a destination path. Orthogonal to
CacheFill(the Remote story) — a blob declares both. - Provider
Admin Join Terminal - Provider
Write Authority Ref - Restore
Authority - The closed authority a restore operation may exercise.
- RowBlob
Authority - The authority state that determines where one row version’s blob lives.
A remote-audience blob remains
PendingRemotewhile its verified plaintext is local and no cloud object has been created;Remotecarries the exact package authority needed to open its committed object. - RowIdentity
- How
(table, id)names one logical row across every device. - Seal
Error - Why sealing or opening a host’s app-data failed.
- Write
Block - A semantic write fault. Retrying transport cannot change this result.
- Write
Resolution - Write
Status - Current durable state of one host transaction.
Constants§
- CHUNK_
SIZE - 64KB plaintext chunks
Traits§
- Blob
Transition Observer - Notified about coven’s blob transitions, for host-specific bookkeeping and UI: per-blob upload progress while a make_remote uploads, per-blob materialize progress while a make_local copies files back, and a completion hook per direction the host turns into its own UI event.
- Clock
- Wall-clock source. Returns a full
DateTime<Utc>; callers derive.timestamp()/.to_rfc3339()as they need. - Cloud
Home - Device
Join Write Revocation Executor - Device
Provider Access Administrator - IdProvider
- Identifier source. Yields a fresh unique id per call.
- Master
KeyCustody - A store’s master keyring’s custody: who unlocks it, where a newly established or rotated one is written, and how it is removed. Implemented once per protection policy (the OS keyring, a passphrase-wrapped file, an in-memory session value, or a host’s own).
- Part
Sink - The one per-provider streaming-upload surface: a session that accepts ordered
parts and commits. The central [
write_blob] driver opens one of these for a large blob and pumpsBlobBodyparts into it — no backend writes its own upload loop, collect, or progress call.
Functions§
- decode_
invite_ code_ info - Decode an invite code and return UI-ready info.
- decode_
join_ request - decode_
restore_ code_ info - Decode a restore code and return UI-ready info.
Type Aliases§
- BoxPart
Sink - A boxed
PartSinkborrowing its home for'a. - Clock
Ref - Shared handle to a clock. Held by
Clonetypes (CovenHandle,CovenReadHandle) so they clone the handle, not the implementation. - IdRef
- Shared handle to an id provider. Held by
Clonetypes that need to share one id source, so they clone the handle, not the implementation. - Upload
Progress - Reports how many bytes of a
writehave reached the backend so far. Called with the cumulative byte count as the body uploads; backends that can’t observe sub-call progress call it once at the end with the full size. The count is of the bytes handed towrite(the encrypted payload).