Platform Overview

Platform Overview

Secure Mesh (SkyConnect Streaming) is an end-to-end encrypted, real-time multimedia messaging and audio/video calling platform built for ultra-high scale, zero-trust security, and production-grade resilience across Web and Mobile clients, powered by a Rust backend, PostgreSQL, and Redis.

Purpose and Scope

This page is the entry point for the Secure Mesh platform as a whole. It explains the system topology, the technology stack, the responsibilities of each major subsystem (Rust backend, Next.js web client, Expo mobile client, PostgreSQL data layer, Redis realtime fan-out, embedded SFU, and the E2EE cryptographic core), and how the pieces fit together end to end.

Because this is an overview page, it intentionally does not drill into the full implementation of each subsystem. Related topics that deserve their own catalog pages include:

• Backend API & gRPC Services — Axum REST endpoints, Tonic gRPC-Web realtime plane, authentication/refresh token rotation.
• Realtime Engine & Signaling — Redis PSUBSCRIBE fan-out, per-user gRPC streams, embedded str0m SFU media forwarding.
• E2EE Cryptography — Signal Protocol X3DH + Double Ratchet + Sender Keys, deterministic identity seed derivation, multi-device payload fan-out.
• Database & Migrations — PostgreSQL schema (001–010), Snowflake IDs, SKIP LOCKED prekey consumption, composite paging indexes.
• Web & Mobile Clients — Next.js 16 app router client and Expo 57 offline-first mobile app.

Overview

Secure Mesh is architected as a zero-trust messaging + calling platform with three surfaces: a Next.js 16 web client, an Expo 57 (React Native) mobile app, and a Rust backend that owns HTTP REST, gRPC realtime, signaling relay, and media forwarding. The backend embeds a pure-Rust WebRTC SFU (str0m) so that audio/video media forwarding runs inside the same process and workspace as the API, rather than requiring a separate media server deployment.

The defining characteristics of the platform are:

• End-to-end encryption by default: Signal Protocol (X3DH handshake, Double Ratchet sessions, Sender Keys for groups) implemented with @privacyresearch/libsignal-protocol-typescript on clients, with server-side prekey storage in PostgreSQL.
• A single realtime plane: All chat and call signaling flows over gRPC (Tonic + tonic-web for browser compatibility). There is no WebSocket; realtime is gRPC, with Redis used as the internal relay for cross-node fan-out via PSUBSCRIBE call_signals:*.
• Embedded media forwarding: str0m 0.21.0 runs as the streaming-sfu crate in the backend workspace, supporting 1-1 and group audio/video.
• Multi-device identity parity: A server-persisted, non-rotating 256-bit identity_seed lets every device (including fresh Incognito sessions and reinstalled apps) deterministically re-derive the same ECDH P-256 identity keypair, so history and new messages stay decryptable everywhere without central key escrow.
• Offline-first mobile: Realm DB (realm ^20.2.0) provides local message storage, with encrypted MMKV for secret material and react-native-callkeep for native call integration.
• Hard modularity constraint: every service, handler, component, hook, and utility across backend, web, and mobile is split into single-responsibility modules.

Architecture

The following diagram reflects the system topology from the master specification — three client/server surfaces, a single Rust backend plane, and two persistence stores.

Architecture notes:

• Clients never talk to each other directly for signaling — all control traffic converges on the Rust backend, which is the single trust boundary for identity, prekeys, and call state.
• WebRTC media flows through the embedded SFU: the str0m-based streaming-sfu crate forwards audio/video packets for 1-1 and group calls, keeping media inside the backend workspace.
• Redis is an internal bus, not a client-facing API: the backend publishes call signals to Redis and subscribes with PSUBSCRIBE call_signals:* so that signaling generated on one connection can be fanned out to the correct per-user gRPC streams.
• PostgreSQL is the system of record for users, groups, messages, per-device ciphertext, prekeys, push tokens, and refresh tokens; all primary/foreign keys are Snowflake BIGINTs.

Main Subsystems

Rust Backend (Axum + Tonic + Embedded SFU)

The backend is a single Rust workspace (Edition 2024, Tokio async runtime) that exposes two protocol surfaces plus media forwarding:

• Axum 0.8.9 serves the REST API (auth, users, keys, files). Multipart uploads are capped with DefaultBodyLimit::max(10 * 1024 * 1024) (10 MB). There is deliberately no WebSocket — realtime is gRPC.
• Tonic 0.14.6 + tonic-web 0.14.6 host the gRPC service, made browser-compatible via gRPC-Web, and this is the single realtime plane for chats and calls.
• str0m 0.21.0 is compiled in as the embedded SFU (streaming-sfu crate) for media forwarding.
• fred 10 is the Redis client with the subscriber-client feature for PSUBSCRIBE call_signals:* fan-out.
• snowflake_me 2.1.1 generates Snowflake IDs, validated against PostgreSQL advisory locks.
• tower-governor 0.8.0 with SmartIpKeyExtractor performs rate limiting behind trusted proxies.
• argon2 0.5.3 hashes passwords with Argon2id; sonic-rs 0.5.8 is used for hot-path JSON serialization.

The dependency manifest pins these versions explicitly:

[package]
name    = "streaming-backend"
version = "0.1.0"
edition = "2024"

[dependencies]
argon2         = "0.5.3"
axum           = { version = "0.8.9", features = ["multipart"] }
str0m          = "0.21.0"       # embedded Rust SFU (media forwarding)
serde          = { version = "1.0.229", features = ["derive"] }
sonic-rs       = "0.5.8"
sqlx           = { version = "0.9.0", features = ["postgres", "runtime-tokio", "macros"] }
tokio          = { version = "1.53.1", features = ["full", "fs"] }
tonic          = "0.14.6"
tonic-web      = "0.14.6"
prost          = "0.14.4"
snowflake_me   = "2.1.1"
jsonwebtoken   = "11.0.0"
reqwest        = { version = "0.13.4", features = ["json", "rustls-tls"] }
rand           = "0.10.2"
sha2           = "0.11.0"
hex            = "0.4.3"
base64         = "0.23.0"
tower-http     = { version = "0.7.0", features = ["cors", "trace"] }
tower-governor  = "0.8.0"
dashmap        = "6.2.1"
chrono         = { version = "0.4.45", features = ["serde"] }
async-trait    = "0.1.91"
fred           = { version = "10.1.0", features = ["subscriber-client"] }   # Redis (current impl)
tracing        = "0.1.44"
tracing-subscriber = "0.3.23"

[build-dependencies]
tonic-build    = "0.14.6"

Migration caution: the spec explicitly warns that jsonwebtoken 9→11, reqwest 0.12→0.13, rand 0.8→0.10, sha2 0.10→0.11, base64 0.22→0.23, tower-http 0.5→0.7, tower-governor 0.4→0.8, dashmap 5→6, and tonic-web 0.12→0.14 are major bumps requiring API adaptation during Phase 1.

Web Client (Next.js 16)

The web app is Next.js 16 with the App Router, using Server Components and Client Hooks. Key traits:

• Middleware proxy.ts rewrites API calls, injects the Bearer token, and performs transparent token refresh with a 30-second Grace Period using HttpOnly cookies.
• UI routing: the web client defines the (auth)/signup & login auth routes and the (app)/chats, (app)/groups, (app)/status, (app)/calls app routes, with core design tokens in app/globals.css.
• Cryptography is provided by @privacyresearch/libsignal-protocol-typescript (X3DH + Double Ratchet + Sender Keys).
• Realtime comes from the shared @securemesh/realtime-core package (a Unified RealtimeEngine consumed by both web and mobile via transpilePackages).
• gRPC connectivity uses grpc-web / google-protobuf (grpc-web ^2.0.2, google-protobuf ^4.0.2).
• Form handling uses react-hook-form + zod ^4.4.3 validation; icons use lucide-react ^1.28.0.

Mobile Client (Expo 57 / React Native)

The mobile app is Expo 57 with Expo Router (typedRoutes), React Native 0.86.2, and React 19.2.3. Its defining traits:

• Offline-first storage: Realm DB (realm ^20.2.0, @realm/react) stores messages locally so history renders without a network round trip.
• Encrypted secret storage: react-native-mmkv ^4.3.2 with OS Keychain/Keystore key via expo-secure-store.
• Calling & media: react-native-webrtc ^124.0.8 for media + DataChannels, react-native-callkeep ^4.3.16 for iOS CallKit and Android ConnectionService.
• UI-thread animations: react-native-reanimated 4.5.3 + react-native-worklets 0.11.3; message bubble spring slide-up/fade, momentum TabBar indicator, scale(0.97) kinetic button press, emerald pulse StatusRing.
• WebCrypto polyfill: react-native-quick-crypto (install() at boot sets global.crypto.subtle), backed by @noble/curves for deterministic seed→keypair derivation.
• Server state: @tanstack/react-query ^5.101.4 for cache/state; the shared @securemesh/realtime-core engine is linked via Metro watchFolders.

Data Layer (PostgreSQL Schema 001–010)

The database is the system of record. Every primary and foreign key is a Snowflake BIGINT (64-bit signed); constraints enforce multi-device payload fan-out, XOR targeting, soft deletes, and composite index performance for cursor pagination. The migration set:

The messages table enforces that a message targets either a direct receiver or a group — never both — via an XOR CHECK constraint, and stores the group Sender Key ciphertext as the primary payload:

CREATE TABLE messages (
  id                BIGINT   PRIMARY KEY,
  sender_id         BIGINT   NOT NULL REFERENCES users(id) ON DELETE CASCADE,
  receiver_id       BIGINT   REFERENCES users(id)  ON DELETE CASCADE,
  group_id          BIGINT   REFERENCES groups(id) ON DELETE CASCADE,
  payload           TEXT     NOT NULL,     -- Primary Payload / Sender Key Ciphertext
  file_url          VARCHAR(512),
  file_type         VARCHAR(64),
  status            VARCHAR(16) NOT NULL DEFAULT 'sent', -- 'sent' | 'delivered' | 'read'
  read_at           BIGINT,
  edited_at         BIGINT,
  deleted_at        BIGINT,                -- Soft delete support
  created_at        BIGINT   NOT NULL,
  CONSTRAINT msg_target_xor CHECK (
    (receiver_id IS NOT NULL) != (group_id IS NOT NULL)
  )
);

-- Multi-Device Signal Protocol Payload Fan-Out Store
CREATE TABLE message_recipients (
  message_id  BIGINT       NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
  device_id   VARCHAR(128) NOT NULL,
  ciphertext  TEXT         NOT NULL,  -- Device-specific Signal Double Ratchet payload
  PRIMARY KEY (message_id, device_id)
);
CREATE INDEX idx_msg_recipients_device ON message_recipients(device_id);

-- High Performance Composite Indexes for Cursor Pagination
CREATE INDEX idx_messages_direct_paging ON messages (receiver_id, sender_id, id DESC);
CREATE INDEX idx_messages_group_paging  ON messages (group_id, id DESC);
CREATE INDEX idx_messages_sender        ON messages (sender_id);

The one_time_prekeys table powers the X3DH handshake. Prekey bundles are consumed atomically and non-blockingly with FOR UPDATE SKIP LOCKED, preventing database deadlocks under concurrent handshakes:

DELETE FROM one_time_prekeys 
WHERE id = (
  SELECT id FROM one_time_prekeys 
  WHERE user_id = $1 AND device_id = $2 
  FOR UPDATE SKIP LOCKED 
  LIMIT 1
) 
RETURNING public_key, key_id;

Realtime Plane (gRPC + Redis Fan-out + str0m SFU)

Realtime architecture is deliberately layered:

1. Transport: clients hold a long-lived gRPC (or gRPC-Web) stream to the backend — the single realtime plane. No WebSocket exists in the stack.
2. Signaling relay: when a user performs an action that must reach another user’s device (message delivered, call signal), the backend publishes to Redis under call_signals:* and subscribes via PSUBSCRIBE call_signals:*, then feeds matching events into the per-user gRPC streams.
3. Media: once signaling establishes a peer connection, WebRTC media flows through the embedded str0m SFU (streaming-sfu crate), which forwards audio/video for 1-1 and group calls.

E2EE Cryptographic Core

• Protocol: Signal Protocol X3DH handshake → Double Ratchet 1-1 sessions → Sender Keys for group fan-out. Server-side, PostgreSQL stores identity keys, signed prekeys, and one-time prekeys per (user_id, device_id).
• Multi-device fan-out: each sender produces device-specific Double Ratchet ciphertexts for every enrolled target device (recipient devices + the sender’s own secondary devices), stored in message_recipients; devices query their own device_id during sync.
• Deterministic identity derivation: users.identity_seed (32 random bytes via gen_random_bytes(32), backfilled with pgcrypto) is returned on every register, login, refresh, and GET /auth/me. Clients derive the ECDH P-256 identity keypair with scalar = HMAC-SHA256(seed, "secure-mesh-identity-v1") then p256.getPublicKey(scalar) via @noble/curves/nist.js, imported into WebCrypto as a JWK.
• Low prekey threshold: when a fetched user’s remaining one-time prekeys drop below 10, the response includes low_prekeys_warning: true and the client automatically generates and uploads 50 fresh one-time prekeys via POST /keys/prekeys.

Core Flow

The sequence below shows the end-to-end path of a chat message and a call signal, illustrating how REST, gRPC, Redis, and PostgreSQL cooperate.

For calls, the same fan-out path carries call_signals (offer/answer/ICE), after which WebRTC media bypasses Redis and flows directly through the embedded str0m SFU.

Usage Examples

1. Prekey bundle consumption (backend, deadlock-free)

The canonical server-side pattern for X3DH prekey handshakes is the SKIP LOCKED delete shown above — it atomically removes exactly one one-time prekey for a device without ever blocking on a competing transaction. This is the mechanism the backend uses in POST /keys/prekeys and prekey-bundle fetch paths.

2. Deterministic identity key derivation (clients)

Web and mobile derive an identical ECDH P-256 keypair from the shared seed so all sessions share one identity key:

scalar = HMAC-SHA256(identity_seed, "secure-mesh-identity-v1")
public_x, public_y = p256.getPublicKey(scalar)          # @noble/curves/nist.js
private_key = importKey(jwk { kty: "EC", crv: "P-256", x, y, d })
public_key  = exportKey(spki)

The derived keypair is cached per user (web: localStorage["sm_e2ee_identity"]; mobile: MMKV) and idempotently re-registered via POST /keys/prekeys once per page load, so it self-heals after a database reset.

3. Mobile WebCrypto bootstrap

React Native has no globalThis.crypto.subtle, so the app installs react-native-quick-crypto at boot (src/lib/crypto/polyfill.ts, imported first in src/app/_layout.tsx), populating global.crypto.subtle + getRandomValues. The polyfill then provides importKey/exportKey (JWK EC P-256, spki, raw), deriveKey (ECDH P-256 → AES-256-GCM), encrypt/decrypt (AES-256-GCM), and sign (HMAC-SHA256).

4. Workspace composition

The backend is a Cargo workspace (streaming-backend, streaming-sfu, plus shared crates) and both clients link the shared @securemesh/realtime-core package (file:../realtime-core), imported by the web via transpilePackages and by mobile via Metro watchFolders.

Configuration Options

Key platform-level constants and their defaults, as pinned by the master spec:

Failure Modes, Edge Cases & Concurrency

• Concurrent prekey consumption: multiple simultaneous X3DH handshakes for the same device could deadlock on row locks; FOR UPDATE SKIP LOCKED guarantees exactly one consumer per one-time prekey and never blocks.
• Identity key rotation staleness: the spec documents the classic failure where a sender encrypts to a stale cached public key after a recipient’s identity key migrated (random → deterministic), leaving the recipient unable to derive the shared secret and falling back to an [Encrypted Message] placeholder. Mitigation: peer-key cache is session-scoped, in-memory only, and every successfully decrypted message refreshes the sender’s cached key from its embedded sender_pub.
• Pre-migration ciphertext: messages encrypted under pre-deterministic random keypairs cannot be decrypted by new sessions; only new sessions are affected.
• identity_seed sensitivity: anyone holding the seed can re-derive the ECDH private key and read messages, so it is returned only to the authenticated user over TLS and never exposed in PublicUser/search responses or logs.
• Message targeting integrity: the msg_target_xor and call_target_xor CHECK constraints make it impossible to store a message/call that targets both a direct user and a group, or neither.
• Dependency-major-bump risk: several pinned crates (jsonwebtoken 11, reqwest 0.13, tower-governor 0.8, tonic-web 0.14) are major versions requiring API migration during Phase 1/2; fred 10 is noted as already current.

Performance & Operational Considerations

• Cursor pagination: composite indexes (receiver_id, sender_id, id DESC) and (group_id, id DESC) are purpose-built for keyset pagination over Snowflake IDs — no OFFSET scans on large message tables.
• Hot-path serialization: sonic-rs 0.5.8 is used for hot-path JSON while serde derives cover general types; dashmap provides concurrent in-memory caches (e.g., peer-key cache).
• Fan-out cost model: every message writes one row to messages plus N rows to message_recipients (one per enrolled device); idx_msg_recipients_device makes device sync lookups cheap.
• Scale strategy: Redis pattern subscription decouples signaling fan-out from PostgreSQL, and the embedded SFU avoids a separate media-server fleet; rate limiting via SmartIpKeyExtractor protects public endpoints behind trusted proxies.
• Modularity constraint as ops lever: the small, single-responsibility modules keep hot paths readable, testable, and independently deployable.

Extension Points

• @securemesh/realtime-core is the shared Unified RealtimeEngine consumed by both web (transpilePackages) and mobile (Metro watchFolders) — new realtime features are implemented once in this package.
• The backend Cargo workspace hosts the streaming-sfu crate alongside streaming-backend, so new media capabilities (SFU policies, recording, simulcast) plug into the same workspace.
• Client-side E2EE layering: the deterministic identity_seed derivation and the libsignal-based session stack are self-contained client modules; new device types can join by reusing the same seed derivation and POST /keys/prekeys registration flow.
• PostgreSQL migration chain (001–010) is additive and idempotent where backfills are needed (e.g., 010_identity_seed backfills existing rows), so schema evolution follows the established migration pattern.

Related Links

• Backend service layer — REST/gRPC endpoints, auth, and refresh-token rotation (sibling page)
• Realtime Engine & Signaling — Redis fan-out and gRPC streams (sibling page)
• E2EE Cryptography — Signal Protocol details and deterministic identity derivation (sibling page)
• Database & Migrations — full schema and index reference (sibling page)
• Web & Mobile Clients — Next.js proxy/middleware and Expo offline-first app details (sibling pages)