Backend Architecture & Bootstrap

Backend Architecture & Bootstrap

The Secure Mesh Rust backend (streaming-backend) is a zero-trust, end-to-end encrypted real-time messaging and calling server built on Axum (REST) + Tonic (gRPC-Web) with PostgreSQL persistence, Redis pub/sub signaling, an embedded Rust SFU (str0m), and a single async bootstrap sequence in src/main.rs that wires every service into shared application state before serving traffic.

Purpose and Scope

This page documents the overall architecture and startup/bootstrap mechanism of the Rust backend:

• The layered module layout under streaming-backend/src/ (HTTP router, gRPC plane, middleware, models, modules, services, state).
• The complete bootstrap sequence in src/main.rs — from .env loading and configuration, through database migrations, Redis connection, Snowflake initialization, service construction, background task spawning, to the final Axum/gRPC server start.
• Shared application state (AppStateData) and how every subsystem is wired together.
• Runtime topology: HTTP port, gRPC port, PostgreSQL, and Redis responsibilities.

The following topics are intentionally left to sibling pages: individual gRPC service contracts (see the gRPC services page), the data model / SQLx entities (see Data Models & Persistence), middleware behavior (see Middleware & Rate Limiting), and the realtime signaling / SFU media plane (see Realtime Signaling & SFU). This page only covers them where they participate in bootstrap.

Overview

Secure Mesh is an end-to-end encrypted, real-time multimedia messaging and calling application. The backend is a single Rust binary (streaming-backend, edition 2024, Tokio async runtime) that runs two servers side by side:

• Axum 0.8.9 — HTTP REST API (auth, users, groups, uploads with a 10 MB body limit).
• Tonic 0.14.6 + tonic-web — the single realtime plane for chats and calls over gRPC-Web, fed by Redis PSUBSCRIBE call_signals:* fan-out.

The backend follows one hard design constraint: every module must be a single-responsibility unit. This is why main.rs itself is a short orchestration script that delegates each concern (config, db, grpc, middleware, models, modules, router, services, state, utils) to a dedicated module, and why AppStateData exists as a single shared handle to all services.

Bootstrap design intent: every component (database pool, Redis, Snowflake, refresh tokens, call tickets, call relay, SFU, file detection, storage) is constructed before any server starts accepting traffic, so the process never serves a request with a half-initialized dependency graph. Fail-fast initialization, then concurrent runtime servers.

Architecture

The diagram mirrors the exact construction order in main.rs: configuration → database (pool + migrations) → Redis → Snowflake → services → shared state → servers. Each arrow is a real dependency created or consumed at startup.

Module Layout

Bootstrap Walkthrough (main.rs)

The entire process lifetime is defined in #[tokio::main] async fn main() in main.rs. It executes the following stages, in order:

1. Environment loading — dotenv().ok() reads .env if present; the error is ignored because production supplies real environment variables.
2. Tracing — tracing_subscriber::fmt::init() starts structured logging.
3. Configuration — Config::from_env() parses all settings once; the resulting cfg is borrowed throughout startup.
4. Database — db::create_pool(&cfg.database_url) builds the SQLx pool, then db::run_migrations_safely(&db_pool) applies migrations before anything else touches the DB.
5. Redis — a fred client is built from FredConfig::from_url(&cfg.redis_url) and redis_client.init().await? connects it.
6. Snowflake IDs — SnowflakeService::verify_and_init(...) validates the machine ID with a PostgreSQL advisory lock, so only one valid ID generator exists.
7. Service construction — RefreshTokenService, CallTicketService, CallRelayService, FileDetectService, LocalStorage, and SfuService are created and shared via Arc/Mutex where interior mutability is needed.
8. SFU relay wiring — an unbounded mpsc channel connects the embedded str0m SFU to the call relay: outgoing signals are forwarded to Redis.
9. Shared state — everything is packed into Arc<AppStateData>.
10. Background tasks — identity key backfill (for existing users missing key bundles) and a 600-second status-purge loop are spawned with tokio::spawn.
11. Servers — the gRPC-Web server is spawned on host:grpc_port, then Axum serves HTTP on host:port and axum::serve awaits forever (the process lives as long as the HTTP server does).

Usage Examples

Bootstrap skeleton: config → DB → Redis → services → state

The startup script shows the fail-fast ordering principle: every dependency is resolved and connected before the first server spawns.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Load .env file first — ignores error if file is absent (production uses real env vars)
    dotenv().ok();

    tracing_subscriber::fmt::init();

    let cfg = Config::from_env();
    info!("Starting Secure Mesh Backend");

    let db_pool = db::create_pool(&cfg.database_url).await?;
    db::run_migrations_safely(&db_pool).await?;

    // Build fred RedisClient and connect
    let fred_config = FredConfig::from_url(&cfg.redis_url)?;
    let redis_client = Builder::from_config(fred_config).build()?;
    redis_client.init().await?;

    let snowflake = Arc::new(SnowflakeService::verify_and_init(&cfg.database_url, cfg.snowflake_machine_id).await?);

Source: main.rs

Note the design intent of verify_and_init being called exactly once: the comment in the next excerpt warns to share the single SnowflakeService via Arc rather than calling verify_and_init twice, because the initialization is guarded by a PostgreSQL advisory lock — a second call would block or fail.

Shared service graph and single-instance discipline

    let refresh_tokens = Arc::new(RefreshTokenService::new(
        db_pool.clone(),
        snowflake.clone(),  // share the single SnowflakeService — never call verify_and_init twice
        cfg.jwt_secret.clone(),
    ));

    let call_tickets = Arc::new(CallTicketService::new(redis_client.clone()));
    let call_relay   = CallRelayService::new(redis_client.clone(), db_pool.clone(), &cfg.redis_url);
    let file_detect  = Arc::new(Mutex::new(FileDetectService::new()?));
    let storage      = Arc::new(LocalStorage::new(&cfg.local_upload_dir));

Source: main.rs

Key wiring facts:

• RefreshTokenService depends on the DB pool, the shared Snowflake generator, and the JWT secret — it issues/rotates refresh tokens with Snowflake-derived IDs.
• CallTicketService and CallRelayService both share the same Redis client (redis_client.clone()); the relay additionally holds the DB pool to persist call state.
• FileDetectService is wrapped in Arc<Mutex<…>> because it maintains mutable internal state (e.g., MIME signatures) and is not Sync-friendly; every other service is shared with plain Arc.
• LocalStorage is a simple filesystem-backed upload store rooted at cfg.local_upload_dir.

Embedded SFU → Redis relay wiring

The str0m-based SFU never talks to Redis directly. A dedicated mpsc forwarding task bridges the two so the SFU’s send_signal API stays synchronous and decoupled:

    // Embedded Rust SFU (str0m) — outgoing signals forward to the relay
    let (sfu_tx, mut sfu_rx) = tokio::sync::mpsc::unbounded_channel();
    let relay_for_sfu = call_relay.clone();
    tokio::spawn(async move {
        while let Some(sig) = sfu_rx.recv().await {
            let _ = relay_for_sfu.send_signal(sig).await;
        }
    });
    let sfu = crate::services::sfu::SfuService::new(sfu_tx).await?;

Source: main.rs

An unbounded channel is used deliberately: call-signaling messages are small and latency-sensitive, so dropping or backpressuring a signal could stall a WebRTC negotiation. The relay push is best-effort (let _ =), matching the pub/sub nature of Redis signaling.

Shared state and background tasks

    let state = Arc::new(AppStateData {
        db: db_pool,
        redis: redis_client,
        snowflake,
        refresh_tokens,
        call_tickets,
        call_relay,
        sfu,
        file_detect,
        storage,
        jwt_secret: cfg.jwt_secret.clone(),
        start_time: Instant::now(),
    });

    // Backfill deterministic identity keys for users missing a key bundle
    // (server-derives from identity_seed so existing users get keys too).
    let backfill_state = state.clone();
    tokio::spawn(async move {
        let _ = crate::services::identity::backfill_missing_keys(&backfill_state).await;
    });

Source: main.rs

AppStateData is the single injection point for the whole process: every Axum handler and gRPC service receives this Arc and pulls only the pieces it needs. Background work (identity backfill, the 600-second status purge, the gRPC server itself) uses state.clone() so the same handle is shared without locking. start_time: Instant::now() records process uptime for health/observability endpoints.

Server startup order

    // Spawn gRPC (gRPC-Web) server
    let grpc_addr = format!("{}:{}", cfg.host, cfg.grpc_port).parse()?;
    tokio::spawn(crate::grpc::server::serve(state.clone(), grpc_addr));

    // Spawn Axum HTTP service on PORT
    let app = router::build_router(state);
    let http_addr = format!("{}:{}", cfg.host, cfg.port);
    let listener = TcpListener::bind(&http_addr).await?;
    info!("HTTP  → {}", http_addr);
    axum::serve(listener, app).await?;

    Ok(())
}

Source: main.rs

The gRPC-Web plane starts first on grpc_port (realtime must be ready), then the Axum HTTP server binds port and axum::serve blocks forever — the HTTP server is the process’s main lifetime anchor. If the HTTP bind fails, the process exits with an error; gRPC keeps running until then.

Core Flow: Process Startup Sequence

Stage-by-stage rationale:

1. Migrations before servers — no request can observe a missing table, because run_migrations_safely completes before any listener is bound.
2. Single Snowflake — the advisory-lock-verified generator is created once and shared; concurrent initialization is impossible by design (see the “never call verify_and_init twice” comment in main.rs).
3. Background tasks after state — backfill and purge loops only start once AppStateData exists, so they can clone it safely; both ignore errors with let _ = because they are periodic, non-critical maintenance work.
4. gRPC before HTTP — realtime signaling is available before REST accepts traffic, then the HTTP server anchors the process.

Configuration Options

All settings are loaded once by Config::from_env() from environment variables (with .env file support via dotenv). The fields below are the ones consumed directly during bootstrap in main.rs; see the configuration page for the full variable list and defaults.

Design intent: configuration is resolved once at the top of main and threaded through constructors rather than read lazily inside services. This makes the startup graph explicit and testable, and it guarantees all services see one consistent view of the environment.

Failure Modes, Edge Cases & Concurrency

The bootstrap code reveals several deliberate failure-handling strategies:

• Missing .env is not fatal. dotenv().ok() discards the error because production supplies real environment variables (main.rs). Conversely, a missing required config value surfaces later at Config::from_env() / FredConfig::from_url(...)? and aborts startup with an error — fail fast, never run half-configured.
• Infrastructure failures abort startup. db::create_pool(...).await?, redis_client.init().await?, and SnowflakeService::verify_and_init(...).await? all use ?, so the process refuses to start if PostgreSQL or Redis is unreachable. This is intentional: serving traffic without the database or signal bus would corrupt state.
• Advisory-lock serialization for Snowflake. verify_and_init validates the machine id against PostgreSQL with a dedicated advisory lock; the source comment explicitly warns against calling it twice, because a second call would attempt to re-acquire the lock. The single Arc<SnowflakeService> is shared by all consumers (main.rs).
• Interior mutability via Mutex. FileDetectService is Arc<Mutex<…>> — the only service needing mutable shared state (signature tables). The mutex is process-local; there is no contention across processes because file detection is per-node.
• Best-effort background maintenance. Identity backfill (let _ = backfill_missing_keys(...)) and the 600-second status purge (let _ = purge_expired_statuses(...)) ignore errors and run in detached tokio::spawn tasks, so a failure in maintenance never takes down the servers (main.rs).
• Best-effort SFU signal relay. The mpsc forwarding loop drops send_signal errors (let _ =), matching the at-most-once semantics of Redis pub/sub call signaling; a lost signal is recovered by ICE/negotiation retries in the realtime layer (main.rs).
• Concurrency model. Everything is Arc-shared; no locks are held across .await points in the bootstrap path. The unbounded mpsc channel between SFU and relay avoids backpressure stalls on the latency-critical signaling path.

Performance & Operational Considerations

• Single-binary, two-server process. One process hosts REST and gRPC-Web; horizontal scale-out is per-node with snowflake_machine_id disambiguating ID generation and Redis pub/sub distributing signals across nodes.
• Redis as the fan-out bus. CallRelayService pushes call signals into Redis (per the master spec’s PSUBSCRIBE call_signals:* pattern), and per-user gRPC streams consume them — this keeps WebRTC signaling off the HTTP path.
• Scheduled cleanup. The status purge loop runs every 600 seconds (tokio::time::interval), bounding the growth of expired status rows (main.rs).
• Startup cost. run_migrations_safely runs on every boot; combined with pool/Redis warm-up, this makes startup deterministic but not instant — deployments should allow a few seconds of readiness before routing traffic.
• Observability. tracing_subscriber::fmt::init() enables structured logs from the first line of startup, and AppStateData.start_time (Instant::now()) supports uptime reporting.

Extension Points

• New feature modules live under src/modules/ (e.g., auth/controller.rs, status/service.rs) and are wired by adding their router into router::build_router(state) — the bootstrap only needs to construct any new service and add it to AppStateData.
• New gRPC services are added under src/grpc/ and registered in grpc::server::serve; main.rs requires no change beyond the existing spawn since serve receives the full Arc<AppStateData>.
• Storage backends implement the same interface as services::storage::local::LocalStorage; swapping local disk for object storage is a construction-time change (LocalStorage::new(&cfg.local_upload_dir) in main.rs).
• Startup hooks follow the established pattern: build the service, pack it into AppStateData, then spawn any background loop with tokio::spawn after state exists.
• Single-responsibility modules are the primary structural extension constraint: new functionality should be added as new single-responsibility files, not by growing main.rs.

Related Links

• Bootstrap entry point — the complete startup sequence documented on this page
• gRPC server entry — the spawned realtime server (see the gRPC Services page)
• Configuration module — Config::from_env() source (see the Configuration page)
• Database module — create_pool / run_migrations_safely (see the Data Models & Persistence page)
• HTTP router — router::build_router(state) (see the REST API page)
• Middleware and rate limiting (see the Middleware & Rate Limiting page)