Architecture and runtime contracts

Responsibilities and boundaries

Quikcast is a data-plane service: source admission, bounded encoded-byte distribution, listener transport, ICY interleaving, and authenticated HLS object ingest/publication/delivery. Laravel/Go own scheduling, business logic, orchestration, databases, and producer operation.

The server never decodes, encodes, transcodes, or invokes FFmpeg or another media process. FFmpeg may be an external test encoder or upstream producer. TLS initially terminates outside Quikcast. Deployment must allow long-lived uploads/listeners without buffering and must route legacy SOURCE traffic through an appropriate TCP path if the HTTP edge rejects it. Do not trust forwarded client identity from arbitrary callers; use the immediate peer unless a trusted-edge policy is explicitly configured.

The two media subsystems are independent:

A logical station may associate both delivery systems, but source failure, reconnect, mount shutdown, and continuous metadata changes do not end HLS. HLS end/delete/expiry does not affect the continuous source. Service shutdown terminates both through the shared supervisor, without coupling media state.

Non-negotiable invariants

1. Listeners never backpressure source ingest. A listener cannot delay retention eviction or acquire a source-permit dependency.

2. Every queue, ring, request body, upload, retained buffer, registry, and task population is bounded.

3. No lock is held during network I/O or across .await.

4. Cleanup for an old source generation cannot mutate, clear, or release newer-generation state.

5. Published HLS segments are immutable.

6. A visible HLS playlist references only completely uploaded, validated, accepted segments.

7. Malformed traffic for one mount is isolated; it cannot close or mutate another mount.

8. Normal client disconnects never panic or destabilize the process.

9. No unbounded task spawning, per-chunk tasks, or unbounded per-listener channels.

10. Every long-lived spawned task has an owner, cancellation path, and joining semantics.

Admission limits protect shared resources. Mount isolation cannot guarantee unlimited capacity against hostile aggregate traffic; aggregate and per-mount caps prevent one client population consuming all memory.

Concrete component boundaries

• Configuration: load once, resolve secret files, validate typed limits and canonical identities, construct immutable mount allowlist. No scattered environment reads.

• Server supervisor: bounded connection admission, accept loops, signal handling, task joins, global cancellation, observability setup.

• Protocol boundary: bounded prefix classification; HTTP and narrow legacy adapters; common path/header normalization and authentication.

• Mount admission: short synchronous state transition, source generation lease, active generation reference. Static registry reads require no global lock.

• SourceSession: normalized metadata/content type, owned ingress stream, generation lease, source timeout and close reason.

• FanoutRing: bounded immutable payload references, byte/entry accounting, cursor lookup and eviction. No transport or authentication logic.

• Listener body: generation-local cursor, bounded pending frames, audio-byte accounting, later ICY transformation. The connection owner handles transport deadlines/cancellation.

• HLS registry/store/publication (milestone 6): separately bounded registry; rendition-local mutation; immutable media and playlist snapshots; no continuous-source dependency.

Use concrete types. Do not introduce generic repositories, codec factories, speculative storage traits, a global Arc<Mutex<Everything>>, custom lock-free structures, or unsafe code.

HTTP/source adapter contract

Tokio owns sockets; Hyper handles ordinary HTTP framing. Concrete finite API/HLS routing uses the existing HTTP service; Axum has not been needed. Do not put source state machines in route handlers.

The protocol boundary distinguishes standard framed PUT from legacy SOURCE and explicitly supported EOF-delimited source PUT. Both adapters produce the same SourceSession inputs and share all admission, authentication, metadata normalization, and source lifecycle logic. Chunked HTTP decoding belongs to the standard HTTP adapter, not the legacy adapter.

Parse only a bounded prefix to choose the adapter. Preserve over-read audio/body bytes and feed all consumed HTTP bytes back to the selected HTTP transport. Never change ICE/1.0 into standard HTTP and accidentally lose its body semantics. Legacy headers use existing HTTP header grammar/types; custom code is limited to the nonstandard request line and explicitly supported raw-body lifecycle.

Authenticate and validate the configured mount and codec before body consumption or an acceptance response. Ambiguous framing (duplicate conflicting Content-Length, Transfer-Encoding with Content-Length, unsupported codings) is rejected, never guessed. The EOF exception is confined to source-upload routes and never enables request pipelining or generic request-body ambiguity. Request/connection ends immediately after raw source use.

Exact acceptance sequencing and standard-HTTP connection ownership must be demonstrated with the captured fixtures before selecting Hyper integration details in milestone 2. If Hyper cannot represent a measured source handshake, extend only the source adapter; do not implement a second general HTTP server. This is a protocol feasibility gate, not permission to ship an unverified transport shortcut.

Listener streaming uses deterministic HTTP responses: Content-Type from supported media, no fabricated metadata, no declared finite Content-Length, and close-delimited output initially to minimize legacy player ambiguity. HEAD returns headers without registering a streaming listener. New GETs to inactive mounts return 404; capacity exhaustion returns 503. Finite APIs can use normal HTTP keep-alive.

Source state and generation leases

Each configured mount exists even without an active source. Its short-lived admission state is Disconnected, Connecting(g), Streaming(g), or Closing(g). A service/mount admission-closed flag is separate from these source states.

1. Parse/validate/authenticate without claiming the mount.

2. Under the mount admission lock, reject shutdown/duplicate source or reserve a fresh monotonic generation g, transitioning to Connecting.

3. Create the bounded generation-local ring/metadata/cancellation state; allocation failure releases only this lease.

4. Complete the measured wire handshake; transition to Streaming only if the active lease still matches g and admission remains open.

5. Source writes only into its own generation. Listeners subscribe to that exact generation.

6. EOF, idle timeout, malformed body, reset, or shutdown records a reason and transitions the matching generation to Closing.

7. Cancel listeners and notify ring waiters. Release the active mount lease and metadata reference only with an identity check for g.

Every deferred cleanup carries a generation token. Compare-and-release is performed atomically under the admission lock. Generation-specific metadata and counters are owned by that generation, not writable shared state after release. Late completions check generation identity. A cleanup guard must not unconditionally clear a mount field. Never reuse a generation ID; reject admission on counter exhaustion rather than wrap.

Duplicate sources are rejected; there is no replacement policy. Sources can reconnect after the matching lease is released. Existing listeners terminate on source disconnect rather than crossing generations. Pending body data is not replayed into a successor. Clear track/source metadata by constructing fresh state, not by clearing another generation's cells.

Fanout and listener concurrency

A ring has immutable owned audio allocations (at most 16 KiB each), monotonic sequence numbers, byte offsets, a retention floor, and fixed byte/entry capacities. Source append and listener reference lookup use a short per-generation synchronous lock; no network operation or wait occurs inside it.

Admission pre-reserves worst-case retained/staging capacity for a source generation and worst-case response-held capacity for each listener. A shared runtime allocation semaphore that can make ingest wait for a stalled listener is prohibited. Evicted allocations still held by responses fit the already reserved listener allowance; each source retains guaranteed headroom to append/evict within its own bound. Transport prefetch limits must be proven to fit those allowances before the fanout implementation is accepted.

Append evicts oldest entries to satisfy both caps, advances the floor, unlocks, then publishes a coalescing watermark. A source never waits for a listener acknowledgment. Chunk reads are bounded; if a transport supplies a large allocation, copy/rechunk into bounded owned allocations and release the large backing store rather than slice and hide its retained capacity.

Each listener cursor moves forward only. Lookup clones a bounded Bytes reference and returns available data, wait, terminal source state, or SlowConsumer. Wait registration and recheck prevent lost wakeups. Coalescing watermark notifications carry state/version only, not an audio queue.

The connection task monitors ring lag and write progress even if Hyper stops polling the body. Lag below the floor closes with SlowConsumer; stalled output closes at its deadline. Use shared watermark/floor changes and connection-owner selection, not an extra watcher task per listener. Test this under truly stopped sockets. A body callback alone is insufficient.

The temporary opaque MP3 slice joins from at most 64 KiB of retained history, bounded by the actual floor. Arbitrary offsets are not a permanent protocol invariant. Later minimal MPEG audio frame parsing identifies validated boundaries (including supported frame-length rules and false-sync checks), enabling clean joins without building a codec library. AAC ADTS and Ogg support get similarly narrow framing/header/page handling only when their milestone starts.

Task ownership and shutdown

The supervisor uses a bounded JoinSet/task tracker plus admission permits. Completed tasks are continuously reaped, not retained until shutdown. Permit lifetimes cover task completion and reference release. Connection tasks own source ingest/listener responses; any child task needs explicit scoped cancellation and join. No detached task is allowed.

Each long-lived operation selects its work against generation and/or service cancellation and appropriate read/write deadlines. Timer state is constant-size per admitted connection; no timer queue grows with audio chunks.

SIGTERM/SIGINT closes admission and listeners, marks mount admission closed, cancels source generations and HLS mutations, and requests finite HTTP work to finish within the shared ten-second shutdown budget. Join tasks while draining completions; at the deadline abort and join remaining tasks, then report unresolved faults. Shutdown correctness does not depend on process destruction.

ICY and track metadata (milestone 4)

The implemented details are in milestone 4. Support only the narrow /admin/metadata compatibility endpoint: mount, mode=updinfo, song, and useful bounded title/artist/album/genre fields. Authenticate against the target mount's source credentials and bind an update to the current source generation. Decode query fields once, reject duplicate control keys and invalid UTF-8, cap lengths, and prevent cross-mount updates. No metadata update may claim or restart a source.

Normalize into typed TrackMetadata. Explicit song controls StreamTitle; otherwise derive from artist/title. Preserve the distinction between omitted fields and explicitly empty fields so clearing can follow reference behavior. The compatibility adapter translates measured wire semantics; internal consumers see immutable normalized state, not query syntax.

Publish metadata atomically. ICY listener state counts only encoded audio bytes toward its interval (milestone 4 uses the measured fixed 16,000-byte interval). Emit the length byte and padded payload without modifying audio. Pending audio/metadata offsets survive partial writes. The protocol permits 255 units of 16 bytes (4,080 bytes); milestone 4 deliberately limits the normalized title to 1,024 UTF-8 bytes and its padded payload to 65 units. Oversized titles are rejected; valid UTF-8, apostrophes and backslashes are preserved. Unchanged/empty emissions and escaping follow measured fixtures. No ICY framing for listeners that did not request it.

The narrow metadata endpoint may return the measured Icecast XML success/error envelope. That does not introduce XML administration, config compatibility, or other admin routes. Audio-history-associated metadata versus current-state metadata is a separate compatibility decision, recorded in the evidence package rather than silently chosen by the handler.

HLS contract (milestone 6)

Separate identities: stream and rendition, with fixed admitted capacities. Source credentials and HLS ingest credentials have separate scopes. Public paths are /hls/{stream}/master.m3u8, /hls/{stream}/{rendition}/index.m3u8, and generation-and-sequence-addressed .ts objects. Accept only bounded canonical identifiers; never translate them to filesystem paths.

Typed internal /internal/v1/hls operations register/end/delete streams and renditions, upload an MPEG-TS segment, publish a media description, and publish a master description. Input expresses sequence, positive duration, declared content type, references, and expected publication revision. It is not an arbitrary .m3u8 or generic-file upload API.

MPEG-TS only: .ts, video/mp2t; media payload opaque after size/type/ownership validation. No fMP4/CMAF, init segments, byte ranges, encryption, LL-HLS, or raw AAC HLS. Store concrete immutable media objects with format and sequence so future init objects can be added without replacing registry/serving ownership.

Reserve upload and allocation budget before reading; hard-limit actual bytes regardless of Content-Length. A completed upload is validated and accepted before insertion into the immutable store. Reject duplicate sequence uploads with 409; do not silently replace a published object. Interrupted uploads release their own reservations and are never visible.

Under short rendition-local publication serialization, verify expected revision, all references and durations, monotonic media sequence, stable target duration, and live-window constraints. Build a bounded immutable snapshot and atomically publish it; stale revisions return 409. Master publication references admitted renditions and declared codec/bandwidth metadata; no invented bitrate. Store readers take immutable references without a global hot-path lock.

Retention honors RFC 8216: after removing a segment from a playlist, keep it for its duration plus the longest published playlist duration that contained it. Active windows remain at least three target durations when pruning live media. Track grace deadlines with monotonic time and bounded bookkeeping. End marks ENDLIST; deletion stops new publication and preserves existing availability promises before release. Capacity failure rejects new ingest/publication rather than evicting promised data.

Atomic snapshot swaps alone are insufficient: referenced segments are accepted first, store lookup and retention preserve availability through the grace window, and response-held references continue charging allocation budgets until the last owner drops. Generation/publication tokens also prevent late HLS cleanup mutating a re-registered identity.

Live playlists use revalidation and ETags; immutable segment cache lifetimes never exceed promised availability. Playlist MIME is application/vnd.apple.mpegurl. Unknown/expired objects return 404. GET/HEAD are supported; Range returns 416 in this milestone. CORS policy is explicit configuration, default disabled, rather than inferred from source headers.

Implementation details, the typed API, allocation charges and evidence are documented in milestone 6.