Resources, errors, dependencies, and validation

Typed startup configuration

Load a single typed TOML file with documented narrow environment overrides for the config path/log level. Resolve high-entropy source/HLS machine secrets through bounded secret files or explicit secret fields. Constant-time compare bounded credential values; no custom SHA-256 password storage. A future human-password feature must use a password KDF. Never log Authorization or production secrets.

Reject invalid configuration before binding: duplicate/colliding canonical mount paths; reserved /admin, /internal, /health, /metrics, /hls namespaces; zero capacities; impossible memory budgets; invalid timeout combinations; duplicate identities; oversized secrets; unsupported codec declarations. Reject dot segments, encoded separators, NUL, controls, and malformed percent encoding. Do not repeatedly decode identities. Use normalized allowlist lookup rather than arbitrary dynamic mount creation.

Initial tunable ceilings (not performance promises):

Continuous source bodies intentionally have no cumulative lifetime byte ceiling: they are live streams. Their resident allocations, read chunks, parser state, and staging are bounded, and reads are subject to inactivity limits. Never collect a live source body. Every finite API/metadata/HLS body additionally has a hard total payload limit enforced while reading, not merely from a declared Content-Length.

• Continuous ring: 4 MiB payload, 4,096 entries per active generation; audio allocations at most 16 KiB; temporary join window 64 KiB.

• Listener admission: 10,000 service-wide and 10,000 per mount by default, both configurable; total connections 12,000. Static configured mount count is additionally capped by startup aggregate budget.

• Handshakes: 128 concurrently; source connections no more than one per mount plus bounded rejected handshakes. No unbounded permit waiters: reject/close at admission when no permit is available.

• Configured mounts: 64 initially. Retained generations: at most two per configured mount (active plus closing); old rings still referenced consume that quota. Continuous retained-audio budget: 1 GiB, independently bounded from transport/task overhead. Listener body prefetch/pending references: at most 32 KiB per listener including transport-queued body frames, plus 1,041 bytes and 128 bytes bookkeeping per listener for response-held ICY blocks. Each retained generation also reserves one such block. Validate reservation arithmetic at startup; lower incompatible budgets fail validation. With 64 mounts, two 4 MiB rings each and 10,000 listeners at 32 KiB, worst-case ring/reference reservations are 824.5 MiB before small ingress/metadata allowances. A reconnect exceeding its retained-generation quota is rejected rather than awaiting old listeners.

• Request target: 2 KiB; headers 16 KiB/64 fields; finite error/API response body 64 KiB except the explicitly larger HLS playlist ceiling. Metadata query fits target cap; milestone 4 bounds aggregate decoded query bytes at 2 KiB/16 fields, normalized title at 1 KiB and padded wire block at 1,041 bytes; source metadata aggregate at most 8 KiB. Credential decoded representation at most 1 KiB.

• Header deadline five seconds; source idle timeout ten seconds; output write-stall timeout ten seconds; service shutdown ten seconds. Body-specific total deadlines apply to finite ingest, not the duration of live source sessions. HLS uploads: 30-second total deadline, ten-second inactivity deadline.

• HLS milestone 6: 2 MiB segment ceiling, 256 KiB playlist description/snapshot ceiling, four concurrent uploads, 128 MiB allocation budget per rendition, 512 MiB aggregate HLS allocation budget. Bound stream/rendition counts and window segment count (256 initially). Registration rejects target duration/window/capacity combinations that cannot preserve advertised and grace retention.

HTTP parser buffers must be explicitly configured; defaults are not a contract. Hyper's max_buf_size is an I/O-buffer limit, not proof of a total header/body memory bound. A bounded prefix/parser and measured body buffering are still required. Configure the required Tokio timer when setting Hyper header deadlines.

Source headers are normalized once by shared code. Unknown fields are ignored within the total header bound. Invalid/overflowing numeric Ice-Audio-Info members are omitted individually; they do not create arbitrary allocations or guessed bitrate values. Supported content types are an explicit allowlist, never inferred from mount filename. Conflicting Content-Type/auth/framing fields are rejected. Construct response HeaderValue safely; unrepresentable source metadata is omitted with bounded diagnostics rather than a panic or header injection.

Hyper's current HTTP/1 builder also exposes max_header_size; configure it alongside max_headers and max_buf_size rather than using an I/O buffer cap as a proxy for header limits. Source transport queue/prefetch ceilings must be verified against the chosen release. Reference: https://docs.rs/hyper/latest/hyper/server/conn/http1/struct.Builder.html (inspected 2026-10-07).

Ownership and copy accounting

Track the complete allocation once, not once per Bytes clone. Arc/Bytes references keep an allocation charged until its final drop. Slices retain the backing allocation; bounded visible length is not enough. Account separately for object/ref bookkeeping and peak transient copies.

Continuous application-memory upper bound consists of:

admitted generations × (ring allocation cap + entry overhead + metadata + ingress staging)

+ admitted connections × (bounded parser/transport buffers + body staging + pending payload references + task/cursor state)

+ bounded handshake state + registry/config + metrics/logging buffers + allocator headroom.

Old generation rings still owned during cancellation remain charged. New source admission cannot exceed aggregate retained-generation budget even if the active lease was released. Cap response-held audio references and transport buffering; do not allow Hyper to eagerly accumulate the entire ring per listener. Source admission cannot block waiting for listeners to drop old allocations; reject it if reservations are exhausted.

The expected copy path is kernel → receive allocation → bounded retained allocation when rechunking is required → socket/kernel. Listener fanout clones references, not encoded audio. ICY creates small bounded metadata blocks and slices audio by reference. Actual transport flattening/TLS copies must be documented after measurement. Do not claim zero-copy.

Admission reservation arithmetic must guarantee ingest headroom even with every admitted listener holding its maximum allowed evicted data. Do not implement a global payload budget whose exhausted permits cause source reads to await listener progress. Bound old-generation reservations until all owners release them; a reconnect may receive CapacityExhausted rather than waiting or invalidating the previous generation's budget.

HLS aggregate allocation accounting includes accepted advertised segments, grace segments, staged accepted-but-unpublished segments, uploads in progress, response-held evicted objects, current/old response-held playlist snapshots, transient generation buffers, bookkeeping and allocator/backing retention. Evict unpublished staging at a bounded timeout (30 seconds initially), unless an in-progress validated publication holds a budgeted reference. Refusal to publish/accept because of budget is a controlled error, not permission to violate a retention promise.

Reserve bytes and object slots before allocation. Uploads without Content-Length reserve the maximum; declared lengths still enforce actual read limits. Bound transient copies when assembling an upload, including simultaneous mutable input and final immutable output. Release permits/reservations on EOF, validation failure, cancellation, and last-reference drop. Separate hard application allocation accounting from measured RSS, allocator caching, kernel sockets, and page cache. Document OS file-descriptor/socket budgets in deployment instructions.

Error contract

Use typed categories, not String as a universal error. Preserve diagnostic source/context internally and map to bounded public messages:

• MalformedRequest/InvalidPath/InvalidMetadata/InvalidPlaylist → 400.

• Unauthorized → 401 with the appropriate Basic challenge for compatibility routes.

• UnknownMount/InactiveMount/UnknownHlsObject → 404.

• MethodNotAllowed → 405 with bounded Allow.

• DuplicateSource/DuplicateSequence/StaleRevision → 409, except any source-wire compatibility-specific rejection captured in protocol evidence.

• BodyTooLarge → 413; UnsupportedMedia → 415; HeadersTooLarge → 431.

• CapacityExhausted/ShuttingDown → 503.

• InternalFault → 500 with no raw internals.

• HeaderTimeout → 408 only if a safe response remains possible, otherwise close.

• SlowConsumer/ClientDisconnected/SourceDisconnected/SourceIdleTimeout/WriteStall/Cancelled → terminate established sessions with a structured reason; never attempt a second HTTP status after headers.

HTTP status mappings are Quikcast policy; reference fixture outcomes are recorded separately. Normal EOF/reset/cancellation is debug-level or a connection summary, not an alarming error. Invalid body framing closes that source generation only. Allocation/admission failures reject the request and retain state integrity.

Dependencies

Do not create a Cargo manifest in milestone 1. Resolve current stable compatible versions at implementation time and commit Cargo.lock; record features and direct/transitive impact. The proposal is:

• Tokio: async sockets/timers/signals and bounded sync channels. tokio-util: cancellation/task tracking where not already covered by JoinSet.

• Hyper + hyper-util + http-body-util: maintained HTTP framing, Tokio adaptation, bounded body implementations. Axum only where routing materially helps.

• bytes: immutable shared encoded payloads. http/httparse: header types and narrow compatibility parsing; no bespoke duplicate HTTP grammar.

• serde + toml: configuration and typed management messages. thiserror: meaningful typed errors.

• tracing + tracing-subscriber: structured logs; metrics + metrics-exporter-prometheus: bounded-label counters/gauges/histograms.

• base64 + subtle (or equivalent maintained constant-time comparison): Basic machine credentials. No password hash crate needed initially; no SHA-256 credential scheme.

• arc-swap: milestone 6 uses charge-owned immutable HLS registry snapshots. Stream/rendition mutations use short synchronous standard mutexes. No DashMap.

• serde_json: milestone 6 uses bounded typed JSON messages, unknown-field rejection and size-limited collection/string visitors.

• getrandom: milestone 6 obtains a startup generation namespace from OS entropy to avoid normal restart object-URI/ETag collisions. No credential hashing.

• Test-only proptest/cargo-fuzz and Criterion when useful. Python standard library for reference capture. FFmpeg/curl/Liquidsoap are external test clients, not runtime Rust dependencies.

Every added crate must solve a concrete need not already covered by std/current stack; verify current maintenance and license, avoid oversized feature sets. Forbid unsafe in Quikcast production code initially (dependencies may have audited internal unsafe; do not conflate these policies).

Observability

Private /health/live, /health/ready, /metrics. Readiness means valid configuration, functioning supervisor, and open admission, not an active source. Bind administrative endpoints separately to loopback/private interface by default.

Metrics: active_sources, active_mounts (mounts with active continuous generations), active_listeners, listeners_connected_total, listeners_disconnected_total, source_connections_total, source_disconnects_total, bytes_ingested_total, bytes_served_total, slow_consumers_total, errors_total. Add HLS request/segment/byte metrics only with that subsystem. Counters increment once at defined lifecycle points; counters/gauges use scoped guards so cancel/error paths release exactly once. Distinguish audio served from ICY overhead in byte accounting; HTTP-body acceptance is not proof that a remote player consumed bytes.

Histograms: finite request duration, source authentication/admission duration, HLS request duration. Continuous connection duration is a separate lifecycle measurement. Resource gauges expose retained allocations, response-held bytes, upload reservations, and rejected admissions. Never create label values from remote IP, user-agent, listener/source IDs, raw paths, arbitrary codec strings, or mount UUIDs. Use fixed enums; no per-mount labels initially.

Connection summaries include mount, listener/source ID, generation, peer, codec/content type, sent bytes, duration, reason. Logs are bounded/rate-limited under malformed floods. No audio-chunk info logs or segment-request info spam; no unbounded async logging queue.

Test strategy and gates

Implement tests alongside each feature. Reference observations are fixtures, not a substitute for testing Quikcast.

• Ring/unit/property: monotonic cursor/floor, fixed byte/entry capacities, tiny chunks, large backing allocations, eviction, wrap/exhaustion, held evicted chunks, wake registration races, source nondependence on listeners.

• Lifecycle/integration: failed handshake rollback, simultaneous source claims, disconnect/reconnect, cleanup delayed past successor admission, metadata generation guards, admission at shutdown, source/client reset and cancellation without panic.

• Transport: exact encoded bytes, over-read headers/body, chunked framing, rejected ambiguous framing, expected admission handshake, HEAD, unsupported media, bounded slowloris. Test stopped sockets (not only slow application polling), verify source progress and healthy listener delivery continue.

• ICY: empty/changed/cleared metadata, max payload, byte length, padding, UTF-8, escaping, interval edges, partial writes, unchanged blocks, cancellation/disconnect. Extract audio and compare byte-for-byte with source.

• Codec: narrow valid/invalid frame/page fixtures, false sync, split input, required Ogg headers, boundary-aligned joins; no decoder tests pretending to be a media library.

• HLS: partial upload invisible; reference absent object rejection; stale revisions; duplicate sequence conflict; fixed target duration; sequence progression; grace deadlines; concurrent read/publication/delete; slow responses holding allocations; exhausted upload/global budgets; playlist generation/MIME/cache/expiry; independent lifecycle.

• Fuzz/properties: source headers/audio-info, bounded path/query decode, ICY encoding, HLS typed descriptions/generation; framing only when introduced. Controlled error, bounded allocation, no panic on hostile input.

Required milestone gates once Rust exists: cargo fmt --check; cargo clippy --all-targets --all-features -- -D warnings; cargo test --all; cargo check --all-targets; cargo audit if available (record unavailable tools). No placeholder paths, ignored warnings, commented-out implementations, runtime unwrap/expect, or dead code reserved for future milestones.

Benchmark strategy

Document a dedicated Linux host, CPU/RAM/kernel, release build/toolchain/dependency lock, file-descriptor/socket limits, and separate load generators. Record exact scripts, duration, repetitions, raw results and limits. External TLS load is a separate scenario. No performance numbers are asserted by this design.

Scenarios: one source/1,000 listeners; 10,000 where hardware permits; many mounts with smaller groups; connect/disconnect churn; stopped/slow consumers; simultaneous resets; HLS playlist QPS/segment serving; mixed continuous/HLS.

Measure CPU user/system, RSS and application allocation budgets over time, allocation counts/sizes, ingress/egress throughput, active connections, source progress, slow-consumer reasons, finite-request p50/p95/p99, shutdown time. Include warmup and sustained runs long enough to traverse eviction/grace windows. Compare baseline and change under identical settings. Require steady-state memory plateau within documented bounds and unchanged source/healthy-listener progress under slow consumers before optimizing.

Milestones and review stops

1. This architecture/evidence package: documentation, exact provenance, compatibility matrix, captured raw interactions. Review before production code.

2. MP3 opaque-byte slice: static mount, machine Basic auth, PUT/required SOURCE, bounded fanout, multiple listeners, source generations, disconnects, shutdown, metrics, integration tests. No ICY implementation or HLS.

3. Prove compatibility, bounded memory, real stalled sockets, churn, and measured performance. Do not advance on superficial tests.

4. ICY track updates/interleaving and full protocol-sensitive tests.

5. AAC ADTS, minimal MP3 boundary awareness where needed, Ogg Vorbis/Opus header/page joins. No decoders.

6. Independent authenticated MPEG-TS HLS with typed ingest, immutable publication, retention, tests and benchmarks.

No /status-json.xsl, XML admin, directory listing, relays, fallback mounts, Icecast config compatibility, generic upload, fMP4/CMAF, LL-HLS, or media-process execution without a later explicit request.