Frozen-scope manual validation checklist

Targets and evidence requirements

Currently available reference tools are FFmpeg 8.1.2 and curl 8.7.1. Their fresh functional proof results are in the freeze review. BUTT is an existing operator workflow, with historical user-reported success; its current version/session has not been reproduced. Required interactive listener/HLS players and the actual TLS/proxy deployment are still to be confirmed. VLC/browser/Nginx are candidate targets, not assumptions about the user's required or installed deployment. Ordinary local Quikcast origin/relay behavior has current smoke evidence; required remote Icecast/HTTPS origins need exact target identities.

For every session record date/operator, Quikcast commit and executable SHA-256, sanitized effective configuration and limits, client/upstream/proxy versions, source codec/MIME, requested URL/mount topology with secrets redacted, actions and expected/actual results, screenshots or bounded request/log captures where useful, decoder/player error messages, final process/resource cleanup and PASS/FAIL/PENDING disposition. Never attach credentials, Authorization headers containing real secrets or raw private configuration. Existing proof captures use explicitly disposable fixture secrets.

Before the overall manual gate can pass, identify the mandatory encoder, continuous player, HLS player/producer and proxy deployment; define which scenarios apply to each. Optional/unneeded candidates can be marked NOT REQUIRED with the scope decision recorded, rather than invented successful tests. Do not promote an optional feature to mandatory without that decision.

1. Fresh install/start and encoder admission — PENDING operator session

1. Follow the operator runbook with separate generated local source, management and HLS credentials; enable only selected subsystems. Start using a draining Unix stdout collector and the documented public/private bindings.

2. Confirm readiness/liveness before sources connect, native management authentication and absence of private routes on the public binding. Missing/invalid configuration or credentials must fail before serving; API requests with the wrong scope must reject.

3. Use the named actual encoder to connect with its supported PUT/SOURCE path. Exercise MP3, AAC ADTS, Vorbis and Opus with appropriate external source tools; do not claim a single encoder supports a codec it cannot produce.

4. Confirm correct MIME/station/source/codec state in native inspection. An authenticated second source on the same mount must reject; no source replacement or priority behavior is expected.

5. Restart the encoder, verify a fresh generation and listener reconnection, and confirm an unrelated active mount continues. Record actual encoder behavior after refusal or reset.

Current supporting evidence: automated admission/ownership tests and six fresh fixture-to-FFmpeg decoder cases. Those are not substitutes for the named encoder session.

Fresh BUTT session observed, 2026-10-08: the user reported connecting BUTT to /beef on the running local server, alongside a relay on /radio.mp3. Concurrent, bounded eight-second public listener captures returned HTTP 200 and audio/mpeg on both mounts. External FFmpeg decoded three seconds from each with exit 0 and no error messages. /beef advertised 320 kbps, stereo, 44.1 kHz producer declarations; /radio.mp3 advertised station Spirit R&B. Curl exit 28 was the deliberate capture timeout. Neither source nor the running server was modified. Evidence: evidence/manual-validation/butt-and-relay-2026-10-08/report.json, headers and audio samples. BUTT version, running executable/configuration identity, audible player judgment and remaining admission/restart scenarios are not yet recorded; the overall gate remains pending.

Operator-confirmed recovery, 2026-10-08: the user confirmed that disconnecting and reconnecting BUTT works for the current /beef session. Record the encoder disconnect/reconnect check as PASS by user observation. This confirmation does not separately establish automatic listener recovery, a particular player's audible playback or metadata display, or the other admission/restart checks.

2. Continuous players, ICY and late join — PENDING operator session

1. Connect each mandatory player before audio and during an established stream. Verify audible playback, station identity, correct supported late join and recovery behavior for all required codecs.

2. For MP3/AAC, update title, clear it, update again, and verify the player's displayed track and the API generation/revision. Compare a listener with icy-metadata: 1 and one without it. Ogg comments remain the external producer's in-band responsibility.

3. Observe a new listener on active Vorbis/Opus after initialization. An ended Ogg chain may reject new joins; do not label that documented contract a failure without a required use case.

4. Disconnect one listener through its opaque native identity and one source with a generation guard. Confirm intended target termination, permit cleanup, old generation protection and continued unrelated playback.

5. Apply the configured listener cap using a small functional session count. Confirm the next admission rejects and capacity is released when a listener ends. Do not turn this into a capacity/load trial.

Current supporting evidence: the existing transport/ICY/concurrency suite and fresh preserved-byte/decoder/late-join proofs. Interactive listening/display/reconnect behavior stays pending.

3. Relay upstreams and controls — PENDING required-upstream/player session

Remote functional session passed, 2026-10-08: the user selected http://152.53.111.31:8000/lofi. A temporary loopback Quikcast instance relayed this anonymous HTTP MP3 source on /lofi using the frozen executable SHA-256 0f259dc7b8b062a87af33e09bff1a75acd39ad80dc2bff842be2d7a1daa8af7c. A 12-second curl capture received 353,566 encoded bytes; external FFmpeg decoded five seconds with exit 0 and no error messages. Curl exit 28 was the intentional sample timeout. Station name Lofi Radio and a nonempty upstream track title reached native source inspection. Administrative reconnect established generation 2 after generation 1; a stale control revision returned 409. Administrative stop remained stopped without another attempt after six seconds, longer than the configured five-second maximum retry delay. The owned server exited with code 0.

Evidence: evidence/manual-validation/remote-lofi-2026-10-08/report.json, listener headers and bounded audio capture. Reproduce with python3 tools/proof/remote-relay.py http://152.53.111.31:8000/lofi --output evidence/manual-validation/remote-lofi-repeat. This validates remote delivery/decoding, observed station/track propagation and native controls. Audible interactive playback, track change/clear, automatic recovery from an upstream outage, required HTTPS/Basic targets and proxy deployment remain pending. No remote service was modified, and no performance measurement was taken.

1. Configure one dedicated local relay mount pointing to the selected remote continuous origin. Exercise required anonymous and Basic-authenticated paths and a certificate-validating HTTPS upstream. Wrong credentials, invalid certificates, redirects and unsupported response/media must remain visible relay-local failures.

2. Listen through the local mount with the mandatory player. Verify station metadata, reliable track change/clear and exact encoded stream behavior. For Ogg, validate active initialization/late join; no ICY injection or comment rewriting is expected.

3. Briefly interrupt upstream service, allow EOF/reset/timeout, and observe bounded connecting/backoff/connected transitions. Verify the old local generation/listener ends and the player reconnects when a new generation becomes available. Seamless migration is outside the contract.

4. Stop the relay through /api/relays{mount}; confirm state becomes stopped and retry stays suppressed. Reconnect with the latest control revision; verify stale revisions reject. Disconnect the relay-owned source with its generation and confirm automatic retry does not undo the action.

5. Rehearse a malformed/partial response only on a controlled fixture origin. Metadata with intact framing is skipped when invalid; untrustworthy/truncated framing closes the attempt. Keep another mount, HLS read and native API usable during the failure.

Current supporting evidence: automated HTTP/TLS/DNS/auth/metadata/control/failure tests and a fresh local Quikcast-origin/curl relay smoke. Specific external Icecast/TLS topology and interactive player recovery stay pending.

4. HLS producers and required players — PENDING live rolling-player session

Live rolling functional session passed, 2026-10-08: the user ran external FFmpeg against http://152.53.111.31:8000/spirit, producing AAC-LC, stereo 44.1 kHz, nominal 128 kbps, MPEG-TS segments and a six-segment rolling playlist in /tmp/quikcast-hls.KdNlxZ. The external test adapter tools/proof/live-hls-publish.py ingested completed segments and published typed media/master descriptions into a separate owned loopback Quikcast instance. Eight advancing windows were published in a bounded 30-second session. External FFmpeg decoded 18 seconds over the served HLS URL with exit 0 and no error messages, including media from new publication windows. The adapter ended its playlist and stream; the temporary server exited 0. Existing BUTT/relay sessions were untouched.

Evidence: evidence/manual-validation/live-spirit-hls-2026-10-08/report.json. This session used executable SHA-256 65a6e9b0a4423c8f0162f9703fd16968b79de4d6d962bfe5f5ee3b32f21f7e6f, which differs from the initial frozen binary; this result establishes this session's functional behavior without carrying forward the earlier build's full test evidence. The decoder used -seekable 0 -http_seekable 0 -http_persistent 0 for the documented whole-object contract. Interactive player acceptance, default-player Range compatibility, interruptions, proxy/TLS and the other lifecycle cases remain pending. This adapter is external validation tooling; no media conversion or directory watcher was added to the daemon.

1. Use the selected external producer to register/ingest/publish already-produced MPEG-TS HLS assets through the typed API. Do not derive HLS from a continuous mount or introduce runtime media conversion.

2. Play a live rolling window in every mandatory HLS player through the intended HTTP/TLS path. Observe ongoing advancement, CORS where needed, ETag/revalidation behavior and recovery after a brief interruption.

3. Record any actual Range requests and their response. Range currently returns 416. FFmpeg's known working whole-object options are -seekable 0 -http_seekable 0 -http_persistent 0; a named required player failing with its required defaults is a blocker to review, not evidence of universal compatibility.

4. Exercise final ENDLIST, expiry/grace, deletion, stale generation/revision and producer authentication/errors; confirm advertised objects remain playable for their promised lifetime and unrelated continuous delivery continues.

5. Drain: producer mutations reject while existing HLS reads remain available; a stopped live producer cannot advance indefinitely from retained content. Restart: volatile objects are absent until the external producer repopulates them. Validate that workflow with the required player.

Current supporting evidence: 31 fresh HTTP/publication/finalized-playback checks with FFmpeg/curl and concurrent continuous delivery, plus existing HLS retention/race/drain tests. Those do not establish a live rolling browser/native player session.

Operator-confirmed HLS playback, 2026-10-08: after running the external publisher against the main local server, the user reported that it printed the playback URL and that playback works. Record local HLS playback as PASS by user observation, alongside the separate automated rolling-publication/decoder session above. The exact player/version, playback URL, observed session duration and main-server executable identity have not been recorded. This confirmation does not independently establish interruption recovery, default-player Range behavior, TLS/proxy compatibility or the remaining lifecycle scenarios.

Playback URL recorded and rolling publication observed, 2026-10-08: the user supplied http://127.0.0.1:8000/hls/spirit-test-8b4b2554/master.m3u8. Read-only checks returned the AAC master and media playlists; the media sequence advanced from 243 to 244 across two observations five seconds apart, retaining a six-segment live window without ENDLIST. Evidence: evidence/manual-validation/live-spirit-hls-2026-10-08/operator-playlist.json. Player identity/version and the other pending deployment/lifecycle checks remain unconfirmed.

5. Intended TLS/proxy/private-management deployment — PENDING target selection/session

Local Nginx/TLS rehearsal passed, 2026-10-08: tools/proof/nginx-tls-smoke.py ran Herd's bundled Nginx 1.25.4 (nginx-x86) as an independent foreground process with its own temporary prefix/configuration, ephemeral loopback TLS port and one-day test certificate. Quikcast ran separately with disposable source/HLS/management credentials. All 36 functional checks passed. Client contexts explicitly trusted only the temporary certificate and verified its identity; a default trust context rejected it. No system trust store, Herd configuration or existing streaming session was changed.

The tested public proxy routed continuous audio and HLS to Quikcast's public binding, used HTTP/1.1 upstream, disabled request/response buffering and caching, replaced incoming XFF with the socket client address, and returned 404 for private API/HLS producer/health/metrics routes. A still-open chunked PUT delivered encoded MP3 bytes before upload EOF; station and ICY track metadata reached HTTPS listeners. Spoofed XFF did not change inspected source/listener identity. HLS publication used the private binding; HTTPS playlist/revalidation and external FFmpeg HLS decoding passed with certificate verification enabled and the documented whole-object options. The user-selected /lofi upstream delivered decodable audio through HTTPS, and reconnect created a working new relay generation. Nginx and Quikcast exited 0; all owned threads/processes joined and temporary credentials/certificate were removed.

Evidence: evidence/manual-validation/nginx-tls-2026-10-08/report.json, including the sanitized Nginx configuration and exact executable identity. Reproduce with python3 tools/proof/nginx-tls-smoke.py --nginx /Applications/Herd.app/Contents/Resources/nginx-x86 --relay-url http://152.53.111.31:8000/lofi --output evidence/manual-validation/nginx-tls-repeat. The configuration follows Nginx's documented proxy buffering/headers and TLS certificate directives.

The initial adapter run stopped at an ETag header capitalization lookup; case-insensitive handling was corrected and the complete rehearsal rerun successfully. This was a test-adapter issue, not a daemon change. Actual deployment proxy/domain/certificate selection remains unconfirmed. This local test is not production deployment qualification and does not establish BUTT's actual TLS/proxy upload behavior, legacy SOURCE compatibility, live rolling HLS through the intended public route, or all trusted/untrusted address-family cases. The chunked PUT fixture is an explicit supported HTTP path; it must not be represented as a fresh BUTT-through-Nginx test. No performance tuning or profiling was run.

1. Record the actual proxy/version, TLS termination, DNS and public/private route layout. Verify the configured certificates and player/encoder paths. Native inbound TLS and PROXY protocol are unselected features; external TLS and trusted XFF are the frozen contract.

2. Verify no upload/response buffering, stream caching or unsuitable short timeout interferes with continuous sources/listeners. Confirm supported PUT or TCP pass-through when a proxy rejects legacy SOURCE.

3. Check trusted XFF with the immediate peer actually used by the proxy, including address family; untrusted/malformed forwarding must fall back to the socket peer. XFF is inspection identity, not an authorization scope.

4. Verify native /api, HLS producer endpoints and health/metrics exposure remain private as intended. A public catch-all reverse proxy must not expose those routes.

5. Perform a representative functional listening session through this route, including metadata, relay recovery and HLS playback. Record actual user-visible interruptions/errors. Do not collect latency/throughput/RSS profiles or tune the stack in this phase.

Current supporting evidence: source review and automated route/XFF/authorization tests. No intended proxy or public TLS deployment is marked passed.

6. Drain, actual signals, restart and credential/configuration changes — PENDING operator session

Owned-process drain/restart rehearsal passed, 2026-10-08: tools/proof/drain-smoke.py ran isolated Quikcast origin/edge processes with disposable credentials, an external encoded-MP3 fixture sender, existing continuous listeners, a connected relay and published live HLS assets. All 54 functional checks passed against executable SHA-256 909aaef281d89d5728c14973c80d49741b31c06710eb3818ebbe0c4e99e16439. Drain was one-way/idempotent with a stable start time; readiness returned 503 while liveness and inspection remained available. Newly produced audio continued to both admitted listeners. New encoder/listener admissions, relay reconnect, HLS upload and publication returned 503. Existing HLS playlist and segment bytes remained readable. Controlled upstream EOF stopped the drained relay with server_draining; no retry occurred beyond the configured retry ceiling and the old relay listener ended.

Actual SIGTERM exited 0 and closed the remaining listener. Restart on the same ports accepted a new encoder and automatically reconnected the configured relay. Volatile HLS was initially absent (404); the external producer then registered, uploaded, republished and restored readable playlist/segment bytes. Actual SIGINT exited 0 and closed the restarted listener. All owned processes exited and threads joined; temporary configuration/credentials were removed. Evidence: evidence/manual-validation/drain-restart-2026-10-08/report.json. Reproduce with python3 tools/proof/drain-smoke.py --output evidence/manual-validation/drain-restart-repeat.

The user's running BUTT, relay, FFmpeg producer and HLS publisher were not changed. This establishes the owned-process drain/signal/restart contract; it does not independently validate GUI player behavior through shutdown, the intended proxy/TLS deployment, credential rotation or logger-loss handling. This executable differs from prior recorded binaries, so comprehensive build/test identity reconciliation remains part of the final acceptance review. No profiling, capacity test or optimization was performed.

1. Keep one encoder/listener, one relay/listener and HLS playback active. Invoke one-way drain: readiness becomes unavailable, liveness/API inspection stays available, new continuous admissions and HLS producer mutations reject, and already admitted playback follows the documented continuation rules.

2. Confirm a connected relay continues after drain, but once its upstream ends it stops without retry. Manual reconnect must reject. Administrative stop/disconnect stays available.

3. Send actual SIGINT/SIGTERM to the owned daemon; verify bounded exit, listener/driver termination and released listening ports. Repeat startup after clean shutdown; do not confuse drain with an automatic shutdown or resume API.

4. Change a selected limit/configuration or rotate a test credential by restart. Confirm new authentication applies, desired relay state comes from static configuration, counters/generations/listeners are process-scoped, and external HLS producers repopulate assets.

5. Confirm the draining log collector remains functional and bounded-loss diagnostics are visible. Record cleanup of sources/players/producers and owned test instances.

Current supporting evidence: automated cancellation/drain/resource ownership tests and real SIGTERM clean exits in the fresh media/HLS proofs. A combined intended-deployment session remains pending.

Failure handling and gate exit

Record each failure with the exact mandatory client, build, sanitized configuration, request/log evidence and a minimal reproducible case. First distinguish a correctness defect, documented unsupported contract, configuration error or an unmet required-client feature. Correctness fixes may proceed within freeze; new functionality requires a scope decision. Revalidate affected paths and freeze identity after changes.

The overall manual gate passes only when mandatory targets are named, applicable scenarios are PASS (or explicitly NOT REQUIRED), required-client failures are resolved, source/build evidence is current, cleanup is complete and an explicit acceptance review records satisfactory results. Never infer a pass from historical evidence, an unavailable tool or elapsed time.

PERFORMANCE OPTIMIZATION REMAINS BLOCKED UNTIL MANUAL VALIDATION IS SATISFACTORY.