Post-type filters, in-article voting, and visible model training

[mullinmax]

Four changes to the feed UI and what sits behind it, plus a follow-up commit answering what testing the branch turned up.

Post type filter

The "Post type" dropdown (all / text only / with media) becomes a set of checkboxes: Image, Video, Link, Text.

The types overlap on purpose - an illustrated article answers to both "image" and "text" - so ticking boxes widens the view rather than slicing it up, and every item matches at least one type (an item with no picture, no video and no body of its own is a "link"). /feed/items takes a post_types query parameter in place of text_only; unknown types are rejected with a 422.

Clearing every box now shows a "no articles match these filters" empty state with a Clear filters button, instead of the "add a source" prompt meant for a genuinely empty feed.

Video playback

Two real bugs in the extraction sidecar's rendition picker:

1. It discarded every format whose codecs the extractor did not state. yt-dlp writes "none" for a track a format genuinely lacks and leaves the field unset when the extractor simply did not say - reading "unset" as "missing" threw away every format on the many sites that report no codec information at all, so nothing was playable there.
2. It rejected HLS outright, which is all an increasing number of sites publish. That path could only ever return "no directly playable rendition".

The picker now keeps formats with unstated codecs while preferring ones that state both tracks (a stated rendition beats a taller one that might be silent), falls back to the HLS playlist when there is no progressive file, and reports is_hls so the client knows what it was handed. The player loads HLS through a vendored hls.js (light build, ~376 KB, fetched only when a stream actually needs it) and natively on Safari/iOS.

Alongside that, the media frame steps down instead of breaking:

• a dead thumbnail falls back to the item's own picture, then to a compact one-line play strip;
• a stream that cannot be resolved, or that the browser refuses once playing, leaves the title and a link to the page - never a dead play button on an empty grey box;
• broken images and videos elsewhere in a card take their frame with them.

playable_url moves into src/ytdlp/formats.py, a plain function of the info dict with no yt-dlp or FastAPI imports, and gets a unit test file that a new CI job runs.

Follow-up: resolving a video for playback no longer shares the ingest budget. A listing walk can afford 180s and several retries; a viewer waiting on a spinner cannot, and a request that outlives the reverse proxy comes back as an opaque gateway error that tells nobody anything. The resolve now gets 40s at the API, one retry and a 10s socket timeout in the extractor, and its failures are distinguished — 504 the site was too slow, 502 the service could not be reached, 422 nothing playable, 501 no extractor configured. Each is logged with the item and how long it took, and the card shows the reason rather than a blanket message (a gateway error carries no message of its own, so that one keeps the generic wording).

Voting from the open article

The three vote buttons now sit at the foot of the reader, so an article can be voted on without closing it first. The card and the reader share one vote row and are repainted together, and voting from the reader collapses the card behind it exactly as voting from the feed does. The reader stays open so you can finish reading.

Model training

Retraining moves to a background thread: POST /feed/rerank starts a run and returns the state it started from, and a new GET /feed/training_status reports where the run has got to. Progress lives in an in-memory registry (ranking/progress.py) keyed by user and feed, so the scheduled ranking job reports through the same path as a requested retrain.

The panel (renamed Model Training) shows when the models were last built (exact timestamp on hover), which model was chosen and how many votes it learned from, how many votes have been cast since — and, for a run in flight, a progress bar with the model currently being trained. Leaving the panel does not stop anything: a dot on the toolbar button stays lit until the run ends, the feed reloads itself when the new ranking lands, and a scheduled retrain lights the same indicator when a feed is opened.

Follow-up — training was doing far more work than it needed to. Every article carries a full embedding vector, and a run read the whole feed before doing anything, to learn from the few dozen labeled rows. That read is also why the progress bar appeared stuck on its first step: nothing could be reported until it finished.

• Training now loads only articles with a vote or a list membership. The rest are read only once a winner exists to score them with, so a feed with too few votes to rank never pays that cost.
• The prediction write was one UPDATE per article — a round trip each. It is now batched through execute_values.
• Random forests fit their trees across every core (n_jobs=-1). This is joblib parallelism over independent trees, not BLAS threading, so it does not fight the thread pinning the container sets; predictions are identical.

Measured on a feed of 8,000 articles with 150 votes: a full run goes from ~30s to ~19s, and the stretch before any progress could be reported goes from ~6.7s to ~0.05s.

Progress reporting got the detail to match: a phase (reading your votes / testing models / training the winner / scoring the feed), the model in hand, a note for what is happening inside the step ("fold 3 of 5", "scoring 8000 articles"), elapsed seconds, and a fractional step so the bar advances fold by fold through the slow models instead of freezing between them. Longest observed gap between updates is now ~2.4s.

Testing

• pytest tests in src/api: 576 passed. The 4 failures that remain locally (source_template x3, admin::test_get_version) fail identically on main in that environment — they need a live rss-bridge and BUILD_VERSION, and the CI test job is green.
• New tests: post-type filtering at the query and route level (including the overlap and coverage properties), the training progress registry, what a run loads and reports, the batched prediction write, the async retrain flow, and the rendition picker.
• Profiled against a seeded feed of 8,000 articles with 768-dim embeddings to get the numbers above.
• Driven in a real browser (Chromium) against a local instance: filter checkboxes, the filtered empty state, voting from the reader, the training panel through a full run, thumbnail failover, unplayable-stream failover, and hls.js loading and reporting fatal errors through to the failover.

🤖 Generated with Claude Code

https://claude.ai/code/session_01E5iKkj3ZLtHGTc8bz6R4yx