diff --git a/README.md b/README.md index ce365a83..85daf635 100644 --- a/README.md +++ b/README.md @@ -43,8 +43,9 @@ Opal replaces the stack you'd otherwise juggle β€” a player, a site for the show, a server front-end, a torrent client, a feed. Say what you want (a title, a -file, a magnet); it searches every source, plays anything, and remembers where -you left off. One native binary β€” [Zig](https://ziglang.org) + +file, a magnet); it searches supported, enabled providers, opens playable media +and reading results, and remembers where you left off. Personal libraries need +a configured server; catalog entries can lead to details or a source search. One native binary β€” [Zig](https://ziglang.org) + [dvui](https://github.com/david-vanderson/dvui) + **mpv** β€” fast and quiet.
@@ -250,7 +251,7 @@ XDG-compliant: One query fanned out across every source, ranked
πŸ”­ One search, every source
- Disk, torrents, Jellyfin, Plex, Stremio, anime, YouTube, live TV, TMDB, manga β€” one ranked, playable list. + Disk, torrents, Jellyfin, Plex, Stremio, anime, YouTube, live TV, TMDB, manga, novels and visual novels β€” one sourced list with playback, reading, or detail actions. @@ -275,7 +276,7 @@ XDG-compliant: | Instead of… | Opal gives you | |---|---| -| **Stremio / Kodi** + a pile of add-ons | one search across every source, a play button on each row | +| **Stremio / Kodi** + a pile of add-ons | one search across supported, enabled providers, with playback, reading, and detail actions | | **an IPTV / live-TV app** | ~40,000 live channels, searchable as you type | | **Jellyfin / Plex** web clients | your own media servers, browsed natively | | **Tachiyomi / Mihon** stuck on your phone | manga extensions on the desktop β€” server bundled, self-managed | diff --git a/docs/browse-sources.md b/docs/browse-sources.md index 5c91ee9f..334e79da 100644 --- a/docs/browse-sources.md +++ b/docs/browse-sources.md @@ -93,3 +93,121 @@ implemented. These remain research entries rather than working source claims. Validation for the expansion: native unit/browse suites and the headless build passed; the feature suite reported 463 passed, 0 failed, 1 warning and 11 skipped. The web lifecycle suite reported 29 passed. + +## Universal search coverage + +Loading improvements dated 2026-10-02. The validation results above describe +the earlier source expansion, not this set of changes. + +Universal search runs independent requests against the selected search sources. +It does not call a Browse tab's search function or replace that tab's query and +results. Source buttons show fetching, results, no matches, partial results, +transport/parse failures, or unavailable configuration separately. Results +already returned remain usable while slower sources finish. + +| Content | Universal search | Result action | +| --- | --- | --- | +| Movies and television | TMDB when configured, public Cinemeta otherwise; Cinemeta also backs up failed TMDB catalog requests | Show details or find playback sources; a metadata hit is not a playable stream | +| Torrents | Installed Nova2 indexes, YTS, configured Torznab/Prowlarr/Jackett, installed EZTV, cached RSS magnets | Resolve the listed magnet or source detail page | +| Anime | Installed anime connector plus torrent/Stremio providers | Open anime discovery; playback still requires an accessible source | +| Comics and manga | Installed comic connector and trusted executable source plugins | Open the comics reader | +| Novels | Independent Wikisource and Internet Archive readable-text searches, plus compatible installed executable plugins | Open the built-in reader by copied work identity | +| Visual novels | Independent public VNDB lookup with the existing cover filter | Open VNDB details; Opal does not run or download a visual novel from a metadata result | +| Drama | Movie/TV catalog searches include drama titles; playback uses the normal configured resolver | Catalog details or source discovery; no dedicated drama streaming scraper is built in | +| YouTube | Independent bounded yt-dlp search | Play the canonical video URL | +| Live television | Loaded IPTV catalog matched against the query | Play with the channel's required headers | +| Music, radio and podcasts | Independent public provider requests | Play audio or open a podcast's episode view | +| Personal video libraries | Connected Jellyfin and Plex server searches | Play the server item; credentials stay out of universal result URLs | +| Local files | Configured download directory | Play the existing local file | +| Installed source plugins | Trusted executable plugins implementing the search protocol | Open the supplied stream or resolve the plugin item | +| OPDS and Audiobookshelf | Their configured Browse libraries remain separate from universal query fan-out | Browse the connected library page | +| Web | URL navigation in the web browser, rather than a media catalog provider | Open a website | + +Catalog results retain real provider covers, descriptions, ratings and genres +when supplied. Empty fields remain unknown. VNDB's provider rating uses a +100-point scale; universal cards convert it to the shared ten-point scale. +Arbitrary executable-plugin scores are not converted into ratings because their +units are unspecified. Search caches include the enrichment and use a new +format key so older cache rows cannot decode as new results. + +Wikisource, Internet Archive and VNDB queries are bounded to twelve records per +provider; YouTube discovery requests ten records. These limits prevent a single +catalog from occupying every universal result slot. Browse pagination remains +the path for exploring a larger catalog. + +## Provider contract research for loading improvements + +The [Stremio catalog request contract](https://github.com/Stremio/stremio-addon-sdk/blob/master/docs/api/requests/defineCatalogHandler.md) +distinguishes search and catalog pagination, and its +[metadata contract](https://github.com/Stremio/stremio-addon-sdk/blob/master/docs/api/responses/meta.md) +defines optional artwork, descriptions, release dates and IMDb ratings. +Cinemeta responses provide those fields directly; a local HTTP inspection +returned 49 movie records with the expected metadata. This inspection verifies +the response shape, not playback of all listed titles. + +The [MediaWiki search API](https://www.mediawiki.org/wiki/API:Search) provides +Wikisource titles and snippets without a reader-side search mutation. The +[VNDB HTTPS API](https://api.vndb.org/kana) provides public visual-novel metadata +and image flags. Direct request inspections returned two Wikisource search +records and two VNDB metadata records. These inspections do not replace app +build checks or interactive verification. + +Failure handling now preserves existing movie/TV cards when refresh fails, +marks cache freshness after successful publication, and distinguishes rejected +Plex authentication from a genuine empty server search. Independent VNDB detail +requests preserve the Browse grid and discard superseded detail responses. +Upstream outages, access restrictions and absent server configuration remain +reported conditions; they cannot be repaired by adding an unverified mirror. + +## Loading and reader promise gaps addressed + +| Promise | Change | Practical limit | +| --- | --- | --- | +| Useful discovery results | Real covers, authors, years, snippets, summaries and ratings reach native and web views | A missing provider field stays empty | +| Keep browsing beyond the first page | Wikisource uses the server continuation; comics and audio advance by consumed provider rows | Fixed result buffers still bound a session | +| Read a manga series | MangaDex has a chapter picker and previous/next navigation; the web reader requests pages that actually finished downloading | 100 chapter rows per feed window, a client navigation bound of 10,000 rows, and 128 pages per chapter | +| Read novels without losing the selected work | Reader workers copy work identity; older requests cannot replace newer text or clear its loading state | 400 chapters and bounded text per work; truncated text is labelled | +| Reliable connected reading catalogs | OPDS checks HTTP success and complete Atom envelopes, decodes next links, deduplicates pages, and snapshots request credentials | Unprefixed OPDS 1.x Atom feeds; 300 entries and a 4 MiB response limit | +| Retain usable television catalogs | IPTV source replacement deletes and inserts in one transaction, rolling back failed replacements | Stream reachability still depends on the broadcaster | +| Accurate search actions | Catalog entries open details and reading entries open readers; queue actions require a usable playback identity | Metadata does not guarantee an accessible stream | +| Correct artwork while results change | Shared cover slots suppress images from superseded rows, including rows with no artwork | Failed images use a fallback | + +Provider contracts used for these changes include the official +[MangaDex OpenAPI](https://api.mangadex.org/docs/static/api.yaml), +[Internet Archive advanced search](https://archive.org/advancedsearch.php), +[OPDS 1.2 specification](https://github.com/opds-community/specs/blob/master/opds-1.2.md), +[Radio Browser API](https://api.radio-browser.info/), +[Audius API](https://docs.audius.org/api/), and +[Audiobookshelf API](https://api.audiobookshelf.org/). +Personal-server playback still requires verification against a configured server. + + +Connected Audiobookshelf searches use the upstream +[library search controller](https://github.com/advplyr/audiobookshelf/blob/master/server/controllers/LibraryController.js) +and [expanded book search response](https://github.com/advplyr/audiobookshelf/blob/master/server/utils/queries/libraryItemsBookFilters.js) +contracts: `GET /api/libraries/{id}/search?q=...&limit=6`, with `book` or +`podcast` matches wrapping `libraryItem` metadata. Up to sixteen permitted +libraries are searched concurrently; universal search surfaces at most twelve +items across them. Responses and each request are bounded. Authentication stays +in headers, and only stable item identities enter universal results. + +Opening an Audiobookshelf result fetches expanded file metadata rather than +passing its download archive to the media player. Native and web views offer +actual audio tracks or downloaded podcast episodes, with up to 128 selectable +files and explicit returned/total/truncation information. Single-file books can +start immediately and use book-wide server resume. Multi-file playback uses +manual track selection; automatic track advancement and book-wide resume across +files are not implemented. No connected personal-server search or playback was +verified during this provider-contract research. + +## Checks for the 2026-10-02 loading changes + +- Native `zig build` passed after integration, including the refresh retry cooldown. +- JavaScript syntax checks passed for `catalog.js`, `media.js` and `integrations.js`. +- Zig formatting and `git diff --check` passed. +- Regression tests and interactive playback checks were not run for these changes. +- Personal-server search/playback has not been verified against a configured account. + +Remaining functional gaps include independent OPDS server search, automatic +Audiobookshelf track advancement, and server resume across multi-file books. +Provider outages and access restrictions remain external availability conditions. diff --git a/engines/engines/animetosho.py b/engines/engines/animetosho.py new file mode 100644 index 00000000..c4c4d10f --- /dev/null +++ b/engines/engines/animetosho.py @@ -0,0 +1,69 @@ +# VERSION: 1.0 +# AUTHORS: Opal +"""AnimeTosho's documented JSON feed; actual torrent metadata, no DDL scraping. + +Contract: https://animetosho.org/about#feeds +Search uses q; swarm counts and total_size are provider fields, never guesses. +""" +import html +import json +import re +from urllib.parse import parse_qsl, unquote, urlencode, urlsplit + +from helpers import retrieve_url +from novaprinter import prettyPrinter + + +class animetosho: + url = 'https://feed.animetosho.org' + name = 'AnimeTosho' + supported_categories = {'all': '', 'movies': '', 'tv': ''} + + def search(self, what, cat='all'): + words = unquote(what).strip() + if not words or cat not in self.supported_categories: + return + data = json.loads(retrieve_url( + self.url.rstrip('/') + '/json?' + urlencode({'q': words}), + unescape_html_entities=False)) + if not isinstance(data, list): + return + seen = set() + for row in data[:100]: + if not isinstance(row, dict) or row.get('status') in ('deleted', 'hidden'): + continue + digest, title = row.get('info_hash'), row.get('title') + if (not isinstance(digest, str) or not re.fullmatch(r'[a-fA-F0-9]{40}', digest) + or not isinstance(title, str) or not title.strip()): + continue + digest = digest.lower() + if digest in seen: + continue + seen.add(digest) + title = ' '.join(html.unescape(title).split()).replace('|', ' ') + # Canonical hex BTIH deduplicates Nyaa and other indexes of the same + # release. Preserve the feed's tracker URLs without its base32 xt. + params = [('dn', title)] + magnet = row.get('magnet_uri', '') + if isinstance(magnet, str) and len(magnet) <= 4096 and magnet.startswith('magnet:?'): + for key, value in parse_qsl(urlsplit(magnet).query): + if key == 'tr' and value.startswith(('https://', 'http://', 'udp://')): + params.append((key, value)) + link = 'magnet:?xt=urn:btih:' + digest + '&' + urlencode(params) + detail = row.get('link', '') + if not isinstance(detail, str) or '\n' in detail or '\r' in detail: + detail = '' + host = urlsplit(detail).hostname or '' + if urlsplit(detail).scheme != 'https' or not (host == 'animetosho.org' or host.endswith('.animetosho.org')): + detail = '' + + def count(key): + try: + return max(-1, int(row.get(key, -1))) + except (TypeError, ValueError): + return -1 + + prettyPrinter({'link': link, 'name': title, + 'size': count('total_size'), 'seeds': count('seeders'), + 'leech': count('leechers'), 'pub_date': count('timestamp'), + 'engine_url': self.url, 'desc_link': detail}) diff --git a/site/src/pages/features.astro b/site/src/pages/features.astro index 1fef8f98..2bf4049f 100644 --- a/site/src/pages/features.astro +++ b/site/src/pages/features.astro @@ -6,7 +6,7 @@ const page = pageFor("/features/"); const groups = [ { title: "Find everything from one search", - body: "One query covers local files, torrents, Jellyfin, Plex, Stremio sources, anime, YouTube, live TV and manga β€” every result labelled with where it came from.", + body: "One query covers enabled search providers for local files, torrents, configured Jellyfin and Plex libraries, Stremio sources, anime, YouTube, live TV, manga, novels and visual novels. Results show their source and offer playback, reading or details.", }, { title: "Stream torrents like ordinary media", @@ -67,7 +67,7 @@ const groups = [

Universal search

One query, clearly sourced results

-

A query fans out to every enabled provider and comes back as a single ranked list, badged by source.

+

A query fans out to supported, enabled search providers and comes back as one list, badged by source. Reading-server catalogs have their own browse views.

Compare Opal with other media apps β†’
diff --git a/src/core/config.zig b/src/core/config.zig index 4f9b44b5..9bd63cf1 100644 --- a/src/core/config.zig +++ b/src/core/config.zig @@ -717,23 +717,17 @@ fn applyConfig(key: []const u8, val: []const u8) void { } else if (std.mem.eql(u8, key, "abs_connected")) { state.app.abs.connected = std.mem.eql(u8, val, "1") and state.app.abs.token_len > 0; } else if (std.mem.eql(u8, key, "opds_url")) { - if (val.len > 0 and val.len < state.app.opds.server_url.len) { - @memcpy(state.app.opds.server_url[0..val.len], val); - state.app.opds.server_url_len = val.len; - } - } else if (std.mem.eql(u8, key, "opds_user")) { - var restored_len: usize = 0; - if (val.len > 0 and loadSecretValue(key, val, state.app.opds.user_buf[0 .. state.app.opds.user_buf.len - 1], &restored_len)) { - state.app.opds.user_buf[restored_len] = 0; - } - } else if (std.mem.eql(u8, key, "opds_pass")) { + @import("../services/opds.zig").setConfiguredServer(val); + } else if (std.mem.eql(u8, key, "opds_user") or std.mem.eql(u8, key, "opds_pass")) { + var buffer: [128]u8 = undefined; + defer @memset(&buffer, 0); var restored_len: usize = 0; - if (val.len > 0 and loadSecretValue(key, val, state.app.opds.pass_buf[0 .. state.app.opds.pass_buf.len - 1], &restored_len)) { - state.app.opds.pass_buf[restored_len] = 0; + if (val.len > 0 and loadSecretValue(key, val, buffer[0 .. buffer.len - 1], &restored_len)) { + const opds = @import("../services/opds.zig"); + if (std.mem.eql(u8, key, "opds_user")) opds.setConfiguredUser(buffer[0..restored_len]) else opds.setConfiguredPassword(buffer[0..restored_len]); } } else if (std.mem.eql(u8, key, "opds_connected")) { - // Connected only if a catalog URL was also restored. - state.app.opds.connected = std.mem.eql(u8, val, "1") and state.app.opds.server_url_len > 0; + @import("../services/opds.zig").setConfiguredConnected(std.mem.eql(u8, val, "1")); } else if (std.mem.eql(u8, key, "win_x")) { state.app.win_x = std.fmt.parseInt(i32, val, 10) catch 0; state.app.win_restore_pending = true; @@ -852,7 +846,9 @@ fn applyConfig(key: []const u8, val: []const u8) void { } else if (std.mem.eql(u8, key, "search_sources")) { const resolver = @import("../services/resolver.zig"); const mask = std.fmt.parseInt(u16, val, 10) catch 0xFF; - resolver.source_mask.store(mask, .release); + // Legacy "all sources" predates the novel/VNDB search bits. Preserve + // custom filters while keeping the all-source preference inclusive. + resolver.source_mask.store(if (mask == 4095) resolver.ALL_SOURCE_BITS else mask, .release); } } diff --git a/src/core/source_config.zig b/src/core/source_config.zig index aa7ae390..fe4c9d8a 100644 --- a/src/core/source_config.zig +++ b/src/core/source_config.zig @@ -253,6 +253,28 @@ pub fn get(id: []const u8, field: []const u8) ?[]const u8 { return null; } +pub const FieldSnapshot = struct { + id: [32]u8 = std.mem.zeroes([32]u8), + id_len: usize = 0, + value: [512]u8 = std.mem.zeroes([512]u8), + value_len: usize = 0, +}; + +/// Copy installed endpoints while holding the reload lock. Detached requests +/// retain owned values rather than slices into a table that can be reloaded. +pub fn copyFields(field: []const u8, out: []FieldSnapshot) usize { + mutex.lock(); + defer mutex.unlock(); + var count: usize = 0; + for (entries[0..entry_count]) |entry| { + if (!std.mem.eql(u8, entry.field[0..entry.field_len], field)) continue; + if (count == out.len) break; + out[count] = .{ .id = entry.id, .id_len = entry.id_len, .value = entry.val, .value_len = entry.val_len }; + count += 1; + } + return count; +} + /// True if ANY source plugin is installed at all. False is the fresh-install / /// post-reset state (Opal ships neutral): every torrent/comics/anime engine is /// inert, so searches "run" but can't return source hits β€” surface that in the diff --git a/src/services/ai_chat.zig b/src/services/ai_chat.zig index d9029ad6..18eb6825 100644 --- a/src/services/ai_chat.zig +++ b/src/services/ai_chat.zig @@ -1078,6 +1078,9 @@ pub fn renderInlineResults() void { .music => "Music", .radio => "Radio", .podcast => "Podcast", + .novels => "Novels", + .vndb => "Visual novels", + .audiobooks => "Audiobooks", }; } }.get; @@ -1179,7 +1182,7 @@ pub fn renderInlineResults() void { } // Queue β€” borderless quiet action. - if (dvui.buttonIcon(@src(), "", icons.tvg.lucide.plus, .{}, .{}, .{ + if (resolver.isRemoteQueueable(item) and dvui.buttonIcon(@src(), "", icons.tvg.lucide.plus, .{}, .{}, .{ .id_extra = ci + 10400, .color_fill = dvui.Color{ .r = 0, .g = 0, .b = 0, .a = 0 }, .color_text = theme.colors.text_secondary, @@ -1193,8 +1196,13 @@ pub fn renderInlineResults() void { queueChatResult(ci); } - // Play β€” accent action. - if (dvui.buttonIcon(@src(), "", icons.tvg.lucide.play, .{}, .{}, .{ + // Catalogs open details; reading results open a reader. + const action_icon = switch (item.source) { + .tmdb, .vndb, .podcast, .audiobooks => icons.tvg.lucide.info, + .comics, .novels => icons.tvg.lucide.@"book-open", + else => icons.tvg.lucide.play, + }; + if (dvui.buttonIcon(@src(), "", action_icon, .{}, .{}, .{ .id_extra = ci + 10100, .color_fill = dvui.Color{ .r = 0, .g = 0, .b = 0, .a = 0 }, .color_text = theme.colors.accent, @@ -1223,6 +1231,7 @@ pub fn playChatResult(idx: usize) void { fn queueChatResult(idx: usize) void { if (idx >= chat_result_count) return; const item = &chat_results[idx]; + if (!resolver.isRemoteQueueable(item)) return; const name = item.name[0..item.name_len]; const url_str = item.url[0..item.url_len]; const src_label = switch (item.source) { @@ -1240,6 +1249,9 @@ fn queueChatResult(idx: usize) void { .music => "music", .radio => "radio", .podcast => "podcast", + .novels => "novels", + .vndb => "vndb", + .audiobooks => "audiobooks", }; @import("queue.zig").addToQueue(url_str, name, src_label); state.showToast("Added to queue"); diff --git a/src/services/anime.zig b/src/services/anime.zig index 567ea2af..80dce494 100644 --- a/src/services/anime.zig +++ b/src/services/anime.zig @@ -1216,6 +1216,9 @@ fn parseJikanDataEx(json: []const u8, my_gen: u32, with_broadcast: bool, start_o const obj_open = std.mem.lastIndexOfScalar(u8, json[0..pos], '{') orelse break; const obj_end = anime_pure.jsonObjectEnd(json, obj_open) orelse break; const obj_slice = json[pos..obj_end]; + // Advance before any rejection: producer/genre objects contain their + // own mal_id fields and must never become cards when a row is skipped. + pos = obj_end; // NSFW filter (Settings β€Ί Behavior): drop adult ratings and explicit // Ecchi/Erotica/Hentai genres, including cached responses. diff --git a/src/services/audiobookshelf.zig b/src/services/audiobookshelf.zig index d569ade3..6ca5ba59 100644 --- a/src/services/audiobookshelf.zig +++ b/src/services/audiobookshelf.zig @@ -33,6 +33,52 @@ const alloc = @import("../core/alloc.zig").allocator; // reads it each frame. is_loading (atomic) only gates re-spawns. var parse_mutex: @import("../core/sync.zig").Mutex = .{}; +pub const ConnectionSnapshot = struct { + connected: bool, + server: [256]u8, + server_len: usize, + token: [256]u8, + token_len: usize, + libraries: [16]pure.Library, + library_count: usize, +}; +/// Immutable credentials/library identities for independent universal searches. +pub fn connectionSnapshot() ConnectionSnapshot { + parse_mutex.lock(); + defer parse_mutex.unlock(); + return .{ .connected = state.app.abs.connected, .server = state.app.abs.server_url, .server_len = @min(state.app.abs.server_url_len, state.app.abs.server_url.len), .token = state.app.abs.token, .token_len = @min(state.app.abs.token_len, state.app.abs.token.len), .libraries = state.app.abs.libraries, .library_count = @min(state.app.abs.library_count, state.app.abs.libraries.len) }; +} +pub fn setServerUrl(server: []const u8) void { + parse_mutex.lock(); + defer parse_mutex.unlock(); + @import("../core/text.zig").setFixedUtf8(&state.app.abs.server_url, &state.app.abs.server_url_len, server); +} + +pub const CatalogSnapshot = struct { + connected: bool, + loading: bool, + loading_more: bool, + has_more: bool, + view: []const u8, + book_count: usize, + library_count: usize, + server: [256]u8, + server_len: usize, + library: [96]u8, + library_len: usize, + error_text: [128]u8, + error_len: usize, +}; +pub fn catalogSnapshot(books: []pure.Book, libraries: []pure.Library) CatalogSnapshot { + parse_mutex.lock(); + defer parse_mutex.unlock(); + const bn = @min(state.app.abs.book_count, @min(books.len, state.app.abs.books.len)); + const ln = @min(state.app.abs.library_count, @min(libraries.len, state.app.abs.libraries.len)); + @memcpy(books[0..bn], state.app.abs.books[0..bn]); + @memcpy(libraries[0..ln], state.app.abs.libraries[0..ln]); + return .{ .connected = state.app.abs.connected, .loading = state.app.abs.is_loading.load(.acquire), .loading_more = loading_more.load(.acquire), .has_more = more_available and bn < state.app.abs.books.len, .view = @tagName(state.app.abs.view), .book_count = bn, .library_count = ln, .server = state.app.abs.server_url, .server_len = @min(state.app.abs.server_url_len, state.app.abs.server_url.len), .library = state.app.abs.selected_lib_name, .library_len = @min(state.app.abs.selected_lib_name_len, state.app.abs.selected_lib_name.len), .error_text = state.app.abs.login_error, .error_len = @min(state.app.abs.login_error_len, state.app.abs.login_error.len) }; +} + // ── Server-side resume state ──────────────────────────────────────────────── // playBook() streams a book immediately, then a detached worker fetches the // server's saved position; tick() (frame loop, UI thread) issues the seek once @@ -61,6 +107,7 @@ var resume_title_len: usize = 0; // render-time reads, which tolerate one frame of staleness like book_count β€” // stay consistent. var current_page: u32 = 0; +var library_generation = std.atomic.Value(u32).init(0); var more_available: bool = true; var loading_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(false); @@ -172,11 +219,13 @@ pub fn authenticate() void { return; }; + parse_mutex.lock(); const tlen = @min(token.len, state.app.abs.token.len); @memcpy(state.app.abs.token[0..tlen], token[0..tlen]); state.app.abs.token_len = tlen; state.app.abs.connected = true; state.app.abs.view = .Libraries; + parse_mutex.unlock(); state.markConfigDirty(); fetchLibrariesSync(); @@ -212,12 +261,19 @@ fn fetchLibrariesSync() void { var url_buf: [512]u8 = undefined; const url = std.fmt.bufPrint(&url_buf, "{s}/api/libraries", .{server}) catch return; - const body = absGet(url) orelse return; + const body = absGet(url) orelse { + if (state.app.abs.connected) setLoginError("Failed to load libraries β€” check your server connection"); + return; + }; defer alloc.free(body); parse_mutex.lock(); defer parse_mutex.unlock(); - state.app.abs.library_count = pure.parseLibraries(body, &state.app.abs.libraries); + state.app.abs.login_error_len = 0; + state.app.abs.library_count = pure.parseLibraryPage(alloc, body, &state.app.abs.libraries) orelse { + setLoginError("Invalid library response β€” check your server connection"); + return; + }; logs.pushLog("info", "audiobookshelf", "Libraries loaded", false); } @@ -226,6 +282,8 @@ pub fn openLibrary(idx: usize) void { if (idx >= state.app.abs.library_count) return; if (state.app.abs.is_loading.load(.acquire) or !state.app.abs.connected) return; + state.app.abs.login_error_len = 0; + const generation = library_generation.fetchAdd(1, .acq_rel) +% 1; const lib = &state.app.abs.libraries[idx]; const ilen = @min(lib.id_len, state.app.abs.selected_lib_id.len); @memcpy(state.app.abs.selected_lib_id[0..ilen], lib.id[0..ilen]); @@ -243,25 +301,40 @@ pub fn openLibrary(idx: usize) void { more_available = true; state.app.abs.thread = @import("../core/workers.zig").spawnLegacy(struct { - fn worker() void { - defer state.app.abs.is_loading.store(false, .release); - const server = state.app.abs.server_url[0..state.app.abs.server_url_len]; - const lib_id = state.app.abs.selected_lib_id[0..state.app.abs.selected_lib_id_len]; + fn worker(gen: u32, lib_buf: [64]u8, lib_len: usize) void { + defer if (library_generation.load(.acquire) == gen) state.app.abs.is_loading.store(false, .release); + var server_buf: [256]u8 = undefined; + const server_len = @min(state.app.abs.server_url_len, server_buf.len); + @memcpy(server_buf[0..server_len], state.app.abs.server_url[0..server_len]); + const server = server_buf[0..server_len]; + const lib_id = lib_buf[0..lib_len]; var url_buf: [640]u8 = undefined; const url = pure.libraryItemsUrl(server, lib_id, ABS_PAGE_LIMIT, 0, &url_buf) orelse return; - const body = absGet(url) orelse return; + const body = absGet(url) orelse { + if (library_generation.load(.acquire) == gen and state.app.abs.connected) { + setLoginError("Failed to load library β€” check your server connection"); + more_available = false; + } + return; + }; defer alloc.free(body); parse_mutex.lock(); defer parse_mutex.unlock(); - const n = pure.parseItems(body, &state.app.abs.books); + if (library_generation.load(.acquire) != gen or !state.app.abs.connected or state.app.abs.view != .Books) return; + const page = pure.parseItemPage(alloc, body, &state.app.abs.books) orelse { + setLoginError("Invalid library response β€” retry this library"); + more_available = false; + return; + }; + const n = page.count; state.app.abs.book_count = n; - if (n < ABS_PAGE_LIMIT or n >= state.app.abs.books.len) more_available = false; + more_available = page.consumed >= ABS_PAGE_LIMIT and n < state.app.abs.books.len; logs.pushLog("info", "audiobookshelf", "Books loaded", false); } - }.worker, .{}) catch blk: { + }.worker, .{ generation, state.app.abs.selected_lib_id, state.app.abs.selected_lib_id_len }) catch blk: { state.app.abs.is_loading.store(false, .release); break :blk null; }; @@ -269,6 +342,10 @@ pub fn openLibrary(idx: usize) void { } pub fn goToLibraries() void { + clearAudioSelection(); + _ = library_generation.fetchAdd(1, .acq_rel); + state.app.abs.is_loading.store(false, .release); + more_available = false; state.app.abs.view = .Libraries; state.app.abs.book_count = 0; } @@ -282,7 +359,7 @@ pub fn loadMore() void { if (!more_available) return; if (state.app.abs.is_loading.load(.acquire)) return; if (loading_more.load(.acquire)) return; - if (state.app.abs.book_count == 0) return; + if (state.app.abs.book_count >= state.app.abs.books.len) { more_available = false; return; @@ -299,14 +376,14 @@ pub fn loadMore() void { @memcpy(lib_id_buf[0..lib_id_len], state.app.abs.selected_lib_id[0..lib_id_len]); const next_page = current_page + 1; - if (@import("../core/workers.zig").spawnLegacy(loadMoreWorker, .{ lib_id_buf, lib_id_len, next_page })) |t| { + if (@import("../core/workers.zig").spawnLegacy(loadMoreWorker, .{ lib_id_buf, lib_id_len, next_page, library_generation.load(.acquire) })) |t| { @import("../core/workers.zig").release(t); } else |_| { loading_more.store(false, .release); } } -fn loadMoreWorker(lib_id_buf: [64]u8, lib_id_len: usize, page: u32) void { +fn loadMoreWorker(lib_id_buf: [64]u8, lib_id_len: usize, page: u32, generation: u32) void { defer loading_more.store(false, .release); const lib_id = lib_id_buf[0..lib_id_len]; @@ -321,14 +398,27 @@ fn loadMoreWorker(lib_id_buf: [64]u8, lib_id_len: usize, page: u32) void { var url_buf: [640]u8 = undefined; const url = pure.libraryItemsUrl(server, lib_id, ABS_PAGE_LIMIT, page, &url_buf) orelse return; - const body = absGet(url) orelse return; + const body = absGet(url) orelse { + if (library_generation.load(.acquire) == generation and state.app.abs.connected) { + setLoginError("Failed to load more books β€” reopen the library to retry"); + more_available = false; + } + return; + }; defer alloc.free(body); // Parse into a heap staging buffer β€” never a big stack buffer on a // spawned thread (CLAUDE.md) β€” before publishing under the lock. const items = alloc.alloc(pure.Book, ABS_PAGE_LIMIT) catch return; defer alloc.free(items); - const n = pure.parseItems(body, items); + const parsed_page = pure.parseItemPage(alloc, body, items) orelse { + if (library_generation.load(.acquire) == generation) { + setLoginError("Invalid library page β€” reopen the library to retry"); + more_available = false; + } + return; + }; + const n = parsed_page.count; parse_mutex.lock(); defer parse_mutex.unlock(); @@ -336,7 +426,8 @@ fn loadMoreWorker(lib_id_buf: [64]u8, lib_id_len: usize, page: u32) void { // The user may have switched (or left) the library while this page was in // flight β€” drop it rather than append a stale library's books onto // whatever is now shown. - if (state.app.abs.selected_lib_id_len != lib_id_len or + if (library_generation.load(.acquire) != generation or !state.app.abs.connected or state.app.abs.view != .Books or + state.app.abs.selected_lib_id_len != lib_id_len or !std.mem.eql(u8, state.app.abs.selected_lib_id[0..lib_id_len], lib_id)) { return; @@ -350,7 +441,7 @@ fn loadMoreWorker(lib_id_buf: [64]u8, lib_id_len: usize, page: u32) void { } state.app.abs.book_count = base + written; current_page = page; - if (n < ABS_PAGE_LIMIT or state.app.abs.book_count >= cap) { + if (parsed_page.consumed < ABS_PAGE_LIMIT or state.app.abs.book_count >= cap) { more_available = false; } @@ -363,113 +454,416 @@ fn loadMoreWorker(lib_id_buf: [64]u8, lib_id_len: usize, page: u32) void { // Playback // ══════════════════════════════════════════════════════════ -/// Stream book `idx`'s audio into mpv. Builds the token-authed stream URL and -/// hands it to browser.loadContentDirectMeta, which creates a player if needed, -/// load_file's the URL, attaches now-playing metadata (title/author/cover), and -/// gotoPlayer()s β€” so the macOS Now Playing card is populated automatically. +const AudioJob = struct { + generation: u32 = 0, + requested_episode: [64]u8 = std.mem.zeroes([64]u8), + requested_episode_len: usize = 0, + id: [64]u8 = std.mem.zeroes([64]u8), + id_len: usize = 0, + title: [256]u8 = std.mem.zeroes([256]u8), + title_len: usize = 0, + author: [160]u8 = std.mem.zeroes([160]u8), + author_len: usize = 0, + server: [256]u8 = std.mem.zeroes([256]u8), + server_len: usize = 0, + token: [256]u8 = std.mem.zeroes([256]u8), + token_len: usize = 0, +}; +const AUDIO_CAP: usize = 128; +var audio_mutex: @import("../core/sync.zig").Mutex = .{}; +var audio_generation = std.atomic.Value(u32).init(0); +var audio_loading = std.atomic.Value(bool).init(false); +var audio_job: AudioJob = .{}; +var audio_tracks: [AUDIO_CAP]pure.AudioTrack = [_]pure.AudioTrack{.{}} ** AUDIO_CAP; +var audio_count: usize = 0; +var audio_total: usize = 0; +var audio_visible: bool = false; +var audio_requested: ?usize = null; +var audio_resume: pure.TrackPosition = .{ .index = 0, .seconds = 0 }; +var audio_request_seconds: f64 = 0; +var audio_complete_book: bool = false; +const ActiveAudio = struct { + job: AudioJob, + count: usize, + index: usize, + complete_book: bool, + url: [2048]u8 = std.mem.zeroes([2048]u8), + url_len: usize = 0, + player_serial: ?u64 = null, + opened: bool = false, + last_position: f64 = 0, + last_sync_ms: i64 = 0, +}; +// Frame-thread owned playback plan. Browsing/hiding a selector cannot rewrite it. +var active_audio: ?ActiveAudio = null; +var active_tracks: [AUDIO_CAP]pure.AudioTrack = [_]pure.AudioTrack{.{}} ** AUDIO_CAP; + +pub const AudioSnapshot = struct { + visible: bool, + complete_book: bool, + loading: bool, + count: usize, + total: usize, + generation: u32, + title: [256]u8, + title_len: usize, +}; +/// Snapshot for presenting tracks. Callers serialize only titles/episode flags. +pub fn audioSnapshot(out: []pure.AudioTrack) AudioSnapshot { + audio_mutex.lock(); + defer audio_mutex.unlock(); + const count = @min(audio_count, out.len); + @memcpy(out[0..count], audio_tracks[0..count]); + return .{ .visible = audio_visible, .complete_book = audio_complete_book, .loading = audio_loading.load(.acquire), .count = count, .total = audio_total, .generation = audio_job.generation, .title = audio_job.title, .title_len = audio_job.title_len }; +} +pub fn clearAudioSelection() void { + audio_mutex.lock(); + defer audio_mutex.unlock(); + _ = audio_generation.fetchAdd(1, .acq_rel); + audio_visible = false; + audio_count = 0; + audio_total = 0; + audio_complete_book = false; + audio_requested = null; + @memset(&audio_job.token, 0); + audio_job.token_len = 0; + audio_loading.store(false, .release); +} pub fn playBook(idx: usize) void { - if (idx >= state.app.abs.book_count) return; - const server = state.app.abs.server_url[0..state.app.abs.server_url_len]; - const token = state.app.abs.token[0..state.app.abs.token_len]; - if (server.len == 0 or token.len == 0) return; - - // Snapshot the book's fields into locals BEFORE the play call β€” a concurrent - // refetch can overwrite books[] mid-frame. - const b = &state.app.abs.books[idx]; - var id_buf: [64]u8 = undefined; - const idlen = @min(b.id_len, id_buf.len); - @memcpy(id_buf[0..idlen], b.id[0..idlen]); - var title_buf: [256]u8 = undefined; - const tlen = @min(b.title_len, title_buf.len); - @memcpy(title_buf[0..tlen], b.title[0..tlen]); - var author_buf: [160]u8 = undefined; - const alen = @min(b.author_len, author_buf.len); - @memcpy(author_buf[0..alen], b.author[0..alen]); - - var url_buf: [1024]u8 = undefined; - const url = pure.streamUrl(server, id_buf[0..idlen], token, &url_buf) orelse { - state.showToast("Cannot play β€” invalid item id"); + parse_mutex.lock(); + if (idx >= state.app.abs.book_count) { + parse_mutex.unlock(); return; - }; - - var cover_buf: [1024]u8 = undefined; - const cover = pure.coverUrl(server, id_buf[0..idlen], token, &cover_buf) orelse ""; - - // ── Arm server-side resume BEFORE the load ── - // Record which book tick() should recognise as loaded, reset the target, and - // kick the async progress fetch. tick() applies the seek once mpv reports a - // duration (file open) β€” seeking earlier is a no-op. Order matters: the book - // id must be published before loadContentDirectMeta so the id-match gate in - // tick() (and the fetch worker's publish guard) sees this play, not a stale one. - resume_mutex.lock(); - resume_target_secs = 0; - const rlen = @min(idlen, resume_item_id.len); - @memcpy(resume_item_id[0..rlen], id_buf[0..rlen]); - resume_item_id_len = rlen; - // Display snapshot for the unified library row. Only stable item identity - // is persisted; token-bearing stream/cover URLs stay in this call frame. - const rt = @min(tlen, resume_title.len); - @memcpy(resume_title[0..rt], title_buf[0..rt]); - resume_title_len = rt; - resume_mutex.unlock(); - resume_decided.store(false, .release); - resume_pending.store(true, .release); - ResumeFetch.spawn(); - - var deep_buf: [128]u8 = undefined; - const deep = std.fmt.bufPrint(&deep_buf, "opal://audiobookshelf/{s}", .{id_buf[0..idlen]}) catch return; - @import("browser.zig").playDirect(.{ - .url = url, - .history_identity = deep, - .restore_target = deep, - .art_url = cover, - .title = title_buf[0..tlen], - .subtitle = author_buf[0..alen], - }); - logs.pushLog("info", "audiobookshelf", "Streaming audiobook", false); + } + const book = state.app.abs.books[idx]; + parse_mutex.unlock(); + playBookById(book.id[0..book.id_len], book.title[0..book.title_len], book.author[0..book.author_len]); } - -/// Reopen a persisted audiobook by stable item id. The current token is added -/// only at the playback edge and never enters the unified library database. +/// Resolve actual audio files before playback; `/download` can be a zip archive. pub fn playBookById(id: []const u8, title: []const u8, author: []const u8) void { - const server = state.app.abs.server_url[0..state.app.abs.server_url_len]; - const token = state.app.abs.token[0..state.app.abs.token_len]; - if (server.len == 0 or token.len == 0) { - state.showToast("Connect Audiobookshelf to resume"); + const connection = connectionSnapshot(); + const slash = std.mem.indexOfScalar(u8, id, '/'); + const item_id = if (slash) |index| id[0..index] else id; + const episode_id = if (slash) |index| id[index + 1 ..] else ""; + if (!connection.connected or !pure.validItemId(item_id) or item_id.len > 64 or + (episode_id.len > 0 and (!pure.validItemId(episode_id) or episode_id.len > 64))) + { + state.showToast("Connect Audiobookshelf to play this item"); return; } - var url_buf: [1024]u8 = undefined; - const url = pure.streamUrl(server, id, token, &url_buf) orelse { - state.showToast("Cannot play this audiobook"); + var job: AudioJob = .{}; + @import("../core/text.zig").setFixedUtf8(&job.id, &job.id_len, item_id); + @import("../core/text.zig").setFixedUtf8(&job.requested_episode, &job.requested_episode_len, episode_id); + @import("../core/text.zig").setFixedUtf8(&job.title, &job.title_len, title); + @import("../core/text.zig").setFixedUtf8(&job.author, &job.author_len, author); + job.server_len = connection.server_len; + @memcpy(job.server[0..job.server_len], connection.server[0..job.server_len]); + job.token_len = connection.token_len; + @memcpy(job.token[0..job.token_len], connection.token[0..job.token_len]); + if (job.server_len == 0 or job.token_len == 0) return; + audio_mutex.lock(); + job.generation = audio_generation.fetchAdd(1, .acq_rel) +% 1; + audio_job = job; + audio_count = 0; + audio_total = 0; + audio_requested = null; + audio_visible = true; + audio_loading.store(true, .release); + audio_mutex.unlock(); + state.navigateToTab(.Audiobooks); + if (@import("../core/workers.zig").spawnLegacy(loadAudioWorker, .{job})) |thread| { + @import("../core/workers.zig").release(thread); + } else |_| { + publishAudioFailure(job.generation, "Could not start audio lookup β€” select the item again"); + finishAudioJob(job.generation); + } +} +fn finishAudioJob(generation: u32) void { + audio_mutex.lock(); + defer audio_mutex.unlock(); + if (audio_generation.load(.acquire) == generation) { + audio_loading.store(false, .release); + state.wakeUi(); + } +} +fn publishAudioFailure(generation: u32, message: []const u8) void { + audio_mutex.lock(); + defer audio_mutex.unlock(); + if (audio_generation.load(.acquire) == generation) setLoginError(message); +} +fn loadAudioWorker(job: AudioJob) void { + defer finishAudioJob(job.generation); + var url_buf: [640]u8 = undefined; + const url = std.fmt.bufPrint(&url_buf, "{s}/api/items/{s}?expanded=1", .{ std.mem.trimEnd(u8, job.server[0..job.server_len], "/"), job.id[0..job.id_len] }) catch return; + var auth_buf: [320]u8 = undefined; + const auth = pure.bearerHeader(job.token[0..job.token_len], &auth_buf) orelse return; + const body_buf = alloc.alloc(u8, 3 * 1024 * 1024) catch return; + defer alloc.free(body_buf); + const body = http.fetch(url, body_buf, .{ .accept = "application/json", .auth_header = auth, .timeout_secs = 15, .max_response = body_buf.len }) orelse { + publishAudioFailure(job.generation, "Cannot load audio files β€” check server access"); + return; + }; + const staged = alloc.alloc(pure.AudioTrack, AUDIO_CAP) catch return; + defer alloc.free(staged); + const page = pure.parseAudioTracks(alloc, body, job.id[0..job.id_len], staged) orelse { + publishAudioFailure(job.generation, "This item has no playable audio files"); + return; + }; + const complete = pure.completeBookTimeline(staged[0..page.count], page.total); + var book_resume: pure.TrackPosition = .{ .index = 0, .seconds = 0 }; + if (complete) { + var progress_url_buf: [640]u8 = undefined; + if (pure.progressUrl(job.server[0..job.server_len], job.id[0..job.id_len], "", &progress_url_buf)) |progress_url| { + var progress_buf: [16384]u8 = undefined; + if (http.fetch(progress_url, &progress_buf, .{ .accept = "application/json", .auth_header = auth, .timeout_secs = 8 })) |progress_body| { + const info = pure.parseProgressValue(alloc, progress_body); + const last = staged[page.count - 1]; + const target = pure.resumeTarget(info.current_time, last.start_offset + last.duration, info.is_finished) orelse 0; + book_resume = pure.locateBookPosition(staged[0..page.count], target) orelse book_resume; + } + } + } + audio_mutex.lock(); + defer audio_mutex.unlock(); + if (audio_generation.load(.acquire) != job.generation or !state.app.abs.connected) return; + @memcpy(audio_tracks[0..page.count], staged[0..page.count]); + audio_count = page.count; + audio_total = page.total; + audio_complete_book = complete; + audio_resume = book_resume; + if (job.requested_episode_len > 0) { + for (staged[0..page.count], 0..) |track, idx| { + if (std.mem.eql(u8, track.episode_id[0..track.episode_id_len], job.requested_episode[0..job.requested_episode_len])) { + audio_requested = idx; + audio_request_seconds = 0; + break; + } + } + } else if (complete) { + audio_requested = book_resume.index; + audio_request_seconds = book_resume.seconds; + } else if (page.count == 1 and page.total == 1) { + audio_requested = 0; + audio_request_seconds = 0; + } + if (page.count == 0) setLoginError("This item has no playable audio files"); + state.wakeUi(); +} +/// Requests playback on the frame thread, including remote/API callers. +pub fn playAudioTrack(idx: usize, generation: u32) bool { + audio_mutex.lock(); + defer audio_mutex.unlock(); + if (generation != audio_job.generation or idx >= audio_count) return false; + audio_requested = idx; + audio_request_seconds = 0; + state.wakeUi(); + return true; +} +pub fn playWholeBook(generation: u32) bool { + audio_mutex.lock(); + defer audio_mutex.unlock(); + if (generation != audio_job.generation or !audio_complete_book) return false; + audio_requested = audio_resume.index; + audio_request_seconds = audio_resume.seconds; + state.wakeUi(); + return true; +} +fn playPendingAudioTrack() void { + audio_mutex.lock(); + const idx = audio_requested orelse { + audio_mutex.unlock(); + return; + }; + audio_requested = null; + if (idx >= audio_count) { + audio_mutex.unlock(); + return; + } + const job = audio_job; + const count = audio_count; + const complete = audio_complete_book; + const seconds = audio_request_seconds; + if (audio_generation.load(.acquire) != job.generation or !state.app.abs.connected) { + audio_mutex.unlock(); + return; + } + if (active_audio) |prior| enqueueProgress(prior, false); + @memcpy(active_tracks[0..count], audio_tracks[0..count]); + active_audio = .{ .job = job, .count = count, .index = idx, .complete_book = complete }; + audio_mutex.unlock(); + loadActiveAudio(seconds); +} +fn loadActiveAudio(seconds: f64) void { + if (active_audio == null) return; + const plan = &active_audio.?; + const track = active_tracks[plan.index]; + var url_buf: [2048]u8 = undefined; + const job = plan.job; + const url = pure.audioTrackUrl(job.server[0..job.server_len], track.content_path[0..track.content_path_len], job.token[0..job.token_len], &url_buf) orelse { + state.showToast("This audio file URL cannot be played"); + active_audio = null; return; }; + @memcpy(plan.url[0..url.len], url); + plan.url_len = url.len; + plan.opened = false; + plan.player_serial = null; + plan.last_position = @max(0, seconds); + plan.last_sync_ms = @import("../core/io_global.zig").monotonicMilliTimestamp(); var cover_buf: [1024]u8 = undefined; - const cover = pure.coverUrl(server, id, token, &cover_buf) orelse ""; - - resume_mutex.lock(); - resume_target_secs = 0; - const rlen = @min(id.len, resume_item_id.len); - @memcpy(resume_item_id[0..rlen], id[0..rlen]); - resume_item_id_len = rlen; - const rt = @min(title.len, resume_title.len); - @memcpy(resume_title[0..rt], title[0..rt]); - resume_title_len = rt; - resume_mutex.unlock(); - resume_decided.store(false, .release); - resume_pending.store(true, .release); - ResumeFetch.spawn(); - - var deep_buf: [128]u8 = undefined; - const deep = std.fmt.bufPrint(&deep_buf, "opal://audiobookshelf/{s}", .{id}) catch return; - @import("browser.zig").playDirect(.{ - .url = url, - .history_identity = deep, - .restore_target = deep, - .art_url = cover, - .title = title, - .subtitle = author, - }); - logs.pushLog("info", "audiobookshelf", "Streaming audiobook", false); + const cover = pure.coverUrl(job.server[0..job.server_len], job.id[0..job.id_len], job.token[0..job.token_len], &cover_buf) orelse ""; + // Server resume was resolved to this file's local seconds before the load. + resume_pending.store(false, .release); + var deep_buf: [192]u8 = undefined; + const deep = if (track.episode) + std.fmt.bufPrint(&deep_buf, "opal://audiobookshelf/{s}/{s}", .{ job.id[0..job.id_len], track.episode_id[0..track.episode_id_len] }) catch return + else + std.fmt.bufPrint(&deep_buf, "opal://audiobookshelf/{s}", .{job.id[0..job.id_len]}) catch return; + @import("browser.zig").playDirect(.{ .url = url, .history_identity = deep, .restore_target = deep, .resume_position_secs = @max(0, seconds), .art_url = cover, .title = if (track.episode) track.title[0..track.title_len] else job.title[0..job.title_len], .subtitle = job.author[0..job.author_len] }); +} +const ProgressJob = struct { + job: AudioJob, + episode: [64]u8 = std.mem.zeroes([64]u8), + episode_len: usize = 0, + position: f64, + duration: f64, + finished: bool, +}; +var progress_mutex: @import("../core/sync.zig").Mutex = .{}; +var progress_jobs: [8]ProgressJob = undefined; +var progress_head: usize = 0; +var progress_count: usize = 0; +var progress_worker: bool = false; +fn enqueueProgress(plan: ActiveAudio, finished: bool) void { + const track = active_tracks[plan.index]; + if (!plan.complete_book and !track.episode) return; // partial books cannot truthfully sync file-local time as book progress + const duration = if (plan.complete_book) active_tracks[plan.count - 1].start_offset + active_tracks[plan.count - 1].duration else track.duration; + const position = if (plan.complete_book) pure.bookPosition(track, plan.last_position) orelse return else @min(plan.last_position, duration); + if (duration <= 0) return; + var request: ProgressJob = .{ .job = plan.job, .position = position, .duration = duration, .finished = finished }; + if (track.episode) { + request.episode = track.episode_id; + request.episode_len = track.episode_id_len; + } + progress_mutex.lock(); + if (progress_count == progress_jobs.len) { + // Keep the newest checkpoint without spawning parallel PATCH writers. + progress_jobs[(progress_head + progress_count - 1) % progress_jobs.len] = request; + } else { + progress_jobs[(progress_head + progress_count) % progress_jobs.len] = request; + progress_count += 1; + } + if (!progress_worker) { + progress_worker = true; + if (@import("../core/workers.zig").spawnLegacy(progressWorker, .{})) |thread| @import("../core/workers.zig").release(thread) else |_| progress_worker = false; + } + progress_mutex.unlock(); +} +fn progressWorker() void { + while (true) { + progress_mutex.lock(); + if (progress_count == 0) { + progress_worker = false; + progress_mutex.unlock(); + return; + } + const request = progress_jobs[progress_head]; + progress_head = (progress_head + 1) % progress_jobs.len; + progress_count -= 1; + progress_mutex.unlock(); + const job = request.job; + var url_buf: [768]u8 = undefined; + const url = pure.progressUrl(job.server[0..job.server_len], job.id[0..job.id_len], request.episode[0..request.episode_len], &url_buf) orelse continue; + var auth_buf: [320]u8 = undefined; + const auth = pure.bearerHeader(job.token[0..job.token_len], &auth_buf) orelse continue; + var payload: [256]u8 = undefined; + const body = pure.progressBody(request.position, request.duration, request.finished, &payload) orelse continue; + var response: [16384]u8 = undefined; + if (http.fetch(url, &response, .{ .method = .PATCH, .payload = body, .content_type = "application/json", .auth_header = auth, .timeout_secs = 8 }) == null) + logs.pushLog("info", "audiobookshelf", "Playback progress could not sync to the server", false); + if (request.episode_len == 0) @import("library_store.zig").upsertProgress("audiobook", job.id[0..job.id_len], job.title[0..job.title_len], "", request.position, request.duration, "", job.id[0..job.id_len]); + } +} +fn tickAudioTimeline() void { + if (active_audio == null) return; + const plan = &active_audio.?; + if (!state.app.abs.connected) { + enqueueProgress(plan.*, false); + active_audio = null; + return; + } + if (state.app.active_player_idx >= state.app.players.items.len) return; + const player = state.app.players.items[state.app.active_player_idx]; + const current = player.current_url[0..@min(player.current_url_len, player.current_url.len)]; + if (!std.mem.eql(u8, current, plan.url[0..plan.url_len])) { + if (plan.opened) { + enqueueProgress(plan.*, false); + active_audio = null; + } + return; + } + if (plan.player_serial) |serial| { + if (serial != player.load_serial) { + enqueueProgress(plan.*, false); + active_audio = null; + return; + } + } else plan.player_serial = player.load_serial; + var duration: f64 = 0; + if (c.mpv.mpv_get_property(player.mpv_ctx, "duration", c.mpv.MPV_FORMAT_DOUBLE, &duration) >= 0 and duration > 0) plan.opened = true; + var position: f64 = 0; + if (c.mpv.mpv_get_property(player.mpv_ctx, "time-pos", c.mpv.MPV_FORMAT_DOUBLE, &position) >= 0 and std.math.isFinite(position) and position >= 0) + plan.last_position = position; + var eof: c_int = 0; + _ = c.mpv.mpv_get_property(player.mpv_ctx, "eof-reached", c.mpv.MPV_FORMAT_FLAG, &eof); + const now = @import("../core/io_global.zig").monotonicMilliTimestamp(); + if (pure.shouldAdvanceTrack(plan.opened, !player.is_loading, eof != 0, player.load_error_len > 0)) { + plan.last_position = active_tracks[plan.index].duration; + const final = !plan.complete_book or plan.index + 1 >= plan.count; + enqueueProgress(plan.*, final); + if (!final) { + plan.index += 1; + loadActiveAudio(0); + } else active_audio = null; + } else if (plan.opened and now - plan.last_sync_ms >= 10000) { + enqueueProgress(plan.*, false); + plan.last_sync_ms = now; + } +} +fn renderAudioSelection() bool { + audio_mutex.lock(); + const visible = audio_visible; + const title_buf = audio_job.title; + const title_len = audio_job.title_len; + const generation = audio_job.generation; + const count = audio_count; + const total = audio_total; + const complete = audio_complete_book; + audio_mutex.unlock(); + if (!visible) return false; + var column = dvui.box(@src(), .{ .dir = .vertical }, .{ .expand = .both }); + defer column.deinit(); + if (components.actionButton(@src(), "Back to library", .secondary, 91001)) { + clearAudioSelection(); + return true; + } + _ = dvui.label(@src(), "{s}", .{title_buf[0..title_len]}, .{ .color_text = theme.colors.text_primary }); + if (audio_loading.load(.acquire)) { + dvui.spinner(@src(), .{ .color_text = theme.colors.accent, .min_size_content = theme.iconSize(.md) }); + return true; + } + _ = dvui.label(@src(), "Choose a track or episode Β· {d} available of {d}", .{ count, total }, .{ .color_text = theme.colors.text_secondary }); + if (complete and components.actionButton(@src(), "Play book from saved position", .primary, 91002)) _ = playWholeBook(generation); + if (complete) _ = dvui.label(@src(), "Book tracks advance automatically and resume across files.", .{}, .{ .color_text = theme.colors.text_secondary }); + var scroll = dvui.scrollArea(@src(), .{}, .{ .expand = .both, .color_fill = theme.colors.bg_surface, .background = true }); + defer scroll.deinit(); + for (0..count) |idx| { + audio_mutex.lock(); + const track = audio_tracks[idx]; + audio_mutex.unlock(); + if (components.actionButton(@src(), track.title[0..track.title_len], .secondary, 91100 + idx)) _ = playAudioTrack(idx, generation); + } + return true; } // ── Resume fetch worker (struct{var}: copies inputs before spawn) ─────────── @@ -551,6 +945,8 @@ const ResumeFetch = struct { /// resumed book open, seek to the server-saved second exactly once. Cheap no-op /// unless a resume is pending. Mirrors anime_skip.tick()'s seek timing + path. pub fn tick() void { + playPendingAudioTrack(); + tickAudioTimeline(); if (!resume_pending.load(.acquire)) return; if (!resume_decided.load(.acquire)) return; // fetch still running if (state.app.active_player_idx >= state.app.players.items.len) return; @@ -595,6 +991,10 @@ pub fn tick() void { /// Disconnect + clear session (keeps the server URL so reconnect is one field). pub fn disconnect() void { + clearAudioSelection(); + parse_mutex.lock(); + defer parse_mutex.unlock(); + _ = library_generation.fetchAdd(1, .acq_rel); state.app.abs.connected = false; @memset(&state.app.abs.token, 0); state.app.abs.token_len = 0; @@ -634,11 +1034,14 @@ fn absGet(url: []const u8) ?[]u8 { } fn expireAuthSession() void { + clearAudioSelection(); parse_mutex.lock(); if (!state.app.abs.connected) { parse_mutex.unlock(); return; } + _ = library_generation.fetchAdd(1, .acq_rel); + state.app.abs.is_loading.store(false, .release); state.app.abs.connected = false; @memset(&state.app.abs.token, 0); state.app.abs.token_len = 0; @@ -664,6 +1067,12 @@ pub fn renderContent() void { renderLoginForm(); return; } + if (state.app.abs.login_error_len > 0) { + var error_buf: [256]u8 = undefined; + const error_text = safeUtf8Buf(state.app.abs.login_error[0..@min(state.app.abs.login_error_len, state.app.abs.login_error.len)], &error_buf); + _ = dvui.label(@src(), "{s}", .{error_text}, .{ .color_text = theme.colors.danger, .padding = dvui.Rect.all(8) }); + } + if (renderAudioSelection()) return; switch (state.app.abs.view) { .Libraries => renderLibraries(), .Books => renderBooks(), @@ -831,8 +1240,9 @@ fn renderBooks() void { } } + if (state.app.abs.book_count == 0 and more_available and !state.app.abs.is_loading.load(.acquire)) loadMore(); if (state.app.abs.book_count == 0) { - if (!state.app.abs.is_loading.load(.acquire)) { + if (!state.app.abs.is_loading.load(.acquire) and !loading_more.load(.acquire)) { components.emptyState(icons.tvg.lucide.@"book-audio", "No audiobooks here", "Choose another library or add books in Audiobookshelf."); return; } diff --git a/src/services/audiobookshelf_pure.zig b/src/services/audiobookshelf_pure.zig index 01cc0f51..c7e26923 100644 --- a/src/services/audiobookshelf_pure.zig +++ b/src/services/audiobookshelf_pure.zig @@ -247,6 +247,29 @@ pub fn parseLibraries(json: []const u8, out: []Library) usize { return forEachArrayObject(json, "\"libraries\":", &sink, LibSink.fill); } +/// Server library metadata is JSON, not dependent on minification/key order. +pub fn parseLibraryPage(a: std.mem.Allocator, json: []const u8, out: []Library) ?usize { + var parsed = std.json.parseFromSlice(std.json.Value, a, json, .{}) catch return null; + defer parsed.deinit(); + const libraries = valueField(parsed.value, "libraries"); + if (libraries != .array) return null; + var count: usize = 0; + for (libraries.array.items) |item| { + if (count == out.len) break; + const id = valueField(item, "id"); + const name = valueField(item, "name"); + if (id != .string or !validItemId(id.string) or id.string.len > 64 or name != .string or name.string.len == 0) continue; + var library: Library = .{}; + copyJsonText(&library.id, &library.id_len, id.string); + copyJsonText(&library.name, &library.name_len, name.string); + const media_type = valueField(item, "mediaType"); + if (media_type == .string) copyJsonText(&library.media_type, &library.media_type_len, media_type.string); + out[count] = library; + count += 1; + } + return count; +} + const BookSink = struct { out: []Book, n: usize = 0, @@ -274,6 +297,196 @@ pub fn parseItems(json: []const u8, out: []Book) usize { return forEachArrayObject(json, "\"results\":", &sink, BookSink.fill); } +/// Structured item pages preserve server row counts even when malformed or +/// incomplete items cannot be shown. null is an invalid response, not no books. +pub const ItemPage = struct { count: usize, consumed: usize }; +pub fn parseItemPage(a: std.mem.Allocator, json: []const u8, out: []Book) ?ItemPage { + var parsed = std.json.parseFromSlice(std.json.Value, a, json, .{}) catch return null; + defer parsed.deinit(); + const results = valueField(parsed.value, "results"); + if (results != .array) return null; + var count: usize = 0; + for (results.array.items) |item| { + if (count == out.len) break; + const id = valueField(item, "id"); + const media = valueField(item, "media"); + const metadata = valueField(media, "metadata"); + const title = valueField(metadata, "title"); + if (id != .string or !validItemId(id.string) or id.string.len > 64 or title != .string or title.string.len == 0) continue; + var book: Book = .{}; + copyJsonText(&book.id, &book.id_len, id.string); + copyJsonText(&book.title, &book.title_len, title.string); + var author = valueField(metadata, "authorName"); + if (author != .string) author = valueField(metadata, "author"); + if (author == .string) copyJsonText(&book.author, &book.author_len, author.string); + const duration = valueField(media, "duration"); + if (duration == .integer and duration.integer > 0) book.duration_secs = duration.integer; + if (duration == .float and std.math.isFinite(duration.float) and duration.float > 0 and duration.float < 9223372036854775807.0) + book.duration_secs = @intFromFloat(duration.float); + out[count] = book; + count += 1; + } + return .{ .count = count, .consumed = results.array.items.len }; +} +fn valueField(value: std.json.Value, key: []const u8) std.json.Value { + return if (value == .object) value.object.get(key) orelse .null else .null; +} +fn copyJsonText(dst: []u8, len: *usize, text: []const u8) void { + len.* = @min(text.len, dst.len); + while (len.* > 0 and !std.unicode.utf8ValidateSlice(text[0..len.*])) len.* -= 1; + @memcpy(dst[0..len.*], text[0..len.*]); +} + +pub const AudioTrack = struct { + title: [256]u8 = std.mem.zeroes([256]u8), + title_len: usize = 0, + content_path: [1024]u8 = std.mem.zeroes([1024]u8), + content_path_len: usize = 0, + episode: bool = false, + episode_id: [64]u8 = std.mem.zeroes([64]u8), + episode_id_len: usize = 0, + start_offset: f64 = 0, + duration: f64 = 0, + timing_valid: bool = false, +}; +pub const AudioPage = struct { count: usize, total: usize }; +/// Expanded items expose actual audio files. A download archive is never audio. +pub fn parseAudioTracks(a: std.mem.Allocator, json: []const u8, item_id: []const u8, out: []AudioTrack) ?AudioPage { + var parsed = std.json.parseFromSlice(std.json.Value, a, json, .{}) catch return null; + defer parsed.deinit(); + const media = valueField(parsed.value, "media"); + const media_type = valueField(parsed.value, "mediaType"); + const podcast = media_type == .string and std.mem.eql(u8, media_type.string, "podcast"); + const entries = valueField(media, if (podcast) "episodes" else "tracks"); + if (entries != .array) return null; + var count: usize = 0; + for (entries.array.items) |entry| { + if (count == out.len) break; + var track: AudioTrack = .{ .episode = podcast }; + const offset = numberField(entry, "startOffset"); + const duration = numberField(entry, "duration") orelse numberField(valueField(entry, "audioFile"), "duration"); + if (duration) |seconds| { + track.duration = seconds; + track.start_offset = if (podcast) 0 else offset orelse 0; + track.timing_valid = seconds > 0 and (podcast or offset != null); + } + if (podcast) { + const episode_id = valueField(entry, "id"); + if (episode_id != .string or !validItemId(episode_id.string) or episode_id.string.len > track.episode_id.len) continue; + copyJsonText(&track.episode_id, &track.episode_id_len, episode_id.string); + } + const title = valueField(entry, "title"); + if (title == .string) copyJsonText(&track.title, &track.title_len, title.string); + var path_buf: [1024]u8 = undefined; + const path = if (podcast) blk: { + const relative = valueField(valueField(valueField(entry, "audioFile"), "metadata"), "relPath"); + if (relative != .string or relative.string.len == 0) continue; + break :blk std.fmt.bufPrint(&path_buf, "/s/item/{s}/{s}", .{ item_id, relative.string }) catch continue; + } else blk: { + const content = valueField(entry, "contentUrl"); + if (content != .string) continue; + break :blk content.string; + }; + var prefix_buf: [128]u8 = undefined; + const prefix = std.fmt.bufPrint(&prefix_buf, "/s/item/{s}/", .{item_id}) catch return null; + if (!std.mem.startsWith(u8, path, prefix) or path.len > track.content_path.len or std.mem.indexOf(u8, path, "..") != null) continue; + copyJsonText(&track.content_path, &track.content_path_len, path); + if (track.title_len == 0) { + const start = if (std.mem.lastIndexOfScalar(u8, path, '/')) |slash| slash + 1 else 0; + copyJsonText(&track.title, &track.title_len, path[start..]); + } + out[count] = track; + count += 1; + } + return .{ .count = count, .total = entries.array.items.len }; +} +fn numberField(value: std.json.Value, key: []const u8) ?f64 { + const field = valueField(value, key); + const n = switch (field) { + .integer => @as(f64, @floatFromInt(field.integer)), + .float => field.float, + else => return null, + }; + return if (std.math.isFinite(n) and n >= 0) n else null; +} +/// Complete ordered tracks are required to promise a continuous book timeline. +pub fn completeBookTimeline(tracks: []const AudioTrack, total: usize) bool { + if (tracks.len == 0 or tracks.len != total) return false; + var end: f64 = 0; + for (tracks) |track| { + if (track.episode or !track.timing_valid or !std.math.isFinite(track.start_offset) or !std.math.isFinite(track.duration) or track.duration <= 0 or @abs(track.start_offset - end) > 0.25) return false; + end = track.start_offset + track.duration; + if (!std.math.isFinite(end)) return false; + } + return true; +} +pub const TrackPosition = struct { index: usize, seconds: f64 }; +pub fn locateBookPosition(tracks: []const AudioTrack, position: f64) ?TrackPosition { + if (!completeBookTimeline(tracks, tracks.len) or !std.math.isFinite(position) or position < 0) return null; + for (tracks, 0..) |track, idx| { + if (position < track.start_offset + track.duration or idx + 1 == tracks.len) + return .{ .index = idx, .seconds = std.math.clamp(position - track.start_offset, 0, @max(0, track.duration - 0.5)) }; + } + return null; +} +pub fn bookPosition(track: AudioTrack, local_seconds: f64) ?f64 { + if (!track.timing_valid or !std.math.isFinite(local_seconds) or local_seconds < 0) return null; + return track.start_offset + @min(local_seconds, track.duration); +} +pub fn parseProgressValue(a: std.mem.Allocator, json: []const u8) ProgressInfo { + var parsed = std.json.parseFromSlice(std.json.Value, a, json, .{}) catch return .{}; + defer parsed.deinit(); + const finished = valueField(parsed.value, "isFinished"); + return .{ .current_time = numberField(parsed.value, "currentTime"), .duration = numberField(parsed.value, "duration"), .is_finished = finished == .bool and finished.bool }; +} +pub fn shouldAdvanceTrack(opened: bool, ready: bool, eof: bool, failed: bool) bool { + return opened and ready and eof and !failed; +} + +pub fn progressUrl(server: []const u8, item: []const u8, episode: []const u8, out: []u8) ?[]const u8 { + if (!validItemId(item) or (episode.len > 0 and !validItemId(episode))) return null; + return if (episode.len > 0) + std.fmt.bufPrint(out, "{s}/api/me/progress/{s}/{s}", .{ std.mem.trimEnd(u8, server, "/"), item, episode }) catch null + else + std.fmt.bufPrint(out, "{s}/api/me/progress/{s}", .{ std.mem.trimEnd(u8, server, "/"), item }) catch null; +} +pub fn progressBody(position: f64, duration: f64, finished: bool, out: []u8) ?[]const u8 { + if (!std.math.isFinite(position) or position < 0 or !std.math.isFinite(duration) or duration <= 0) return null; + const pos = @min(position, duration); + return std.fmt.bufPrint(out, "{{\"currentTime\":{d:.3},\"duration\":{d:.3},\"progress\":{d:.6},\"isFinished\":{s}}}", .{ pos, duration, pos / duration, if (finished) "true" else "false" }) catch null; +} + +/// Encode real server-relative audio paths, including filenames with spaces. +pub fn audioTrackUrl(server: []const u8, content_path: []const u8, token: []const u8, out: []u8) ?[]const u8 { + if (!std.mem.startsWith(u8, content_path, "/s/item/") or std.mem.indexOf(u8, content_path, "..") != null) return null; + var path_buf: [3072]u8 = undefined; + var n: usize = 0; + const hex = "0123456789ABCDEF"; + var i: usize = 0; + while (i < content_path.len) : (i += 1) { + const c = content_path[i]; + if (c == '%' and i + 2 < content_path.len and std.ascii.isHex(content_path[i + 1]) and std.ascii.isHex(content_path[i + 2])) { + if (n + 3 > path_buf.len) return null; + @memcpy(path_buf[n..][0..3], content_path[i..][0..3]); + n += 3; + i += 2; + } else if (std.ascii.isAlphanumeric(c) or c == '/' or c == '-' or c == '_' or c == '.' or c == '~') { + if (n == path_buf.len) return null; + path_buf[n] = c; + n += 1; + } else { + if (n + 3 > path_buf.len) return null; + path_buf[n] = '%'; + path_buf[n + 1] = hex[c >> 4]; + path_buf[n + 2] = hex[c & 15]; + n += 3; + } + } + var tok_buf: [768]u8 = undefined; + const encoded_token = @import("music_subsonic_pure.zig").percentEncode(token, &tok_buf); + return std.fmt.bufPrint(out, "{s}{s}?token={s}", .{ std.mem.trimEnd(u8, server, "/"), path_buf[0..n], tok_buf[0..encoded_token] }) catch null; +} + /// Parse `GET /api/me/progress/{id}` β†’ the saved `currentTime` (seconds). Null /// when the payload has no progress (server returns 404/empty for an unstarted /// item), so the caller starts from 0. @@ -531,3 +744,54 @@ test "validItemId accepts ABS ids, rejects injection" { try std.testing.expect(!validItemId("id?token=x")); try std.testing.expect(!validItemId("a" ** 65)); } + +test "multifile book resume selects boundary file and relative time" { + const tracks = [_]AudioTrack{ + .{ .start_offset = 0, .duration = 30, .timing_valid = true }, + .{ .start_offset = 30, .duration = 40, .timing_valid = true }, + .{ .start_offset = 70, .duration = 50, .timing_valid = true }, + }; + try std.testing.expect(completeBookTimeline(&tracks, 3)); + const boundary = locateBookPosition(&tracks, 30).?; + try std.testing.expectEqual(@as(usize, 1), boundary.index); + try std.testing.expectEqual(@as(f64, 0), boundary.seconds); + const middle = locateBookPosition(&tracks, 82).?; + try std.testing.expectEqual(@as(usize, 2), middle.index); + try std.testing.expectEqual(@as(f64, 12), middle.seconds); + try std.testing.expectEqual(@as(f64, 82), bookPosition(tracks[2], 12).?); + try std.testing.expect(locateBookPosition(&tracks, std.math.nan(f64)) == null); + try std.testing.expect(!completeBookTimeline(&tracks, 4)); + var broken = tracks; + broken[1].start_offset = 35; + try std.testing.expect(!completeBookTimeline(&broken, 3)); +} +test "expanded audio parses real ordered tracks and podcast episode identities" { + var tracks: [4]AudioTrack = undefined; + const page = parseAudioTracks(std.testing.allocator, "{\"mediaType\":\"book\",\"media\":{\"tracks\":[{\"startOffset\":0,\"duration\":30,\"contentUrl\":\"/s/item/book1/Part 1.mp3\"},{\"startOffset\":30,\"duration\":40,\"contentUrl\":\"/s/item/book1/Part 2.mp3\"}]}}", "book1", &tracks).?; + try std.testing.expectEqual(@as(usize, 2), page.count); + try std.testing.expect(completeBookTimeline(tracks[0..page.count], page.total)); + try std.testing.expectEqualStrings("Part 1.mp3", tracks[0].title[0..tracks[0].title_len]); + const episodes = parseAudioTracks(std.testing.allocator, "{\"mediaType\":\"podcast\",\"media\":{\"episodes\":[{\"id\":\"ep_1\",\"title\":\"Episode\",\"duration\":60,\"audioFile\":{\"metadata\":{\"relPath\":\"Episode.mp3\"}}}]}}", "podcast1", &tracks).?; + try std.testing.expectEqual(@as(usize, 1), episodes.count); + try std.testing.expectEqualStrings("ep_1", tracks[0].episode_id[0..tracks[0].episode_id_len]); + try std.testing.expect(!completeBookTimeline(tracks[0..1], 1)); +} +test "book progress preserves whole timeline and episode scope" { + var out: [512]u8 = undefined; + try std.testing.expectEqualStrings("https://server/api/me/progress/book1/ep_1", progressUrl("https://server/", "book1", "ep_1", &out).?); + const body = progressBody(82, 120, false, &out).?; + const parsed = parseProgressValue(std.testing.allocator, body); + try std.testing.expectEqual(@as(f64, 82), parsed.current_time.?); + try std.testing.expectEqual(@as(f64, 120), parsed.duration.?); + try std.testing.expect(!parsed.is_finished); + try std.testing.expect(progressBody(std.math.inf(f64), 120, false, &out) == null); + try std.testing.expect(progressUrl("https://server", "../secret", "", &out) == null); +} + +test "track advance requires actual opened EOF and no player failure" { + try std.testing.expect(shouldAdvanceTrack(true, true, true, false)); + try std.testing.expect(!shouldAdvanceTrack(false, true, true, false)); + try std.testing.expect(!shouldAdvanceTrack(true, false, true, false)); + try std.testing.expect(!shouldAdvanceTrack(true, true, false, false)); + try std.testing.expect(!shouldAdvanceTrack(true, true, true, true)); +} diff --git a/src/services/comics.zig b/src/services/comics.zig index 5ea23ae3..9df4b54a 100644 --- a/src/services/comics.zig +++ b/src/services/comics.zig @@ -1,4 +1,5 @@ const std = @import("std"); +const reader_components = @import("../ui/components.zig"); const dvui = @import("dvui"); const state = @import("../core/state.zig"); const theme = @import("../ui/theme.zig"); @@ -79,6 +80,12 @@ pub fn drainPendingLoad() void { drainPendingSearch(); var url: @TypeOf(state.app.comic.pending_load_url) = undefined; comic_command_mutex.lock(); + // Retain the last command while metadata is loading. Consuming it here + // would call loadComic while busy, silently dropping chapter selections. + if (state.app.comic.is_loading.load(.acquire)) { + comic_command_mutex.unlock(); + return; + } const close = state.app.comic.pending_close.swap(false, .acq_rel); const load = state.app.comic.pending_load.swap(false, .acq_rel); const n = @min(state.app.comic.pending_load_len, url.len); @@ -250,6 +257,8 @@ pub fn loadComic(url: []const u8) void { // Drop any per-chapter Referer from a previous load (a Madara load sets its // own before staging pages); prevents leaking one site's Referer to another. state.app.comic.referer_len = 0; + state.app.comic.prev_url_len = 0; + state.app.comic.next_url_len = 0; state.app.comic.is_loading.store(true, .release); state.app.comic.page_count = 0; state.app.comic.dl_progress.store(0, .release); @@ -366,6 +375,13 @@ fn psePageDownloadThread(gen: u32) void { /// UI thread nor a download worker β€” today just `copyPage` (the HTTP page route). var pages_mutex: @import("../core/sync.zig").Mutex = .{}; +/// Actual page readiness, independent of completion order of download workers. +pub fn pageReady(i: usize) bool { + pages_mutex.lock(); + defer pages_mutex.unlock(); + return i < state.app.comic.page_count and i < state.app.comic.page_pixels.len and state.app.comic.page_pixels[i] != null; +} + /// Copy page `i`'s encoded bytes for an off-UI-thread consumer. Caller frees with /// `a`. Null = out of range or not downloaded yet (the caller answers 404 and the /// client retries as `dl_progress` climbs). @@ -479,7 +495,7 @@ fn fetchComicThread(gen: u32) void { // would otherwise curl a non-URL and find no images. Once page_urls are // staged the shared downloadPages() pipeline takes over unchanged. if (pure.mangaIdFromRoute(url)) |manga_id| { - const ok = loadMangadexPages(manga_id); + const ok = loadMangadexPages(manga_id, pure.chapterOffsetFromRoute(url) orelse 0); state.app.comic.is_loading.store(false, .release); if (!ok) { logs.pushLog("error", "comics", "MangaDex chapter failed to load", true); @@ -645,32 +661,93 @@ fn fetchMaybeUnblocked(url: []const u8, dst: []u8) usize { /// 1. /manga/{id}/feed?…&order[chapter]=asc&limit=1 β†’ the earliest chapter /// 2. /at-home/server/{chapterId} β†’ baseUrl + hash + files /// 3. {baseUrl}/data/{hash}/{file} β†’ one URL per page -fn loadMangadexPages(manga_id: []const u8) bool { - // ── 1. Earliest English chapter ── - var feed_url_buf: [320]u8 = undefined; - const feed_url = pure.buildFeedUrl(&feed_url_buf, manga_id, 1, 0) orelse return false; +const MANGADEX_CHAPTER_WINDOW: u32 = 100; +pub const MangaDexChapterRow = struct { + route_buf: [64]u8 = std.mem.zeroes([64]u8), + route_len: usize = 0, + title_buf: [160]u8 = std.mem.zeroes([160]u8), + title_len: usize = 0, + selected: bool = false, + pub fn route(self: *const MangaDexChapterRow) []const u8 { + return self.route_buf[0..self.route_len]; + } + pub fn title(self: *const MangaDexChapterRow) []const u8 { + return self.title_buf[0..self.title_len]; + } +}; +var md_chapters: [MANGADEX_CHAPTER_WINDOW]MangaDexChapterRow = [_]MangaDexChapterRow{.{}} ** MANGADEX_CHAPTER_WINDOW; +var md_chapter_count: usize = 0; +var md_chapter_mutex: @import("../core/sync.zig").Mutex = .{}; +var md_chapter_picker: bool = false; // owner-thread UI state + +pub fn mangaDexChapterCount() usize { + md_chapter_mutex.lock(); + defer md_chapter_mutex.unlock(); + return md_chapter_count; +} +pub fn mangaDexChapterRow(i: usize) ?MangaDexChapterRow { + md_chapter_mutex.lock(); + defer md_chapter_mutex.unlock(); + return if (i < md_chapter_count) md_chapters[i] else null; +} - const feed_buf = alloc.alloc(u8, 128 * 1024) catch return false; +fn loadMangadexPages(manga_id: []const u8, chapter_offset: u32) bool { + // Fetch a bounded chapter window, enabling direct chapter selection without + // loading thousands of metadata records or assuming integer chapter labels. + const window_offset = (chapter_offset / MANGADEX_CHAPTER_WINDOW) * MANGADEX_CHAPTER_WINDOW; + var feed_url_buf: [320]u8 = undefined; + const feed_url = pure.buildFeedUrl(&feed_url_buf, manga_id, MANGADEX_CHAPTER_WINDOW, window_offset) orelse return false; + const feed_buf = alloc.alloc(u8, 1024 * 1024) catch return false; defer alloc.free(feed_buf); const feed_n = fetchUrl(feed_url, feed_buf); if (feed_n == 0 or workers.isQuitting()) return false; - - const chapter_id = pure.firstChapterId(feed_buf[0..feed_n]) orelse { - logs.pushLog("warn", "comics", "MangaDex: no English chapter for this title", false); - return false; - }; - // Copy out of feed_buf before it's reused/freed. + const parsed = std.json.parseFromSlice(std.json.Value, alloc, feed_buf[0..feed_n], .{ .ignore_unknown_fields = true }) catch return false; + defer parsed.deinit(); + if (parsed.value != .object) return false; + const data = parsed.value.object.get("data") orelse return false; + if (data != .array) return false; + const selected_index = chapter_offset - window_offset; + if (selected_index >= data.array.items.len) return false; + const selected = data.array.items[selected_index]; + if (selected != .object) return false; + const id_value = selected.object.get("id") orelse return false; + if (id_value != .string or !pure.isValidId(id_value.string)) return false; var chap_id_buf: [64]u8 = undefined; - if (chapter_id.len > chap_id_buf.len) return false; - @memcpy(chap_id_buf[0..chapter_id.len], chapter_id); - const chap_id = chap_id_buf[0..chapter_id.len]; - + @memcpy(chap_id_buf[0..id_value.string.len], id_value.string); + const chap_id = chap_id_buf[0..id_value.string.len]; var chap_no_buf: [24]u8 = undefined; var chap_no_len: usize = 0; - if (pure.firstChapterNumber(feed_buf[0..feed_n])) |no| { - chap_no_len = @min(no.len, chap_no_buf.len); - @memcpy(chap_no_buf[0..chap_no_len], no[0..chap_no_len]); - } + const attrs = selected.object.get("attributes") orelse return false; + if (attrs == .object) if (attrs.object.get("chapter")) |value| { + if (value == .string) { + chap_no_len = @min(value.string.len, chap_no_buf.len); + @memcpy(chap_no_buf[0..chap_no_len], value.string[0..chap_no_len]); + } + }; + var chapter_rows: [MANGADEX_CHAPTER_WINDOW]MangaDexChapterRow = [_]MangaDexChapterRow{.{}} ** MANGADEX_CHAPTER_WINDOW; + var chapter_count: usize = 0; + for (data.array.items, 0..) |entry, idx| { + if (entry != .object or chapter_count >= chapter_rows.len) continue; + const attributes = entry.object.get("attributes") orelse continue; + if (attributes != .object) continue; + const no = attributes.object.get("chapter"); + const title_value = attributes.object.get("title"); + const no_text = if (no != null and no.? == .string) no.?.string else "Special"; + const title_text = if (title_value != null and title_value.? == .string) title_value.?.string else ""; + const row = &chapter_rows[chapter_count]; + const route = pure.buildChapterRoute(&row.route_buf, manga_id, window_offset + @as(u32, @intCast(idx))) orelse continue; + row.route_len = route.len; + const label = std.fmt.bufPrint(&row.title_buf, "Ch. {s}{s}{s}", .{ no_text, if (title_text.len > 0) " Β· " else "", title_text[0..@min(title_text.len, 100)] }) catch no_text; + if (label.ptr != row.title_buf[0..].ptr) @memcpy(row.title_buf[0..@min(label.len, row.title_buf.len)], label[0..@min(label.len, row.title_buf.len)]); + row.title_len = @min(label.len, row.title_buf.len); + row.selected = idx == selected_index; + chapter_count += 1; + } + const total_value = parsed.value.object.get("total"); + const total: u32 = if (total_value != null and total_value.? == .integer and total_value.?.integer > 0) + @intCast(@min(total_value.?.integer, 10000)) + else + window_offset + @as(u32, @intCast(data.array.items.len)); // ── 2. The @Home node serving that chapter's images ── var ah_url_buf: [160]u8 = undefined; @@ -699,6 +776,20 @@ fn loadMangadexPages(manga_id: []const u8) bool { } if (count == 0) return false; state.app.comic.page_count = count; + state.app.comic.prev_url_len = 0; + state.app.comic.next_url_len = 0; + if (chapter_offset > 0) { + const route = pure.buildChapterRoute(&state.app.comic.prev_url, manga_id, chapter_offset - 1) orelse ""; + state.app.comic.prev_url_len = route.len; + } + if (chapter_offset + 1 < total) { + const route = pure.buildChapterRoute(&state.app.comic.next_url, manga_id, chapter_offset + 1) orelse ""; + state.app.comic.next_url_len = route.len; + } + md_chapter_mutex.lock(); + md_chapters = chapter_rows; + md_chapter_count = chapter_count; + md_chapter_mutex.unlock(); // Title: the card click already staged the series title; append the chapter // number so the reader header reads "Berserk Β· Ch. 1". A load that didn't @@ -706,7 +797,8 @@ fn loadMangadexPages(manga_id: []const u8) bool { if (chap_no_len > 0) { var t_buf: [256]u8 = undefined; const existing = state.app.comic.title[0..state.app.comic.title_len]; - const base_title: []const u8 = if (existing.len > 0) existing else "MangaDex"; + const title_end = std.mem.indexOf(u8, existing, " Β· Ch. ") orelse existing.len; + const base_title: []const u8 = if (title_end > 0) existing[0..title_end] else "MangaDex"; const t = std.fmt.bufPrint(&t_buf, "{s} Β· Ch. {s}", .{ base_title, chap_no_buf[0..chap_no_len] }) catch base_title; const tl = @min(t.len, state.app.comic.title.len); @memcpy(state.app.comic.title[0..tl], t[0..tl]); @@ -2244,7 +2336,7 @@ fn searchWorker(gen: u32) void { // short page from every active source means we've hit the end. if (search_gen.load(.acquire) != gen) return; sr_count = filled; - if (filled < RESULTS_PER_PAGE) more_available = false; + if (filled == 0 or filled >= MAX_SEARCH_RESULTS) more_available = false; // SWR write: persist ONLY the default landing feed (default query on the // `all` source) so the next cold start seeds instantly. User searches and // source-filtered feeds are not cached. Skip if a newer search superseded us. @@ -2308,7 +2400,10 @@ fn loadMoreWorker(gen: u32) void { if (sourceActive(src) and search_gen.load(.acquire) == gen) added += fetchExpandedComicPage(query, next_page, gen, sr_count, @tagName(src)); } - if (added == 0 or added < RESULTS_PER_PAGE) more_available = false; + // Sources use different page sizes (HeanCms serves 12, MangaDex 24). + // A short merged page therefore cannot establish exhaustion. Stop only + // when no source contributed a row, or the bounded listing is full. + if (added == 0 or sr_count >= MAX_SEARCH_RESULTS) more_available = false; if (added > 0) sr_page = next_page; } @@ -3562,6 +3657,26 @@ pub fn renderPaneContent(pane_idx: usize) void { return; } + if (pure.mangaIdFromRoute(state.app.comic.url_buf[0..state.app.comic.url_len]) != null) { + if (reader_components.actionButton(@src(), if (md_chapter_picker) "Hide chapters" else "Choose chapter", .secondary, 72000)) md_chapter_picker = !md_chapter_picker; + if (md_chapter_picker) { + var chapter_scroll = dvui.scrollArea(@src(), .{}, .{ .expand = .horizontal, .min_size_content = .{ .h = 100 }, .max_size_content = .{ .w = std.math.floatMax(f32), .h = 180 } }); + defer chapter_scroll.deinit(); + var chapter_buttons = dvui.flexbox(@src(), .{ .justify_content = .start }, .{ .expand = .horizontal }); + defer chapter_buttons.deinit(); + var i: usize = 0; + while (mangaDexChapterRow(i)) |row| : (i += 1) { + var label_buf: [161]u8 = undefined; + const label = std.fmt.bufPrintZ(&label_buf, "{s}", .{safeUtf8(row.title())}) catch continue; + if (reader_components.actionButton(@src(), label, if (row.selected) .primary else .secondary, 72100 + i)) { + md_chapter_picker = false; + loadComic(row.route()); + break; + } + } + } + } + // Title bar + navigation + controls { var nav_row = dvui.box(@src(), .{ .dir = .horizontal }, .{ diff --git a/src/services/comics_pure.zig b/src/services/comics_pure.zig index aba4be46..dae914be 100644 --- a/src/services/comics_pure.zig +++ b/src/services/comics_pure.zig @@ -329,11 +329,32 @@ pub fn buildRouteUrl(out: []u8, manga_id: []const u8) ?[]const u8 { /// one, or if the id doesn't validate). pub fn mangaIdFromRoute(url: []const u8) ?[]const u8 { if (!std.mem.startsWith(u8, url, MD_SCHEME)) return null; - const id = url[MD_SCHEME.len..]; - if (!isValidId(id)) return null; + const rest = url[MD_SCHEME.len..]; + const end = std.mem.indexOfScalar(u8, rest, ':') orelse rest.len; + const id = rest[0..end]; + if (!isValidId(id) or chapterOffsetFromRoute(url) == null) return null; return id; } +/// A bounded English-feed row offset, not a chapter number (fractional chapters +/// and multiple scanlation releases are valid). Legacy series routes start at0. +pub fn chapterOffsetFromRoute(url: []const u8) ?u32 { + if (!std.mem.startsWith(u8, url, MD_SCHEME)) return null; + const rest = url[MD_SCHEME.len..]; + const colon = std.mem.indexOfScalar(u8, rest, ':') orelse return if (isValidId(rest)) 0 else null; + if (!isValidId(rest[0..colon])) return null; + const number = rest[colon + 1 ..]; + if (number.len == 0 or number.len > 5) return null; + for (number) |ch| if (!std.ascii.isDigit(ch)) return null; + const offset = std.fmt.parseInt(u32, number, 10) catch return null; + return if (offset < 10000) offset else null; // bounded client navigation cursor +} + +pub fn buildChapterRoute(out: []u8, manga_id: []const u8, offset: u32) ?[]const u8 { + if (!isValidId(manga_id) or offset >= 10000) return null; + return std.fmt.bufPrint(out, "{s}{s}:{d}", .{ MD_SCHEME, manga_id, offset }) catch null; +} + /// MangaDex spells its array/map params `includes[]` / `order[relevance]`. Those /// brackets MUST be percent-encoded: we fetch by exec'ing `curl`, and curl treats /// a bare `[...]` in a URL as a glob RANGE β€” `order[relevance]` makes it abort @@ -364,7 +385,8 @@ pub fn buildFeedUrl(out: []u8, manga_id: []const u8, limit: u32, offset: u32) ?[ if (!isValidId(manga_id)) return null; return std.fmt.bufPrint( out, - "{s}/manga/{s}/feed?translatedLanguage%5B%5D=en&order%5Bchapter%5D=asc" ++ + "{s}/manga/{s}/feed?translatedLanguage%5B%5D=en&order%5Bvolume%5D=asc&order%5Bchapter%5D=asc" ++ + "&includeEmptyPages=0&includeFuturePublishAt=0&includeExternalUrl=0" ++ "&limit={d}&offset={d}" ++ CONTENT_RATING, .{ MD_API, manga_id, limit, offset }, ) catch null; @@ -424,6 +446,8 @@ pub fn parseMangaEntry(obj: []const u8) ?MangaEntry { if (findJsonNode(obj, "\"title\"")) |tnode| { if (findJsonStr(tnode, "\"en\":\"")) |en| { title = en; + } else if (findJsonStr(tnode, "\"ja-ro\":\"")) |romaji| { + title = romaji; } else if (std.mem.indexOfScalar(u8, tnode, ':')) |c| { // First value in the title map, whatever its language key is. var i = c + 1; @@ -434,11 +458,40 @@ pub fn parseMangaEntry(obj: []const u8) ?MangaEntry { } } } + // Preserve a canonical English/romaji name rather than replacing it with + // an abbreviated alias such as OPM. For native-script maps lacking both, + // an actual English alternative makes discovery readable. + const title_node = findJsonNode(obj, "\"title\""); + const canonical_english = if (title_node) |node| findJsonStr(node, "\"en\":\"") else null; + const canonical_romaji = if (title_node) |node| findJsonStr(node, "\"ja-ro\":\"") else null; + if ((canonical_english == null or canonical_english.?.len == 0) and + (canonical_romaji == null or canonical_romaji.?.len == 0)) + { + if (findJsonNode(obj, "\"altTitles\"")) |alternatives| { + var titles = ObjIter{ .buf = alternatives }; + while (titles.next()) |alternative| { + if (findJsonStr(alternative, "\"en\":\"")) |english| { + if (english.len > 0) { + title = english; + break; + } + } + } + } + } if (title.len == 0) return null; var cover: []const u8 = ""; - if (std.mem.indexOf(u8, obj, "\"type\":\"cover_art\"")) |ca| { - if (findJsonStr(obj[ca..], "\"fileName\":\"")) |fn_| cover = fn_; + if (findJsonNode(obj, "\"relationships\"")) |relationships| { + var relations = ObjIter{ .buf = relationships }; + while (relations.next()) |relation| { + const kind = findJsonStr(relation, "\"type\":\"") orelse continue; + if (!std.mem.eql(u8, kind, "cover_art")) continue; + // Stay inside this relationship. A missing cover filename must not + // borrow a filename from an unrelated subsequent relationship. + cover = findJsonStr(relation, "\"fileName\":\"") orelse ""; + if (cover.len > 0) break; + } } return .{ .id = id, .title = title, .cover_file = cover }; @@ -1003,3 +1056,10 @@ test "comic page mime: short/unknown bytes fall back to jpeg, never crash" { try std.testing.expectEqualStrings("image/jpeg", imageMime("\x89PNG")); try std.testing.expectEqualStrings("image/jpeg", imageMime("404")); } + +test "MangaDex English alternate enriches native script without replacing canonical romaji" { + const obj = "{\"id\":\"801513ba-a712-498c-8f57-cae55b38cc92\",\"attributes\":{\"title\":{\"ja\":\"東京\"},\"altTitles\":[{\"en\":\"Tokyo\"}]}}"; + try std.testing.expectEqualStrings("Tokyo", parseMangaEntry(obj).?.title); + const canonical = "{\"id\":\"801513ba-a712-498c-8f57-cae55b38cc92\",\"attributes\":{\"title\":{\"en\":\"Tokyo Ghoul\"},\"altTitles\":[{\"en\":\"TG\"}]}}"; + try std.testing.expectEqualStrings("Tokyo Ghoul", parseMangaEntry(canonical).?.title); +} diff --git a/src/services/iptv.zig b/src/services/iptv.zig index 03d72bd0..4f9c8e8f 100644 --- a/src/services/iptv.zig +++ b/src/services/iptv.zig @@ -470,8 +470,11 @@ fn ingestOne(id: []const u8, kind: sources.Kind, fallback_url: []const u8, force // Replace the source's rows atomically: clear then insert. ingestChannels // applies the adult gate (group denylist + parser flag) per channel too. - catalog.clearSource(id); - const inserted = catalog.ingestChannels(id, buf[0..n], force_adult); + const inserted = catalog.replaceChannels(id, buf[0..n], force_adult); + if (inserted == 0) { + logs.pushLog("error", "iptv", "Catalog refresh failed; previous channels kept", true); + return false; + } catalog.markIngested(id, inserted); var lb: [96]u8 = undefined; diff --git a/src/services/iptv_catalog.zig b/src/services/iptv_catalog.zig index 43b0c5ec..c3ffb9d5 100644 --- a/src/services/iptv_catalog.zig +++ b/src/services/iptv_catalog.zig @@ -7,9 +7,8 @@ //! text, the LIKE escaping and the adult gate come from iptv_catalog_pure.zig //! (tested); this module owns the DB calls and thread-safety. //! -//! Ingest is idempotent per source: clearSource(id) then ingestChannels(id, …), -//! both inside one transaction, so a re-fetch never leaves a half-updated -//! source. The (source_id, url_hash) UNIQUE key dedupes within a playlist. +//! replaceChannels deletes and inserts inside one transaction, so a failed +//! refresh retains the previous source catalog. The (source_id, url_hash) UNIQUE key dedupes within a playlist. const std = @import("std"); const db = @import("../core/db.zig"); @@ -59,17 +58,38 @@ pub fn clearSource(source_id: []const u8) void { /// user-declared adult playlist whose channel names/groups don't self-identify. /// Runs in one transaction. Returns rows actually inserted (post-dedupe). pub fn ingestChannels(source_id: []const u8, channels: []const pure.IptvChannel, force_adult: bool) usize { + return writeChannels(source_id, channels, force_adult, false); +} + +/// Replace a successfully fetched source inside one transaction. A failed +/// insert rolls back the delete, preserving the previously usable catalog. +pub fn replaceChannels(source_id: []const u8, channels: []const pure.IptvChannel, force_adult: bool) usize { + return writeChannels(source_id, channels, force_adult, true); +} + +fn writeChannels(source_id: []const u8, channels: []const pure.IptvChannel, force_adult: bool, replace: bool) usize { if (channels.len == 0) return 0; write_mutex.lock(); defer write_mutex.unlock(); - db.exec("BEGIN"); + const begin = db.prepare("BEGIN IMMEDIATE") orelse return 0; + const begun = db.step(begin) == db.c.SQLITE_DONE; + db.finalize(begin); + if (!begun) return 0; + var committed = false; + defer if (!committed) db.exec("ROLLBACK"); + if (replace) { + const deletion = db.prepare("DELETE FROM iptv_catalog WHERE source_id=?1") orelse return 0; + db.bindText(deletion, 1, source_id); + const deleted = db.step(deletion) == db.c.SQLITE_DONE; + db.finalize(deletion); + if (!deleted) return 0; + } const sql = "INSERT OR IGNORE INTO iptv_catalog" ++ "(source_id,url_hash,name,name_lc,url,logo,category,country,quality,user_agent,referrer,nsfw,quality_tier)" ++ " VALUES(?1,?2,?3,?4,?5,?6,?7,?8,?9,?10,?11,?12,?13)"; const stmt = db.prepare(sql) orelse { - db.exec("ROLLBACK"); return 0; }; defer db.finalize(stmt); @@ -102,11 +122,17 @@ pub fn ingestChannels(source_id: []const u8, channels: []const pure.IptvChannel, db.bindText(stmt, 11, ch.referrer[0..ch.referrer_len]); db.bindInt(stmt, 12, if (is_adult) 1 else 0); db.bindInt(stmt, 13, @intCast(cpure.qualityTier(ch.quality[0..ch.quality_len]))); - if (db.step(stmt) == db.c.SQLITE_DONE) inserted += 1; + if (db.step(stmt) != db.c.SQLITE_DONE) return 0; + if (db.get()) |handle| if (db.c.sqlite3_changes(handle) > 0) { + inserted += 1; + }; } - db.exec("COMMIT"); - return inserted; + if (replace and inserted == 0) return 0; + const commit = db.prepare("COMMIT") orelse return 0; + committed = db.step(commit) == db.c.SQLITE_DONE; + db.finalize(commit); + return if (committed) inserted else 0; } fn lowerInto(s: []const u8, out: []u8) []const u8 { @@ -118,6 +144,8 @@ fn lowerInto(s: []const u8, out: []u8) []const u8 { /// Total channels matching `q` across the whole catalog β€” drives the "N /// channels" heading and the has-more test for infinite scroll. pub fn count(q: Query) usize { + write_mutex.lock(); + defer write_mutex.unlock(); var like_buf: [520]u8 = undefined; var sql_buf: [512]u8 = undefined; const sql = buildWhere("SELECT COUNT(*) FROM iptv_catalog", q, &sql_buf) orelse return 0; @@ -131,6 +159,8 @@ pub fn count(q: Query) usize { /// Fill up to `out.len` channels starting at `offset`, ordered name-first. /// Returns the number written. This is the render path's data source. pub fn queryPage(out: []pure.IptvChannel, offset: usize, q: Query) usize { + write_mutex.lock(); + defer write_mutex.unlock(); if (out.len == 0) return 0; var like_buf: [520]u8 = undefined; var sql_buf: [512]u8 = undefined; diff --git a/src/services/music_audius_pure.zig b/src/services/music_audius_pure.zig index 86d0acaf..9420bc22 100644 --- a/src/services/music_audius_pure.zig +++ b/src/services/music_audius_pure.zig @@ -14,6 +14,8 @@ fn truth(v: std.json.Value) bool { } fn copy(out: []u8, len: *usize, s: []const u8) void { len.* = @min(s.len, out.len); + // Never split a multibyte title/publisher at a fixed-buffer boundary. + while (len.* > 0 and !std.unicode.utf8ValidateSlice(s[0..len.*])) len.* -= 1; @memcpy(out[0..len.*], s[0..len.*]); } @@ -36,7 +38,12 @@ pub fn parseTrack(item: std.json.Value, base: []const u8) ?mp.MusicSong { copy(&row.id, &row.id_len, id); copy(&row.title, &row.title_len, title); copy(&row.artist, &row.artist_len, string(field(field(item, "user"), "name"))); - copy(&row.cover, &row.cover_len, string(field(field(item, "artwork"), "150x150"))); + const artwork = field(item, "artwork"); + var cover = string(field(artwork, "480x480")); + if (cover.len == 0) cover = string(field(artwork, "150x150")); + if (cover.len == 0) cover = string(field(artwork, "1000x1000")); + if (cover.len == 0) cover = string(field(field(field(item, "user"), "profile_picture"), "150x150")); + copy(&row.cover, &row.cover_len, cover); const url = std.fmt.bufPrint(&row.play_url, "{s}/v1/tracks/{s}/stream?app_name=Opal", .{ std.mem.trimEnd(u8, base, "/"), id }) catch return null; row.play_url_len = url.len; row.download_allowed = truth(field(item, "is_downloadable")); diff --git a/src/services/music_discovery.zig b/src/services/music_discovery.zig index 0c0c1085..03f59abf 100644 --- a/src/services/music_discovery.zig +++ b/src/services/music_discovery.zig @@ -274,9 +274,12 @@ fn fetchCached(kind: pure.CacheKind, id: []const u8, url: []const u8, origin_key const key = pure.cacheKey(kind, id, &key_buf); const ttl = pure.ttlFor(kind); if (key) |k| { - if (cacheGet(k, ttl, buf)) |hit| return hit; + if (cacheGet(k, ttl, buf)) |hit| { + if (pure.validResponse(alloc, hit, kind)) return hit; + } } const body = fetchJson(url, origin_key, rate, buf) orelse return null; + if (!pure.validResponse(alloc, body, kind)) return null; if (key) |k| cachePut(k, body); return body; } diff --git a/src/services/music_discovery_pure.zig b/src/services/music_discovery_pure.zig index e10a663e..38b93534 100644 --- a/src/services/music_discovery_pure.zig +++ b/src/services/music_discovery_pure.zig @@ -267,6 +267,17 @@ pub fn buildReleaseGroupUrl(out: []u8, artist_mbid: []const u8, limit: u16) ?[]c pub const CacheKind = enum { artist_mbid, similar, albums }; +/// An HTML gateway page or API error must never become a long-lived cache hit. +pub fn validResponse(a: std.mem.Allocator, json: []const u8, kind: CacheKind) bool { + var parsed = std.json.parseFromSlice(std.json.Value, a, json, .{}) catch return false; + defer parsed.deinit(); + if (kind == .similar) return parsed.value == .array; + if (parsed.value != .object) return false; + const key = if (kind == .artist_mbid) "artists" else "release-groups"; + const rows = parsed.value.object.get(key) orelse return false; + return rows == .array; +} + /// Seconds an entry of this kind stays usable. Artistβ†’MBID is essentially /// immutable, similarity is recomputed upstream on a slow cadence, and a /// discography changes only when something is released. diff --git a/src/services/music_jellyfin_pure.zig b/src/services/music_jellyfin_pure.zig index b795deb5..14a35f07 100644 --- a/src/services/music_jellyfin_pure.zig +++ b/src/services/music_jellyfin_pure.zig @@ -162,6 +162,21 @@ pub fn parseSong(obj: []const u8, id_buf: []u8, title_buf: []u8, artist_buf: []u return .{ .id = id_buf[0..idn], .title = title_buf[0..tn], .artist = artist_buf[0..an] }; } +pub fn parseSongValue(obj: std.json.Value) ?music.MusicSong { + const id = music.valueString(music.valueField(obj, "Id")); + const title = music.valueString(music.valueField(obj, "Name")); + if (id.len == 0 or id.len > 128 or title.len == 0) return null; + var row: music.MusicSong = .{}; + music.copyValueText(&row.id, &row.id_len, id); + music.copyValueText(&row.title, &row.title_len, title); + var artist = music.valueString(music.valueField(obj, "AlbumArtist")); + const artists = music.valueField(obj, "Artists"); + if (artist.len == 0 and artists == .array and artists.array.items.len > 0) artist = music.valueString(artists.array.items[0]); + music.copyValueText(&row.artist, &row.artist_len, artist); + music.copyValueText(&row.cover, &row.cover_len, id); + return row; +} + // ══════════════════════════════════════════════════════════ // Tests // ══════════════════════════════════════════════════════════ diff --git a/src/services/music_jiosaavn_pure.zig b/src/services/music_jiosaavn_pure.zig index 37b7dfd0..57bc4410 100644 --- a/src/services/music_jiosaavn_pure.zig +++ b/src/services/music_jiosaavn_pure.zig @@ -14,6 +14,7 @@ //! "perma_url":"https://www.jiosaavn.com/song/kesariya/…"}, … ] } const std = @import("std"); +const music = @import("music_subsonic_pure.zig"); pub const API = "https://www.jiosaavn.com/api.php"; @@ -156,6 +157,40 @@ pub fn parseSong(obj: []const u8, title_buf: []u8, artist_buf: []u8, url_buf: [] return .{ .title = title_buf[0..tn], .artist = artist_buf[0..an], .perma_url = url_buf[0..un], .image = img_buf[0..in] }; } +pub fn parseSongValue(obj: std.json.Value) ?music.MusicSong { + const title = music.valueString(music.valueField(obj, "song")); + const url = music.valueString(music.valueField(obj, "perma_url")); + if (title.len == 0 or !isPlayableUrl(url) or url.len > 256) return null; + var row: music.MusicSong = .{}; + decodeText(title, &row.title, &row.title_len); + decodeText(music.valueString(music.valueField(obj, "primary_artists")), &row.artist, &row.artist_len); + music.copyValueText(&row.play_url, &row.play_url_len, url); + var buf: [400]u8 = undefined; + const cover = coverUpgrade(music.valueString(music.valueField(obj, "image")), &buf); + if (cover.len <= row.cover.len) music.copyValueText(&row.cover, &row.cover_len, cover); + return row; +} +fn decodeText(src: []const u8, dst: []u8, len: *usize) void { + var i: usize = 0; + var n: usize = 0; + while (i < src.len and n < dst.len) { + var consumed: usize = 1; + var ch = src[i]; + for ([_][]const u8{ "&", """, "'", "'" }, [_]u8{ '&', '"', 39, 39 }) |entity, replacement| { + if (std.mem.startsWith(u8, src[i..], entity)) { + consumed = entity.len; + ch = replacement; + break; + } + } + dst[n] = ch; + n += 1; + i += consumed; + } + while (n > 0 and !std.unicode.utf8ValidateSlice(dst[0..n])) n -= 1; + len.* = n; +} + // ══════════════════════════════════════════════════════════ // Tests // ══════════════════════════════════════════════════════════ diff --git a/src/services/music_plex_pure.zig b/src/services/music_plex_pure.zig index b575940c..4666cf66 100644 --- a/src/services/music_plex_pure.zig +++ b/src/services/music_plex_pure.zig @@ -201,6 +201,28 @@ pub fn parseSong( }; } +pub fn parseSongValue(obj: std.json.Value) ?music.MusicSong { + const id = music.valueString(music.valueField(obj, "ratingKey")); + const title = music.valueString(music.valueField(obj, "title")); + if (id.len == 0 or id.len > 128 or title.len == 0) return null; + const media = music.valueField(obj, "Media"); + if (media != .array or media.array.items.len == 0) return null; + const parts = music.valueField(media.array.items[0], "Part"); + if (parts != .array or parts.array.items.len == 0) return null; + const key = music.valueString(music.valueField(parts.array.items[0], "key")); + if (key.len == 0 or key.len > 256 or !isValidPath(key)) return null; + var row: music.MusicSong = .{}; + music.copyValueText(&row.id, &row.id_len, id); + music.copyValueText(&row.title, &row.title_len, title); + music.copyValueText(&row.artist, &row.artist_len, music.valueString(music.valueField(obj, "grandparentTitle"))); + var thumb = music.valueString(music.valueField(obj, "thumb")); + if (thumb.len == 0) thumb = music.valueString(music.valueField(obj, "parentThumb")); + if (thumb.len == 0) thumb = music.valueString(music.valueField(obj, "grandparentThumb")); + if (thumb.len <= row.cover.len and isValidPath(thumb)) music.copyValueText(&row.cover, &row.cover_len, thumb); + music.copyValueText(&row.play_url, &row.play_url_len, key); + return row; +} + // ══════════════════════════════════════════════════════════ // Tests // ══════════════════════════════════════════════════════════ diff --git a/src/services/music_subsonic.zig b/src/services/music_subsonic.zig index dca7f664..d86892b2 100644 --- a/src/services/music_subsonic.zig +++ b/src/services/music_subsonic.zig @@ -106,6 +106,8 @@ pub fn selectSource(src: u8) bool { search_request.cancel(&state.app.music.is_loading); parse_mutex.lock(); defer parse_mutex.unlock(); + active_search = null; + next_offset = 0; state.app.music.source = src; state.app.music.result_count = 0; state.app.music.fetch_error = false; @@ -281,6 +283,8 @@ const SearchJob = struct { const RESULTS_CAP: usize = 200; const PAGE_SIZE: usize = 40; var more_available: bool = false; +var active_search: ?SearchJob = null; +var next_offset: u32 = 0; var loading_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(false); pub fn searchMusic(query: []const u8) void { @@ -292,7 +296,14 @@ pub fn searchMusic(query: []const u8) void { more_available = true; loading_more.store(false, .release); const job = makeSearchJob(query, my_gen, false, 0) orelse return abortSearch(my_gen); - if (!spawnSearchJob(job)) search_request.finish(my_gen, &state.app.music.is_loading); + parse_mutex.lock(); + active_search = job; + next_offset = 0; + parse_mutex.unlock(); + if (!spawnSearchJob(job)) { + state.app.music.fetch_error = true; + search_request.finish(my_gen, &state.app.music.is_loading); + } } fn makeSearchJob(query: []const u8, generation: u32, append: bool, offset: u32) ?SearchJob { @@ -351,16 +362,20 @@ fn spawnSearchJob(job: SearchJob) bool { fn loadMore() void { if (!more_available or loading_more.load(.acquire) or state.app.music.is_loading.load(.acquire)) return; const count = resultCount(); - if (count == 0 or count >= RESULTS_CAP) { + if (count >= RESULTS_CAP) { more_available = false; return; } - const query = std.mem.sliceTo(&state.app.music.search_buf, 0); - if (query.len == 0 or loading_more.swap(true, .acq_rel)) return; - const job = makeSearchJob(query, search_request.current(), true, @intCast(count)) orelse { + if (loading_more.swap(true, .acq_rel)) return; + parse_mutex.lock(); + var job = active_search orelse { + parse_mutex.unlock(); loading_more.store(false, .release); return; }; + job.append = true; + job.offset = next_offset; + parse_mutex.unlock(); if (!spawnSearchJob(job)) loading_more.store(false, .release); } @@ -375,20 +390,27 @@ fn abortSearch(my_gen: u32) void { fn finishJob(job: SearchJob, published: bool) void { if (job.append) { if (search_request.isCurrent(job.generation)) { - if (!published) more_available = false; + if (!published) { + more_available = false; + state.app.music.fetch_error = true; + } loading_more.store(false, .release); } - } else search_request.finish(job.generation, &state.app.music.is_loading); + } else { + if (!published and search_request.isCurrent(job.generation)) state.app.music.fetch_error = true; + search_request.finish(job.generation, &state.app.music.is_loading); + } } -fn publishPage(job: SearchJob, added: usize, base: usize) void { +fn publishPage(job: SearchJob, added: usize, base: usize, consumed: usize) void { state.app.music.result_count = base + added; - more_available = added == PAGE_SIZE and state.app.music.result_count < RESULTS_CAP; + next_offset = job.offset +| @as(u32, @intCast(@min(consumed, std.math.maxInt(u32)))); + more_available = consumed >= PAGE_SIZE and state.app.music.result_count < RESULTS_CAP; + state.app.music.fetch_error = false; if (added == 0 and !job.append) logs.pushLog("info", "music", "No tracks found", false); } -/// JioSaavn search worker β€” public API, no auth. Fills play_url (perma_url) + -/// a full cover URL; playback hands perma_url to mpv/yt-dlp. +/// Audius public discovery. Unavailable or access-conditioned tracks are omitted. fn audiusWorker(job: SearchJob) void { var published = false; defer finishJob(job, published); @@ -402,21 +424,21 @@ fn audiusWorker(job: SearchJob) void { defer alloc.free(body); var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch return; defer parsed.deinit(); - if (parsed.value != .object) return; - const data = parsed.value.object.get("data") orelse return; - if (data != .array) return; + const provider_rows = pure.pageRows(parsed.value, .audius) orelse return; + const consumed = provider_rows.len; parse_mutex.lock(); defer parse_mutex.unlock(); if (!search_request.isCurrent(job.generation)) return; const start = if (job.append) state.app.music.result_count else 0; var count: usize = 0; - for (data.array.items) |item| { + for (provider_rows) |item| { if (start + count >= RESULTS_CAP) break; const row = au_pure.parseTrack(item, base_url) orelse continue; state.app.music.results[start + count] = row; + if (duplicateRow(start + count)) continue; count += 1; } - publishPage(job, count, start); + publishPage(job, count, start, consumed); published = true; } @@ -434,32 +456,25 @@ fn jiosaavnWorker(job: SearchJob) void { return; }; defer alloc.free(body); + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch return; + defer parsed.deinit(); + const provider_rows = pure.pageRows(parsed.value, .jiosaavn) orelse return; + const consumed = provider_rows.len; if (search_request.current() != my_gen) return; parse_mutex.lock(); defer parse_mutex.unlock(); - if (search_request.current() != my_gen) return; - + if (!search_request.isCurrent(my_gen)) return; const base: usize = if (job.append) state.app.music.result_count else 0; var count: usize = 0; - var it = js_pure.SongIter{ .json = body }; - while (it.next()) |obj| { + for (provider_rows) |obj| { if (count >= PAGE_SIZE or base + count >= RESULTS_CAP) break; - var tb: [160]u8 = undefined; - var ab: [128]u8 = undefined; - var ub: [400]u8 = undefined; - var ib: [400]u8 = undefined; - const s = js_pure.parseSong(obj, &tb, &ab, &ub, &ib) orelse continue; - var row = &state.app.music.results[base + count]; - row.* = .{}; - copyField(&row.title, &row.title_len, s.title); - copyField(&row.artist, &row.artist_len, s.artist); - copyField(&row.play_url, &row.play_url_len, s.perma_url); - var cov: [256]u8 = undefined; - copyField(&row.cover, &row.cover_len, js_pure.coverUpgrade(s.image, &cov)); + const row = js_pure.parseSongValue(obj) orelse continue; + state.app.music.results[base + count] = row; + if (duplicateRow(base + count)) continue; count += 1; } - publishPage(job, count, base); + publishPage(job, count, base, consumed); published = true; } @@ -476,39 +491,26 @@ fn subsonicWorker(job: SearchJob) void { return; }; defer alloc.free(body); + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch return; + defer parsed.deinit(); + const provider_rows = pure.pageRows(parsed.value, .subsonic) orelse return; + const consumed = provider_rows.len; if (search_request.current() != my_gen) return; // superseded - if (!pure.responseOk(body)) { - if (!job.append) state.app.music.fetch_error = true; - logs.pushLog("info", "music", "Subsonic auth/search failed β€” check server URL + credentials", false); - return; - } - parse_mutex.lock(); defer parse_mutex.unlock(); - if (search_request.current() != my_gen) return; - - const scope = pure.songsScope(body); + if (!search_request.isCurrent(my_gen)) return; const base: usize = if (job.append) state.app.music.result_count else 0; var count: usize = 0; - var it = pure.SongIter{ .json = scope }; - while (it.next()) |obj| { + for (provider_rows) |obj| { if (count >= PAGE_SIZE or base + count >= RESULTS_CAP) break; - var idb: [128]u8 = undefined; - var tb: [160]u8 = undefined; - var ab: [128]u8 = undefined; - var cb: [96]u8 = undefined; - const s = pure.parseSong(obj, &idb, &tb, &ab, &cb) orelse continue; - var row = &state.app.music.results[base + count]; - row.* = .{}; - copyField(&row.id, &row.id_len, s.id); - copyField(&row.title, &row.title_len, s.title); - copyField(&row.artist, &row.artist_len, s.artist); - copyField(&row.cover, &row.cover_len, s.cover); + const row = pure.parseSongValue(obj) orelse continue; + state.app.music.results[base + count] = row; + if (duplicateRow(base + count)) continue; count += 1; } - publishPage(job, count, base); + publishPage(job, count, base, consumed); published = true; } @@ -528,38 +530,26 @@ fn jellyfinMusicWorker(job: SearchJob) void { return; }; defer alloc.free(body); + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch return; + defer parsed.deinit(); + const provider_rows = pure.pageRows(parsed.value, .jellyfin) orelse return; + const consumed = provider_rows.len; if (search_request.current() != my_gen) return; // superseded parse_mutex.lock(); defer parse_mutex.unlock(); - if (search_request.current() != my_gen) return; - - const scope = jf_pure.itemsScope(body); - if (scope.len == 0) { - if (!job.append) state.app.music.fetch_error = true; - logs.pushLog("info", "music", "Jellyfin search failed β€” check the server sign-in", false); - return; - } - + if (!search_request.isCurrent(my_gen)) return; const base: usize = if (job.append) state.app.music.result_count else 0; var count: usize = 0; - var it = jf_pure.ItemIter{ .json = scope }; - while (it.next()) |obj| { + for (provider_rows) |obj| { if (count >= PAGE_SIZE or base + count >= RESULTS_CAP) break; - var idb: [128]u8 = undefined; - var tb: [160]u8 = undefined; - var ab: [128]u8 = undefined; - const s = jf_pure.parseSong(obj, &idb, &tb, &ab) orelse continue; - var row = &state.app.music.results[base + count]; - row.* = .{}; - copyField(&row.id, &row.id_len, s.id); - copyField(&row.title, &row.title_len, s.title); - copyField(&row.artist, &row.artist_len, s.artist); - copyField(&row.cover, &row.cover_len, s.id); // Primary image is keyed by the item id + const row = jf_pure.parseSongValue(obj) orelse continue; + state.app.music.results[base + count] = row; + if (duplicateRow(base + count)) continue; count += 1; } - publishPage(job, count, base); + publishPage(job, count, base, consumed); published = true; } @@ -581,47 +571,38 @@ fn plexMusicWorker(job: SearchJob) void { return; }; defer alloc.free(body); + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch return; + defer parsed.deinit(); + const provider_rows = pure.pageRows(parsed.value, .plex) orelse return; + const consumed = provider_rows.len; if (search_request.current() != my_gen) return; // superseded parse_mutex.lock(); defer parse_mutex.unlock(); - if (search_request.current() != my_gen) return; - - // Plex omits "Metadata" entirely on a zero-result search, so an empty scope - // is "no tracks", NOT an error β€” only a body that isn't a MediaContainer is. - const scope = px_pure.metadataScope(body); - if (scope.len == 0 and std.mem.indexOf(u8, body, "MediaContainer") == null) { - if (!job.append) state.app.music.fetch_error = true; - logs.pushLog("info", "music", "Plex search failed β€” check the server sign-in", false); - return; - } - + if (!search_request.isCurrent(my_gen)) return; const base: usize = if (job.append) state.app.music.result_count else 0; var count: usize = 0; - var it = px_pure.TrackIter{ .json = scope }; - while (it.next()) |obj| { + for (provider_rows) |obj| { if (count >= PAGE_SIZE or base + count >= RESULTS_CAP) break; - var idb: [128]u8 = undefined; - var tb: [160]u8 = undefined; - var ab: [128]u8 = undefined; - var cb: [200]u8 = undefined; - var pb: [256]u8 = undefined; - const s = px_pure.parseSong(obj, &idb, &tb, &ab, &cb, &pb) orelse continue; - if (s.part_key.len == 0) continue; // unplayable without a Part - var row = &state.app.music.results[base + count]; - row.* = .{}; - copyField(&row.id, &row.id_len, s.id); - copyField(&row.title, &row.title_len, s.title); - copyField(&row.artist, &row.artist_len, s.artist); - copyField(&row.cover, &row.cover_len, s.thumb); - copyField(&row.play_url, &row.play_url_len, s.part_key); + const row = px_pure.parseSongValue(obj) orelse continue; + state.app.music.results[base + count] = row; + if (duplicateRow(base + count)) continue; count += 1; } - publishPage(job, count, base); + publishPage(job, count, base, consumed); published = true; } +/// Caller holds parse_mutex; compare stable provider IDs or direct play URLs. +fn duplicateRow(index: usize) bool { + const row = &state.app.music.results[index]; + for (state.app.music.results[0..index]) |*existing| { + if (pure.sameTrack(existing, row)) return true; + } + return false; +} + fn copyField(dst: []u8, len: *usize, src: []const u8) void { @import("../core/text.zig").setFixedUtf8(dst, len, src); } @@ -1115,16 +1096,16 @@ fn renderCover(i: usize, song: *const pure.MusicSong) void { var url_buf: [1024]u8 = undefined; const cover_url = coverUrlFor(song, &url_buf); - if (cover_url.len > 0) { - const h = std.hash.Fnv1a_64.hash(cover_url); - if (slot.url_hash != h and !slot.fetching) { - poster.deinitPoster(&slot.pixels, &slot.tex); - slot.w = 0; - slot.h = 0; - slot.attempted = false; - slot.failed = false; - slot.url_hash = h; - } + const h = std.hash.Fnv1a_64.hash(cover_url); + if (slot.url_hash != h and !slot.fetching) { + poster.deinitPoster(&slot.pixels, &slot.tex); + slot.w = 0; + slot.h = 0; + slot.attempted = false; + slot.failed = false; + slot.url_hash = h; + } + if (cover_url.len > 0 and slot.url_hash == h) { _ = poster.uploadIfReady(&slot.pixels, slot.w, slot.h, &slot.tex); if (slot.fetching) slot.attempted = true else if (slot.attempted and slot.pixels == null and slot.tex == null) slot.failed = true; if (!slot.failed and slot.tex == null and !slot.fetching and slot.pixels == null) { @@ -1133,8 +1114,9 @@ fn renderCover(i: usize, song: *const pure.MusicSong) void { } } - if (slot.tex) |*tex| { - _ = dvui.image(@src(), .{ .source = .{ .texture = tex.* } }, .{ + const visible_texture = if (cover_url.len > 0 and slot.url_hash == h) slot.tex else null; + if (visible_texture) |tex| { + _ = dvui.image(@src(), .{ .source = .{ .texture = tex } }, .{ .id_extra = i + 1000, .expand = .both, .corner_radius = dvui.Rect.all(8), @@ -1223,7 +1205,8 @@ fn renderResults() void { const total = @min(state.app.music.result_count, state.app.music.results.len); parse_mutex.unlock(); const src = state.app.music.source; - if (total == 0 and !state.app.music.is_loading.load(.acquire)) { + if (total == 0 and more_available and !state.app.music.is_loading.load(.acquire)) loadMore(); + if (total == 0 and !state.app.music.is_loading.load(.acquire) and !loading_more.load(.acquire)) { const hint: []const u8 = if (!sourceConfigured(src)) switch (src) { SRC_JELLYFIN => "Sign in to Jellyfin (Jellyfin tab) to play its music library", SRC_PLEX => "Sign in to Plex (Plex tab) to play its music library", diff --git a/src/services/music_subsonic_pure.zig b/src/services/music_subsonic_pure.zig index 09c3c897..16265d3a 100644 --- a/src/services/music_subsonic_pure.zig +++ b/src/services/music_subsonic_pure.zig @@ -23,6 +23,69 @@ pub const CLIENT = "Opal"; /// scheme routing, though the music tab plays directly.) pub const SCHEME = "subsonic:"; +/// Provider rows consumed by a page, before filtering unavailable tracks. +/// null distinguishes malformed/error responses from a successful empty page. +pub const SearchResponseKind = enum { jiosaavn, subsonic, jellyfin, plex, audius }; +pub fn pageRowCount(a: std.mem.Allocator, body: []const u8, kind: SearchResponseKind) ?usize { + var parsed = std.json.parseFromSlice(std.json.Value, a, body, .{}) catch return null; + defer parsed.deinit(); + return (pageRows(parsed.value, kind) orelse return null).len; +} +pub fn pageRows(root: std.json.Value, kind: SearchResponseKind) ?[]const std.json.Value { + if (root != .object) return null; + const rows = switch (kind) { + .jiosaavn => root.object.get("results") orelse return null, + .jellyfin => root.object.get("Items") orelse return null, + .audius => root.object.get("data") orelse return null, + .plex => blk: { + const container = root.object.get("MediaContainer") orelse return null; + if (container != .object) return null; + break :blk container.object.get("Metadata") orelse return &.{}; + }, + .subsonic => blk: { + const response = root.object.get("subsonic-response") orelse return null; + if (response != .object) return null; + const status = response.object.get("status") orelse return null; + if (status != .string or !std.mem.eql(u8, status.string, "ok")) return null; + const search = response.object.get("searchResult3") orelse return null; + if (search != .object) return null; + break :blk search.object.get("song") orelse return &.{}; + }, + }; + return if (rows == .array) rows.array.items else null; +} + +pub fn valueField(v: std.json.Value, key: []const u8) std.json.Value { + return if (v == .object) v.object.get(key) orelse .null else .null; +} +pub fn valueString(v: std.json.Value) []const u8 { + return if (v == .string) v.string else ""; +} +pub fn copyValueText(dst: []u8, len: *usize, src: []const u8) void { + len.* = @min(dst.len, src.len); + while (len.* > 0 and !std.unicode.utf8ValidateSlice(src[0..len.*])) len.* -= 1; + @memcpy(dst[0..len.*], src[0..len.*]); +} +pub fn parseSongValue(obj: std.json.Value) ?MusicSong { + const id = valueString(valueField(obj, "id")); + const title = valueString(valueField(obj, "title")); + if (id.len == 0 or id.len > 128 or title.len == 0) return null; + var row: MusicSong = .{}; + copyValueText(&row.id, &row.id_len, id); + copyValueText(&row.title, &row.title_len, title); + copyValueText(&row.artist, &row.artist_len, valueString(valueField(obj, "artist"))); + const cover = valueString(valueField(obj, "coverArt")); + if (cover.len <= row.cover.len) copyValueText(&row.cover, &row.cover_len, cover); + return row; +} + +pub fn sameTrack(a: *const MusicSong, b: *const MusicSong) bool { + if (a.id_len > 0 and b.id_len > 0) + return std.mem.eql(u8, a.id[0..@min(a.id_len, a.id.len)], b.id[0..@min(b.id_len, b.id.len)]); + return a.play_url_len > 0 and b.play_url_len > 0 and + std.mem.eql(u8, a.play_url[0..@min(a.play_url_len, a.play_url.len)], b.play_url[0..@min(b.play_url_len, b.play_url.len)]); +} + // ── Validation gates ── /// The server base must be a plain http(s) origin we can safely prefix onto diff --git a/src/services/novel_sources_pure.zig b/src/services/novel_sources_pure.zig index 04b7c509..78ce51ce 100644 --- a/src/services/novel_sources_pure.zig +++ b/src/services/novel_sources_pure.zig @@ -28,7 +28,7 @@ pub const themesia = @import("manga_themesia_pure.zig"); /// Which engine a novel came from β€” carried per search-result so `openNovel` / /// `openChapter` dispatch to the right chapter-list + chapter-text extractor. -pub const NovelSource = enum { wikisource, madara_novel, lightnovelwp, readwn, readnovelfull, internet_archive, royalroad, novelfire }; +pub const NovelSource = enum { wikisource, madara_novel, lightnovelwp, readwn, readnovelfull, internet_archive, royalroad, novelfire, gutenberg, openlibrary }; // ══════════════════════════════════════════════════════════ // Chapter-TEXT container selectors (the ONE new thing per shared engine) @@ -203,7 +203,7 @@ pub fn chapterContentHtml(html: []const u8, source: NovelSource) ?[]const u8 { if (containerInner(html, "id=\"chapter-container\"")) |c| if (htmlHasText(c)) return c; return null; }, - .wikisource, .internet_archive => return null, // handled directly by novels.zig + .wikisource, .internet_archive, .gutenberg, .openlibrary => return null, // handled directly by novels.zig } } diff --git a/src/services/novels.zig b/src/services/novels.zig index 3bf80696..85eee2d4 100644 --- a/src/services/novels.zig +++ b/src/services/novels.zig @@ -16,6 +16,7 @@ //! next/prev/resume β†’ openChapter(current Β± 1) / the persisted chapter. const std = @import("std"); +const safeUtf8 = @import("../core/text.zig").safeUtf8; const dvui = @import("dvui"); const icons = @import("icons"); const state = @import("../core/state.zig"); @@ -28,6 +29,7 @@ const pure = @import("novels_pure.zig"); const nsp = @import("novel_sources_pure.zig"); const archive = @import("archive_pure.zig"); const expanded = @import("expanded_reading_pure.zig"); +const books = @import("public_books_pure.zig"); const source_config = @import("../core/source_config.zig"); // Anti-block fetch layer β€” used by the HTML-scraper engines' GETs so a // Cloudflare/DDoS-Guard/captcha-fronted source resolves through the anti-detect @@ -86,6 +88,8 @@ var nr_urls: [MAX_RESULTS][512]u8 = undefined; var nr_url_lens: [MAX_RESULTS]usize = std.mem.zeroes([MAX_RESULTS]usize); var nr_source: [MAX_RESULTS]NovelSource = undefined; var nr_count: usize = 0; +var nr_metadata: [MAX_RESULTS]NovelMetadata = [_]NovelMetadata{.{}} ** MAX_RESULTS; +var novel_covers: [MAX_RESULTS]components.CoverSlot = [_]components.CoverSlot{.{}} ** MAX_RESULTS; const MAX_CHAPTERS: usize = 400; // For Wikisource: the full page title ("Frankenstein/Chapter 1") β€” the fetch key @@ -116,12 +120,32 @@ var text_gen: std.atomic.Value(u32) = std.atomic.Value(u32).init(0); // never read a row mid-rewrite. Copy semantics, not slices: the caller is on // another thread and holds nothing once the lock drops. +pub const NovelMetadata = struct { + author_buf: [256]u8 = std.mem.zeroes([256]u8), + author_len: usize = 0, + overview_buf: [768]u8 = std.mem.zeroes([768]u8), + overview_len: usize = 0, + cover_buf: [768]u8 = std.mem.zeroes([768]u8), + cover_len: usize = 0, + year: u16 = 0, + pub fn author(self: *const NovelMetadata) []const u8 { + return self.author_buf[0..self.author_len]; + } + pub fn overview(self: *const NovelMetadata) []const u8 { + return self.overview_buf[0..self.overview_len]; + } + pub fn cover(self: *const NovelMetadata) []const u8 { + return self.cover_buf[0..self.cover_len]; + } +}; + pub const ListRow = struct { title_buf: [256]u8 = std.mem.zeroes([256]u8), title_len: usize = 0, url_buf: [1024]u8 = std.mem.zeroes([1024]u8), url_len: usize = 0, source: u8 = 0, + metadata: NovelMetadata = .{}, pub fn title(self: *const ListRow) []const u8 { return self.title_buf[0..@min(self.title_len, self.title_buf.len)]; @@ -131,6 +155,74 @@ pub const ListRow = struct { } }; +/// A complete API read, copied under the publication mutex. Allocate this on +/// the heap: text plus catalog/chapter rows exceed a connection-thread stack. +pub const ReaderSnapshot = struct { + view: @TypeOf(state.app.novels.view), + loading: bool, + chapters_loading: bool, + text_loading: bool, + fetch_error: bool, + current_chapter: usize, + text_truncated: bool, + work_title: [256]u8, + work_title_len: usize, + chapter_label: [256]u8, + chapter_label_len: usize, + text_buf: [TEXT_CAP]u8, + text_len: usize, + results: [MAX_RESULTS]ListRow, + result_count: usize, + chapter_titles: [MAX_CHAPTERS][256]u8, + chapter_title_lens: [MAX_CHAPTERS]usize, + chapter_count: usize, + has_more: bool, + loading_more: bool, + search_generation: u32, + chapter_generation: u32, + text_generation: u32, +}; + +pub fn copyReaderSnapshot(out: *ReaderSnapshot) void { + parse_mutex.lock(); + defer parse_mutex.unlock(); + @memset(std.mem.asBytes(out), 0); + const n = &state.app.novels; + out.view = n.view; + out.loading = n.is_loading.load(.acquire); + out.chapters_loading = n.chapters_loading.load(.acquire); + out.text_loading = n.text_loading.load(.acquire); + out.fetch_error = n.fetch_error; + out.current_chapter = n.current_chapter; + out.text_truncated = n.text_truncated; + out.work_title_len = @min(n.work_title_len, out.work_title.len); + @memcpy(out.work_title[0..out.work_title_len], n.work_title[0..out.work_title_len]); + out.chapter_label_len = @min(n.chapter_label_len, out.chapter_label.len); + @memcpy(out.chapter_label[0..out.chapter_label_len], n.chapter_label[0..out.chapter_label_len]); + out.text_len = @min(n.text_len, out.text_buf.len); + @memcpy(out.text_buf[0..out.text_len], n.text_buf[0..out.text_len]); + out.result_count = @min(nr_count, MAX_RESULTS); + for (0..out.result_count) |idx| { + const row = &out.results[idx]; + row.title_len = @min(nr_title_lens[idx], row.title_buf.len); + @memcpy(row.title_buf[0..row.title_len], nr_titles[idx][0..row.title_len]); + row.url_len = @min(nr_url_lens[idx], row.url_buf.len); + @memcpy(row.url_buf[0..row.url_len], nr_urls[idx][0..row.url_len]); + row.source = @intFromEnum(nr_source[idx]); + row.metadata = nr_metadata[idx]; + } + out.chapter_count = @min(ch_count, MAX_CHAPTERS); + for (0..out.chapter_count) |idx| { + out.chapter_title_lens[idx] = @min(ch_title_lens[idx], out.chapter_titles[idx].len); + @memcpy(out.chapter_titles[idx][0..out.chapter_title_lens[idx]], ch_titles[idx][0..out.chapter_title_lens[idx]]); + } + out.has_more = out.result_count > 0 and more_available.load(.acquire); + out.loading_more = loading_more.load(.acquire); + out.search_generation = search_request.current(); + out.chapter_generation = chapters_gen.load(.acquire); + out.text_generation = text_gen.load(.acquire); +} + pub fn resultCount() usize { parse_mutex.lock(); defer parse_mutex.unlock(); @@ -147,6 +239,7 @@ pub fn resultRow(i: usize) ?ListRow { out.url_len = @min(nr_url_lens[i], out.url_buf.len); @memcpy(out.url_buf[0..out.url_len], nr_urls[i][0..out.url_len]); out.source = @intFromEnum(nr_source[i]); + out.metadata = nr_metadata[i]; return out; } @@ -181,9 +274,13 @@ pub fn chapterRow(i: usize) ?ListRow { var current_page: u32 = 1; var more_available: std.atomic.Value(bool) = std.atomic.Value(bool).init(true); var loading_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(false); +var wiki_next_offset: u32 = 0; // protected by parse_mutex; API cursor, never deduplicated row count var wiki_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(true); var madara_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(true); var lnwp_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(true); +var gutenberg_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(true); +var openlibrary_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(true); +var gutenberg_next: u32 = 1; // authoritative provider cursor, parse_mutex var archive_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(true); // readwn's search POST has no page parameter β€” it returns one fixed set, so it is // never re-fetched by loadMore (implicitly exhausted after page 1). @@ -203,6 +300,24 @@ var chapter_snap_len: usize = 0; var chapter_url_snap: [1024]u8 = undefined; var chapter_url_snap_len: usize = 0; +const ReaderJob = struct { + generation: u32, + source: NovelSource, + title: [256]u8 = undefined, + title_len: usize = 0, + url: [1024]u8 = undefined, + url_len: usize = 0, +}; + +fn readerJob(generation: u32, title: []const u8, url: []const u8) ReaderJob { + var job: ReaderJob = .{ .generation = generation, .source = open_source }; + job.title_len = @min(title.len, job.title.len); + job.url_len = @min(url.len, job.url.len); + @memcpy(job.title[0..job.title_len], title[0..job.title_len]); + @memcpy(job.url[0..job.url_len], url[0..job.url_len]); + return job; +} + const SearchJob = struct { generation: u32, query: [256]u8 = undefined, @@ -220,14 +335,15 @@ const TEXT_CAP: usize = 131072; pub fn searchNovels(query: []const u8) void { if (query.len == 0 or query.len >= query_snap.len) return; - state.app.novels.fetch_error = false; - state.app.novels.view = .search; - const my_gen = search_request.begin(&state.app.novels.is_loading); const n = @min(query.len, query_snap.len); parse_mutex.lock(); + state.app.novels.fetch_error = false; + state.app.novels.view = .search; @memcpy(query_snap[0..n], query[0..n]); query_snap_len = n; + wiki_next_offset = 0; + gutenberg_next = 1; parse_mutex.unlock(); var job: SearchJob = .{ .generation = my_gen, .query_len = n }; @memcpy(job.query[0..n], query[0..n]); @@ -239,6 +355,8 @@ pub fn searchNovels(query: []const u8) void { madara_more.store(true, .release); lnwp_more.store(true, .release); archive_more.store(true, .release); + gutenberg_more.store(true, .release); + openlibrary_more.store(true, .release); if (@import("../core/workers.zig").spawnLegacy(searchWorker, .{job})) |t| { @import("../core/workers.zig").release(t); @@ -256,6 +374,10 @@ fn searchWorker(job: SearchJob) void { const query = job.query[0..job.query_len]; parse_mutex.lock(); + if (!search_request.isCurrent(my_gen)) { + parse_mutex.unlock(); + return; + } nr_count = 0; parse_mutex.unlock(); @@ -263,6 +385,11 @@ fn searchWorker(job: SearchJob) void { filled += fetchWikisource(query, my_gen, filled, 0); if (!search_request.isCurrent(my_gen)) return; + filled += fetchGutenberg(query, my_gen, filled, 1); + if (!search_request.isCurrent(my_gen)) return; + filled += fetchOpenLibrary(query, my_gen, filled, 1); + if (!search_request.isCurrent(my_gen)) return; + // A second legal, zero-configuration source. Reuse the same Internet // Archive parser and rate-limit bucket as universal search; only items with // a readable full-text derivative are returned by the Archive query. @@ -304,6 +431,7 @@ fn addResult(idx: usize, src: NovelSource, title: []const u8, url: []const u8) v @memcpy(nr_urls[idx][0..ulen], url[0..ulen]); nr_url_lens[idx] = ulen; nr_source[idx] = src; + nr_metadata[idx] = .{}; } /// True when a row already present in nr_*[0..upto) matches the candidate β€” the @@ -323,19 +451,6 @@ fn rowExists(src: NovelSource, title: []const u8, url: []const u8, upto: usize) return false; } -/// Count the Wikisource rows currently in nr_*[0..nr_count) β€” the `sroffset` for -/// the next Wikisource page (how many of its results we've already consumed). -/// Caller must hold `parse_mutex`. -fn wikiCountLocked() u32 { - var c: u32 = 0; - var i: usize = 0; - const cap = @min(nr_count, MAX_RESULTS); - while (i < cap) : (i += 1) { - if (nr_source[i] == .wikisource) c += 1; - } - return c; -} - /// Decode HTML entities out of a scraped title (it carries no block tags, so /// htmlToText just entity-decodes + collapses whitespace). Into `out`. fn cleanTitle(raw: []const u8, out: []u8) []const u8 { @@ -372,14 +487,39 @@ fn fetchExpandedNovel(query: []const u8, gen: u32, start: usize, page: u32, src: const abs = expanded.html.sourceUrl(&abs_buf, base, item.url) orelse continue; var title_buf: [256]u8 = undefined; const title = cleanTitle(item.title, &title_buf); - if (title.len == 0 or rowExists(src, title, abs, start + n)) continue; + if (title.len == 0) continue; + if (rowExists(src, title, abs, start + n)) { + // Image and title anchors for the same work are often separate. + // Merge a real cover from either anchor before discarding the link. + for (0..start + n) |existing| { + if (std.mem.eql(u8, nr_urls[existing][0..nr_url_lens[existing]], abs)) { + setNovelCover(&nr_metadata[existing], base, item.cover); + break; + } + } + continue; + } addResult(start + n, src, title, abs); + setNovelCover(&nr_metadata[start + n], base, item.cover); n += 1; } nr_count = start + n; return n; } +fn setNovelCover(meta: *NovelMetadata, base: []const u8, raw: []const u8) void { + if (meta.cover_len > 0 or raw.len == 0 or std.mem.indexOfAny(u8, raw, "\r\n") != null) return; + // Providers use external image CDNs. Only permit network schemes; ordinary + // relative image links still resolve against the installed source base. + const cover = if (std.mem.startsWith(u8, raw, "https://") or std.mem.startsWith(u8, raw, "http://")) + std.fmt.bufPrint(&meta.cover_buf, "{s}", .{raw}) catch return + else if (std.mem.startsWith(u8, raw, "//")) + std.fmt.bufPrint(&meta.cover_buf, "https:{s}", .{raw}) catch return + else + expanded.html.sourceUrl(&meta.cover_buf, base, raw) orelse return; + meta.cover_len = cover.len; +} + /// Wikisource `list=search` β†’ nr_* rows (the always-on default source). `offset` /// is the `sroffset` continuation (0 for the first page); rows are appended at /// `start` and deduped by title so infinite-scroll pages never repeat a work. @@ -397,7 +537,13 @@ fn fetchWikisource(query: []const u8, my_gen: u32, start: usize, offset: u32) us defer parse_mutex.unlock(); if (search_request.current() != my_gen) return 0; - const arr = pure.searchArray(body) orelse return 0; + const arr = pure.searchArray(body) orelse { + state.app.novels.fetch_error = true; + return 0; + }; + const next_offset = pure.searchContinuation(body); + wiki_next_offset = next_offset orelse offset; + wiki_more.store(next_offset != null and wiki_next_offset > offset, .release); var it = pure.cj.ObjIter{ .buf = arr }; var n: usize = 0; while (it.next()) |obj| { @@ -408,6 +554,12 @@ fn fetchWikisource(query: []const u8, my_gen: u32, start: usize, offset: u32) us if (dn == 0) continue; if (rowExists(.wikisource, dec[0..dn], "", start + n)) continue; addResult(start + n, .wikisource, dec[0..dn], ""); + if (pure.cj.findJsonStr(obj, "\"snippet\":\"")) |snippet| { + var snippet_buf: [2048]u8 = undefined; + const decoded = pure.cj.jsonUnescape(snippet, &snippet_buf); + const meta = &nr_metadata[start + n]; + meta.overview_len = pure.htmlToText(snippet_buf[0..decoded], &meta.overview_buf); + } n += 1; } nr_count = start + n; @@ -425,7 +577,7 @@ fn fetchInternetArchive(query: []const u8, my_gen: u32, start: usize, page: u32) var url_buf: [1400]u8 = undefined; const url = std.fmt.bufPrint( &url_buf, - "https://archive.org/advancedsearch.php?q={s}&fl[]=identifier&fl[]=title&fl[]=year&rows={d}&page={d}&output=json", + "https://archive.org/advancedsearch.php?q={s}&fl[]=identifier&fl[]=title&fl[]=year&fl[]=creator&fl[]=description&rows={d}&page={d}&output=json", .{ encoded, PAGE_SIZE, page }, ) catch return 0; @import("../core/rate_limit.zig").acquire("archive", 1.0); @@ -437,19 +589,108 @@ fn fetchInternetArchive(query: []const u8, my_gen: u32, start: usize, page: u32) defer parse_mutex.unlock(); if (!search_request.isCurrent(my_gen)) return 0; - var it = archive.iterateDocs(body); + const parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{ .ignore_unknown_fields = true }) catch return 0; + defer parsed.deinit(); + if (parsed.value != .object) return 0; + const response = parsed.value.object.get("response") orelse return 0; + if (response != .object) return 0; + const docs = response.object.get("docs") orelse return 0; + if (docs != .array) return 0; var n: usize = 0; - while (it.next()) |doc| { + for (docs.array.items) |doc| { if (start + n >= MAX_RESULTS) break; - var id_buf: [512]u8 = undefined; - const id_len = pure.cj.jsonUnescape(doc.identifier, &id_buf); - if (id_len == 0) continue; - var title_buf: [256]u8 = undefined; - const title_len = if (doc.title.len > 0) pure.cj.jsonUnescape(doc.title, &title_buf) else 0; - const title = if (title_len > 0) title_buf[0..title_len] else id_buf[0..id_len]; - const id = id_buf[0..id_len]; + if (doc != .object) continue; + const id = metadataString(doc.object.get("identifier")); + if (id.len == 0 or id.len > nr_urls[0].len) continue; + const raw_title = metadataString(doc.object.get("title")); + const title = if (raw_title.len > 0) raw_title else id; if (rowExists(.internet_archive, title, id, start + n)) continue; addResult(start + n, .internet_archive, title, id); + const meta = &nr_metadata[start + n]; + const author = metadataString(doc.object.get("creator")); + meta.author_len = pure.htmlToText(author, &meta.author_buf); + meta.overview_len = pure.htmlToText(metadataString(doc.object.get("description")), &meta.overview_buf); + const year_value = doc.object.get("year"); + if (year_value) |value| { + if (value == .integer and value.integer > 0 and value.integer <= 9999) meta.year = @intCast(value.integer) else if (value == .string) meta.year = std.fmt.parseInt(u16, value.string, 10) catch 0; + } + var encoded_id: [1536]u8 = undefined; + const encoded_len = pure.cj.percentEncodeStrict(id, &encoded_id); + const cover = std.fmt.bufPrint(&meta.cover_buf, "https://archive.org/services/img/{s}", .{encoded_id[0..encoded_len]}) catch ""; + meta.cover_len = cover.len; + n += 1; + } + nr_count = start + n; + return n; +} + +/// Archive fields can be scalar strings or arrays of contributor/description +/// values. Preserve the first actual value; absent fields stay absent. +fn metadataString(value: ?std.json.Value) []const u8 { + const v = value orelse return ""; + if (v == .string) return v.string; + if (v == .array) for (v.array.items) |entry| { + if (entry == .string and entry.string.len > 0) return entry.string; + }; + return ""; +} + +fn fetchGutenberg(query: []const u8, gen: u32, start: usize, cursor: u32) usize { + var url_buf: [2048]u8 = undefined; + const url = books.gutenbergSearch(&url_buf, query, cursor) orelse return 0; + @import("../core/rate_limit.zig").acquire("gutenberg", 1.0); + const body = fetchBody(url, 1024 * 1024) orelse return 0; + defer alloc.free(body); + parse_mutex.lock(); + defer parse_mutex.unlock(); + if (!search_request.isCurrent(gen)) return 0; + const next = books.gutenbergNext(body); + gutenberg_more.store(next != null and next.? > cursor, .release); + gutenberg_next = next orelse cursor; + var it = books.GutenbergIter{ .body = body }; + var n: usize = 0; + while (it.next()) |item| { + if (start + n >= MAX_RESULTS) break; + var title_buf: [256]u8 = undefined; + const title = cleanTitle(item.title, &title_buf); + if (title.len == 0 or rowExists(.gutenberg, title, item.id, start + n)) continue; + addResult(start + n, .gutenberg, title, item.id); + const meta = &nr_metadata[start + n]; + meta.author_len = pure.htmlToText(item.author, &meta.author_buf); + setNovelCover(meta, "https://www.gutenberg.org", item.cover); + n += 1; + } + nr_count = start + n; + return n; +} + +fn fetchOpenLibrary(query: []const u8, gen: u32, start: usize, page: u32) usize { + var url_buf: [2048]u8 = undefined; + const url = books.openLibrarySearch(&url_buf, query, page) orelse return 0; + @import("../core/rate_limit.zig").acquire("openlibrary", 1.0); + const body = fetchBody(url, 1024 * 1024) orelse return 0; + defer alloc.free(body); + const parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{ .ignore_unknown_fields = true }) catch return 0; + defer parsed.deinit(); + if (parsed.value != .object) return 0; + const docs = parsed.value.object.get("docs") orelse return 0; + if (docs != .array) return 0; + parse_mutex.lock(); + defer parse_mutex.unlock(); + if (!search_request.isCurrent(gen)) return 0; + const total = parsed.value.object.get("numFound"); + const has_more = if (total != null and total.? == .integer) total.?.integer > @as(i64, page) * 20 else docs.array.items.len == 20; + openlibrary_more.store(has_more, .release); + var n: usize = 0; + for (docs.array.items) |doc| { + if (start + n >= MAX_RESULTS) break; + const item = books.openLibraryItem(doc) orelse continue; + if (rowExists(.openlibrary, item.title, item.id, start + n)) continue; + addResult(start + n, .openlibrary, item.title, item.id); + const meta = &nr_metadata[start + n]; + meta.author_len = pure.htmlToText(item.author, &meta.author_buf); + meta.year = item.year; + if (books.openLibraryCover(&meta.cover_buf, item.cover_id)) |cover| meta.cover_len = cover.len; n += 1; } nr_count = start + n; @@ -574,6 +815,29 @@ fn fetchReadwn(query: []const u8, my_gen: u32, start: usize) usize { /// fresh query supersedes it. Wikisource continues by `sroffset`; the scraper /// engines by page number (`current_page + 1`); readwn's search can't page and is /// never re-fetched here. +pub fn hasMoreResults() bool { + return more_available.load(.acquire); +} +pub fn loadingMoreResults() bool { + return loading_more.load(.acquire); +} + +/// Return through retained reader/catalog snapshots instead of re-opening a +/// stale search row or issuing an empty search after a universal-search open. +pub fn back() void { + parse_mutex.lock(); + defer parse_mutex.unlock(); + if (state.app.novels.view == .reader) { + _ = text_gen.fetchAdd(1, .acq_rel); + state.app.novels.text_loading.store(false, .release); + state.app.novels.view = .chapters; + } else { + _ = chapters_gen.fetchAdd(1, .acq_rel); + state.app.novels.chapters_loading.store(false, .release); + state.app.novels.view = .search; + } +} + pub fn loadMore() void { if (!more_available.load(.acquire)) return; if (state.app.novels.is_loading.load(.acquire)) return; @@ -621,12 +885,27 @@ fn loadMoreWorker(job: SearchJob, next_page: u32) void { if (wiki_more.load(.acquire)) { parse_mutex.lock(); - const offset = wikiCountLocked(); + const offset = wiki_next_offset; parse_mutex.unlock(); const n = fetchWikisource(query, my_gen, start, offset); if (search_request.current() != my_gen) return; start += n; - if (n == 0) wiki_more.store(false, .release); + // The server continuation determines exhaustion, even if this page + // contains only duplicate or unsupported rows. + + } + if (gutenberg_more.load(.acquire) and start < MAX_RESULTS) { + parse_mutex.lock(); + const cursor = gutenberg_next; + parse_mutex.unlock(); + const n = fetchGutenberg(query, my_gen, start, cursor); + if (!search_request.isCurrent(my_gen)) return; + start += n; + } + if (openlibrary_more.load(.acquire) and start < MAX_RESULTS) { + const n = fetchOpenLibrary(query, my_gen, start, next_page); + if (!search_request.isCurrent(my_gen)) return; + start += n; } if (archive_more.load(.acquire) and start < MAX_RESULTS) { const n = fetchInternetArchive(query, my_gen, start, next_page); @@ -655,6 +934,7 @@ fn loadMoreWorker(job: SearchJob, next_page: u32) void { } const any = expanded_more or wiki_more.load(.acquire) or archive_more.load(.acquire) or + gutenberg_more.load(.acquire) or openlibrary_more.load(.acquire) or (madara_more.load(.acquire) and madaraNovelBase() != null) or (lnwp_more.load(.acquire) and lightnovelwpBase() != null); more_available.store(any and start < MAX_RESULTS, .release); @@ -667,6 +947,8 @@ fn loadMoreWorker(job: SearchJob, next_page: u32) void { pub fn openNovel(idx: usize) void { const row = resultRow(idx) orelse return; + parse_mutex.lock(); + const my_gen = chapters_gen.fetchAdd(1, .acq_rel) + 1; // Snapshot the selected work (source + title + URL) BEFORE spawning β€” the // search grid can be reordered by a fresh search while chapters load. @@ -682,16 +964,17 @@ pub fn openNovel(idx: usize) void { @memcpy(work_url_snap[0..ulen], row.url_buf[0..ulen]); work_url_snap_len = ulen; + _ = text_gen.fetchAdd(1, .acq_rel); + state.app.novels.text_loading.store(false, .release); state.app.novels.view = .chapters; state.app.novels.chapters_loading.store(true, .release); state.app.novels.fetch_error = false; - parse_mutex.lock(); ch_count = 0; - parse_mutex.unlock(); - const my_gen = chapters_gen.fetchAdd(1, .acq_rel) + 1; - if (@import("../core/workers.zig").spawnLegacy(chaptersWorker, .{my_gen})) |t| { + const job = readerJob(my_gen, work_snap[0..work_snap_len], work_url_snap[0..work_url_snap_len]); + parse_mutex.unlock(); + if (@import("../core/workers.zig").spawnLegacy(chaptersWorker, .{job})) |t| { @import("../core/workers.zig").release(t); } else |_| { state.app.novels.chapters_loading.store(false, .release); @@ -699,25 +982,37 @@ pub fn openNovel(idx: usize) void { } /// Dispatch the chapter-list fetch to the open novel's engine. -fn chaptersWorker(my_gen: u32) void { - defer state.app.novels.chapters_loading.store(false, .release); - switch (open_source) { - .wikisource => chaptersWikisource(my_gen), - .madara_novel => chaptersMadara(my_gen), - .lightnovelwp => chaptersLightnovelwp(my_gen), - .readwn => chaptersReadwn(my_gen), +fn chaptersWorker(job: ReaderJob) void { + const my_gen = job.generation; + defer { + parse_mutex.lock(); + if (chapters_gen.load(.acquire) == my_gen) state.app.novels.chapters_loading.store(false, .release); + parse_mutex.unlock(); + } + if (chapters_gen.load(.acquire) != my_gen) return; + switch (job.source) { + .wikisource => chaptersWikisource(job), + .madara_novel => chaptersMadara(job), + .lightnovelwp => chaptersLightnovelwp(job), + .readwn => chaptersReadwn(job), .readnovelfull => {}, // not shipped in v1 - .internet_archive => chaptersInternetArchive(my_gen), - .royalroad, .novelfire => chaptersExpanded(my_gen), + .internet_archive, .openlibrary => chaptersInternetArchive(job), + .gutenberg => chaptersGutenberg(job), + .royalroad, .novelfire => chaptersExpanded(job), } + parse_mutex.lock(); + const capped = chapters_gen.load(.acquire) == my_gen and ch_count >= MAX_CHAPTERS; + parse_mutex.unlock(); + if (capped) logs.pushLog("warn", "novels", "Chapter list reached the 400-chapter reader limit", false); } -fn chaptersExpanded(gen: u32) void { - const src = open_source; +fn chaptersExpanded(job: ReaderJob) void { + const gen = job.generation; + const src = job.source; var base_buf: [512]u8 = undefined; const base = copyBase(source_config.get(@tagName(src), "base") orelse return, &base_buf); var work_buf: [1024]u8 = undefined; - const work = copyBase(work_url_snap[0..work_url_snap_len], &work_buf); + const work = copyBase(job.url[0..job.url_len], &work_buf); var n: usize = 0; // NovelFire paginates its chapter directory; Royal Road returns it whole. var page: usize = 1; @@ -785,16 +1080,17 @@ fn reverseChapters(count: usize) void { /// Wikisource: `list=allpages` subpages. When a work has none (single-page work), /// synthesize one chapter = the work page so the reader still opens. -fn chaptersWikisource(my_gen: u32) void { +fn chaptersWikisource(job: ReaderJob) void { + const my_gen = job.generation; var work: [256]u8 = undefined; - const wlen = @min(work_snap_len, work.len); - @memcpy(work[0..wlen], work_snap[0..wlen]); + const wlen = @min(job.title_len, work.len); + @memcpy(work[0..wlen], job.title[0..wlen]); var url_buf: [1024]u8 = undefined; const url = pure.buildSubpagesUrl(&url_buf, work[0..wlen], MAX_CHAPTERS) orelse return; const body = fetchBody(url, 512 * 1024) orelse { - state.app.novels.fetch_error = true; + if (chapters_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; }; defer alloc.free(body); @@ -827,19 +1123,20 @@ fn chaptersWikisource(my_gen: u32) void { /// Madara-novel: REUSES the manga Madara `ChapterIter` over the details HTML, with /// the same admin-ajax.php / `{url}ajax/chapters` fallbacks the comics reader uses. -fn chaptersMadara(my_gen: u32) void { +fn chaptersMadara(job: ReaderJob) void { + const my_gen = job.generation; const base_raw = madaraNovelBase() orelse return; var base_buf: [256]u8 = undefined; const base = copyBase(base_raw, &base_buf); var murl_buf: [512]u8 = undefined; - const mn = @min(work_url_snap_len, murl_buf.len); - @memcpy(murl_buf[0..mn], work_url_snap[0..mn]); + const mn = @min(job.url_len, murl_buf.len); + @memcpy(murl_buf[0..mn], job.url[0..mn]); const murl = murl_buf[0..mn]; if (murl.len == 0) return; const body = scrapeHtml(murl, 1024 * 1024) orelse { - state.app.novels.fetch_error = true; + if (chapters_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; }; defer alloc.free(body); @@ -902,19 +1199,20 @@ fn chaptersMadara(my_gen: u32) void { } /// lightnovelwp: REUSES the MangaThemesia `chapterIter` over the series HTML. -fn chaptersLightnovelwp(my_gen: u32) void { +fn chaptersLightnovelwp(job: ReaderJob) void { + const my_gen = job.generation; const base_raw = lightnovelwpBase() orelse return; var base_buf: [256]u8 = undefined; const base = copyBase(base_raw, &base_buf); var murl_buf: [512]u8 = undefined; - const mn = @min(work_url_snap_len, murl_buf.len); - @memcpy(murl_buf[0..mn], work_url_snap[0..mn]); + const mn = @min(job.url_len, murl_buf.len); + @memcpy(murl_buf[0..mn], job.url[0..mn]); const murl = murl_buf[0..mn]; if (murl.len == 0) return; const body = scrapeHtml(murl, 1024 * 1024) orelse { - state.app.novels.fetch_error = true; + if (chapters_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; }; defer alloc.free(body); @@ -943,19 +1241,20 @@ fn chaptersLightnovelwp(my_gen: u32) void { /// readwn: the standalone `.chapter-list` iterator. readwn lists chapters /// ascending already, so no reversal. -fn chaptersReadwn(my_gen: u32) void { +fn chaptersReadwn(job: ReaderJob) void { + const my_gen = job.generation; const base_raw = readwnBase() orelse return; var base_buf: [256]u8 = undefined; const base = copyBase(base_raw, &base_buf); var murl_buf: [512]u8 = undefined; - const mn = @min(work_url_snap_len, murl_buf.len); - @memcpy(murl_buf[0..mn], work_url_snap[0..mn]); + const mn = @min(job.url_len, murl_buf.len); + @memcpy(murl_buf[0..mn], job.url[0..mn]); const murl = murl_buf[0..mn]; if (murl.len == 0) return; const body = scrapeHtml(murl, 1024 * 1024) orelse { - state.app.novels.fetch_error = true; + if (chapters_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; }; defer alloc.free(body); @@ -981,13 +1280,31 @@ fn chaptersReadwn(my_gen: u32) void { logs.pushLog("info", "novels", "Novel chapter list loaded (readwn)", false); } +fn chaptersGutenberg(job: ReaderJob) void { + var url_buf: [512]u8 = undefined; + const url = books.gutenbergDetail(&url_buf, job.url[0..job.url_len]) orelse return; + @import("../core/rate_limit.zig").acquire("gutenberg", 1.0); + const body = fetchBody(url, 1024 * 1024) orelse return; + defer alloc.free(body); + if (chapters_gen.load(.acquire) != job.generation) return; + var text_url_buf: [1024]u8 = undefined; + const text_url = books.gutenbergTextUrl(&text_url_buf, body) orelse return; + parse_mutex.lock(); + defer parse_mutex.unlock(); + if (chapters_gen.load(.acquire) != job.generation) return; + addChapter(0, "Full text", text_url); + ch_count = 1; + logs.pushLog("info", "novels", "Gutenberg advertised UTF-8 text resolved", false); +} + /// Internet Archive: resolve one real readable file from item metadata and /// expose it as a single "Full text" chapter. This keeps the existing reader, /// resume, and deep-link paths shared with every other novel source. -fn chaptersInternetArchive(my_gen: u32) void { +fn chaptersInternetArchive(job: ReaderJob) void { + const my_gen = job.generation; var id_buf: [512]u8 = undefined; - const id_len = @min(work_url_snap_len, id_buf.len); - @memcpy(id_buf[0..id_len], work_url_snap[0..id_len]); + const id_len = @min(job.url_len, id_buf.len); + @memcpy(id_buf[0..id_len], job.url[0..id_len]); if (id_len == 0) return; var enc_id_buf: [1536]u8 = undefined; @@ -996,19 +1313,22 @@ fn chaptersInternetArchive(my_gen: u32) void { const meta_url = std.fmt.bufPrint(&meta_url_buf, "https://archive.org/metadata/{s}", .{enc_id_buf[0..enc_id_len]}) catch return; @import("../core/rate_limit.zig").acquire("archive", 1.0); const metadata = fetchBody(meta_url, 2 * 1024 * 1024) orelse { - state.app.novels.fetch_error = true; + if (chapters_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; }; defer alloc.free(metadata); if (chapters_gen.load(.acquire) != my_gen) return; - const raw_file = archive.pickBestTextFile(metadata) orelse { - state.app.novels.fetch_error = true; + const parsed_metadata = std.json.parseFromSlice(std.json.Value, alloc, metadata, .{ .ignore_unknown_fields = true }) catch return; + defer parsed_metadata.deinit(); + const raw_file = books.publicArchiveText(parsed_metadata.value) orelse { + if (chapters_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; logs.pushLog("info", "novels", "Archive item has no readable text file", false); return; }; var file_buf: [512]u8 = undefined; - const file_len = pure.cj.jsonUnescape(raw_file, &file_buf); + const file_len = @min(raw_file.len, file_buf.len); + @memcpy(file_buf[0..file_len], raw_file[0..file_len]); if (file_len == 0) return; var enc_file_buf: [1536]u8 = undefined; const enc_file_len = archive.encodePathSegment(file_buf[0..file_len], &enc_file_buf); @@ -1033,6 +1353,8 @@ fn chaptersInternetArchive(my_gen: u32) void { pub fn openChapter(idx: usize) void { const row = chapterRow(idx) orelse return; + parse_mutex.lock(); + const my_gen = text_gen.fetchAdd(1, .acq_rel) + 1; state.app.novels.current_chapter = idx; state.app.novels.view = .reader; @@ -1059,8 +1381,9 @@ pub fn openChapter(idx: usize) void { // Persist resume: this is now the last-read chapter for this work. saveResume(idx); - const my_gen = text_gen.fetchAdd(1, .acq_rel) + 1; - if (@import("../core/workers.zig").spawnLegacy(textWorker, .{my_gen})) |t| { + const job = readerJob(my_gen, chapter_snap[0..chapter_snap_len], chapter_url_snap[0..chapter_url_snap_len]); + parse_mutex.unlock(); + if (@import("../core/workers.zig").spawnLegacy(textWorker, .{job})) |t| { @import("../core/workers.zig").release(t); } else |_| { state.app.novels.text_loading.store(false, .release); @@ -1078,28 +1401,37 @@ pub fn prevChapter() void { } /// Dispatch the chapter-text fetch to the open novel's engine. -fn textWorker(my_gen: u32) void { - defer state.app.novels.text_loading.store(false, .release); - if (open_source == .wikisource) { - textWikisource(my_gen); - } else if (open_source == .internet_archive) { - textInternetArchive(my_gen); +fn textWorker(job: ReaderJob) void { + const my_gen = job.generation; + defer { + parse_mutex.lock(); + if (text_gen.load(.acquire) == my_gen) state.app.novels.text_loading.store(false, .release); + parse_mutex.unlock(); + } + if (text_gen.load(.acquire) != my_gen) return; + if (job.source == .wikisource) { + textWikisource(job); + } else if (job.source == .internet_archive or job.source == .openlibrary) { + textInternetArchive(job); + } else if (job.source == .gutenberg) { + textGutenberg(job); } else { - textSourced(my_gen); + textSourced(job); } } /// Wikisource: `action=parse&prop=text` β†’ JSON `parse.text` HTML β†’ reading text. -fn textWikisource(my_gen: u32) void { +fn textWikisource(job: ReaderJob) void { + const my_gen = job.generation; var page: [256]u8 = undefined; - const plen = @min(chapter_snap_len, page.len); - @memcpy(page[0..plen], chapter_snap[0..plen]); + const plen = @min(job.title_len, page.len); + @memcpy(page[0..plen], job.title[0..plen]); var url_buf: [1024]u8 = undefined; const url = pure.buildChapterUrl(&url_buf, page[0..plen]) orelse return; const body = fetchBody(url, 2 * 1024 * 1024) orelse { - state.app.novels.fetch_error = true; + if (text_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; }; defer alloc.free(body); @@ -1108,13 +1440,13 @@ fn textWikisource(my_gen: u32) void { // Two-stage, both heap/state (never a big buffer on the worker stack): // JSON parse.text β†’ HTML (heap) β†’ clean reading text (state.text_buf). const html = alloc.alloc(u8, 2 * 1024 * 1024) catch { - state.app.novels.fetch_error = true; + if (text_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; }; defer alloc.free(html); const html_len = pure.extractParseHtml(body, html); if (html_len == 0) { - state.app.novels.fetch_error = true; + if (text_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; logs.pushLog("info", "novels", "Chapter had no extractable text", false); return; } @@ -1133,18 +1465,19 @@ fn textWikisource(my_gen: u32) void { /// Internet Archive's selected derivative is already plain text. Run it /// through the common whitespace/entity normalizer so OCR reads like the other /// sources and remains bounded by the same fixed reader buffer. -fn textInternetArchive(my_gen: u32) void { +fn textInternetArchive(job: ReaderJob) void { + const my_gen = job.generation; var url_buf: [1024]u8 = undefined; - const url_len = @min(chapter_url_snap_len, url_buf.len); - @memcpy(url_buf[0..url_len], chapter_url_snap[0..url_len]); + const url_len = @min(job.url_len, url_buf.len); + @memcpy(url_buf[0..url_len], job.url[0..url_len]); const url = url_buf[0..url_len]; if (url.len == 0 or !std.mem.startsWith(u8, url, "https://archive.org/download/")) { - state.app.novels.fetch_error = true; + if (text_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; } @import("../core/rate_limit.zig").acquire("archive", 1.0); const body = fetchBody(url, 2 * 1024 * 1024) orelse { - state.app.novels.fetch_error = true; + if (text_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; }; defer alloc.free(body); @@ -1156,32 +1489,51 @@ fn textInternetArchive(my_gen: u32) void { const n = pure.htmlToText(body, state.app.novels.text_buf[0..TEXT_CAP]); state.app.novels.text_len = n; state.app.novels.text_truncated = n >= TEXT_CAP; - if (n == 0) state.app.novels.fetch_error = true; + if (n == 0 and text_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; logs.pushLog("info", "novels", "Novel full text loaded (Internet Archive)", false); } +fn textGutenberg(job: ReaderJob) void { + const url = job.url[0..job.url_len]; + if (!std.mem.startsWith(u8, url, "https://www.gutenberg.org/")) return; + @import("../core/rate_limit.zig").acquire("gutenberg", 1.0); + const body = fetchBody(url, 4 * 1024 * 1024) orelse return; + defer alloc.free(body); + parse_mutex.lock(); + defer parse_mutex.unlock(); + if (text_gen.load(.acquire) != job.generation) return; + // A plain-text book is not HTML: preserve comparison signs, indentation + // and paragraph breaks rather than stripping text between '<' and '>'. + const text = books.plainTextPrefix(body, TEXT_CAP); + @memcpy(state.app.novels.text_buf[0..text.len], text); + state.app.novels.text_len = text.len; + state.app.novels.text_truncated = body.len > TEXT_CAP; + if (text.len == 0) state.app.novels.fetch_error = true; +} + /// Scraper engines: GET the chapter URL, extract the source's prose container /// (via novel_sources_pure), then HTMLβ†’clean text. The container selector is the /// ONLY per-engine difference; everything else is the shared reader pipeline. -fn textSourced(my_gen: u32) void { +fn textSourced(job: ReaderJob) void { + const my_gen = job.generation; var url_buf: [1024]u8 = undefined; - const un = @min(chapter_url_snap_len, url_buf.len); - @memcpy(url_buf[0..un], chapter_url_snap[0..un]); + const un = @min(job.url_len, url_buf.len); + @memcpy(url_buf[0..un], job.url[0..un]); const chapter_url = url_buf[0..un]; if (chapter_url.len == 0 or !std.mem.startsWith(u8, chapter_url, "http")) { - state.app.novels.fetch_error = true; + if (text_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; } const body = scrapeHtml(chapter_url, 2 * 1024 * 1024) orelse { - state.app.novels.fetch_error = true; + if (text_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; return; }; defer alloc.free(body); if (text_gen.load(.acquire) != my_gen) return; - const content = nsp.chapterContentHtml(body, open_source) orelse { - state.app.novels.fetch_error = true; + const content = nsp.chapterContentHtml(body, job.source) orelse { + if (text_gen.load(.acquire) == my_gen) state.app.novels.fetch_error = true; logs.pushLog("info", "novels", "Chapter had no extractable text", false); return; }; @@ -1245,32 +1597,42 @@ fn saveResume(chapter: usize) void { pub fn openDeepLink(link: []const u8) void { const parsed = pure.parseDeepLink(link) orelse return; const src = std.meta.stringToEnum(NovelSource, parsed.source) orelse return; + openCatalogResult(@intFromEnum(src), parsed.title, parsed.url); +} +/// Open an immutable universal-search identity without touching the browse +/// query/results or relying on their mutable row indices. Owner-thread only. +pub fn openCatalogResult(source: u8, title: []const u8, url: []const u8) void { + const src = std.enums.fromInt(NovelSource, source) orelse return; + if (title.len == 0 or title.len > work_snap.len or url.len > work_url_snap.len) return; + parse_mutex.lock(); + const my_gen = chapters_gen.fetchAdd(1, .acq_rel) + 1; open_source = src; - const tlen = @min(parsed.title.len, state.app.novels.work_title.len); - @memcpy(state.app.novels.work_title[0..tlen], parsed.title[0..tlen]); + const tlen = @min(title.len, state.app.novels.work_title.len); + @memcpy(state.app.novels.work_title[0..tlen], title[0..tlen]); state.app.novels.work_title_len = tlen; - @memcpy(work_snap[0..tlen], parsed.title[0..tlen]); + @memcpy(work_snap[0..tlen], title[0..tlen]); work_snap_len = tlen; - const ulen = @min(parsed.url.len, work_url_snap.len); - @memcpy(work_url_snap[0..ulen], parsed.url[0..ulen]); + const ulen = @min(url.len, work_url_snap.len); + @memcpy(work_url_snap[0..ulen], url[0..ulen]); work_url_snap_len = ulen; + _ = text_gen.fetchAdd(1, .acq_rel); + state.app.novels.text_loading.store(false, .release); state.app.novels.view = .chapters; state.app.novels.chapters_loading.store(true, .release); state.app.novels.fetch_error = false; - parse_mutex.lock(); ch_count = 0; - parse_mutex.unlock(); state.app.browse_source = .Novels; state.app.router.navigate(.browse); - const my_gen = chapters_gen.fetchAdd(1, .acq_rel) + 1; - if (@import("../core/workers.zig").spawnLegacy(chaptersWorker, .{my_gen})) |t| { + const job = readerJob(my_gen, work_snap[0..work_snap_len], work_url_snap[0..work_url_snap_len]); + parse_mutex.unlock(); + if (@import("../core/workers.zig").spawnLegacy(chaptersWorker, .{job})) |t| { @import("../core/workers.zig").release(t); } else |_| { state.app.novels.chapters_loading.store(false, .release); @@ -1514,6 +1876,8 @@ fn novelSourceLabel(source: NovelSource) []const u8 { return switch (source) { .wikisource => "Wikisource", .internet_archive => "Internet Archive", + .gutenberg => "Project Gutenberg", + .openlibrary => "Open Library", .royalroad => "Royal Road", .novelfire => "NovelFire", else => "Connected source", @@ -1560,20 +1924,14 @@ fn renderNovelCard(item: ListRow, index: usize) void { { var spine = dvui.box(@src(), .{ .dir = .vertical }, .{ .id_extra = 54000 + index, - .min_size_content = .{ .w = 48, .h = 112 }, - .max_size_content = .{ .w = 48, .h = 112 }, + .min_size_content = .{ .w = 76, .h = 112 }, + .max_size_content = .{ .w = 76, .h = 112 }, .background = true, .color_fill = theme.colors.accent.lerp(theme.colors.bg_surface, 0.35), .corner_radius = .{ .x = theme.radius.md, .y = theme.radius.md, .w = 0, .h = 0 }, }); defer spine.deinit(); - dvui.icon(@src(), "novel-source", novelSourceIcon(source), .{}, .{ - .id_extra = index, - .color_text = theme.colors.text_on_accent, - .min_size_content = theme.iconSize(.lg), - .gravity_x = 0.5, - .gravity_y = 0.5, - }); + components.coverArt(@src(), 54500 + index, &novel_covers[index], item.metadata.cover(), novelSourceIcon(source), theme.radius.md); } { @@ -1589,6 +1947,22 @@ fn renderNovelCard(item: ListRow, index: usize) void { .color_text = theme.colors.text_primary, .font = dvui.themeGet().font_heading, }); + if (item.metadata.author_len > 0 or item.metadata.overview_len > 0) { + var subtitle_buf: [256]u8 = undefined; + const subtitle = if (item.metadata.author_len > 0) + safeUtf8(item.metadata.author()) + else + safeUtf8(item.metadata.overview()[0..@min(item.metadata.overview_len, 100)]); + const subtitle_text = if (item.metadata.year > 0) + std.fmt.bufPrint(&subtitle_buf, "{s} Β· {d}", .{ subtitle, item.metadata.year }) catch subtitle + else + subtitle; + _ = dvui.label(@src(), "{s}", .{subtitle_text}, .{ + .id_extra = 56500 + index, + .expand = .horizontal, + .color_text = theme.colors.text_secondary, + }); + } { var spacer = dvui.box(@src(), .{}, .{ .id_extra = 57000 + index, .expand = .vertical }); spacer.deinit(); diff --git a/src/services/novels_pure.zig b/src/services/novels_pure.zig index dcb7125f..184d2ec1 100644 --- a/src/services/novels_pure.zig +++ b/src/services/novels_pure.zig @@ -95,6 +95,20 @@ pub fn buildChapterUrl(out: []u8, page_title: []const u8) ?[]const u8 { ) catch null; } +/// MediaWiki's continuation cursor counts consumed server rows, including rows +/// the client filters or deduplicates. Absent continuation means exhausted. +pub fn searchContinuation(json: []const u8) ?u32 { + const continuation = cj.findJsonNode(json, "\"continue\"") orelse return null; + const key = std.mem.indexOf(u8, continuation, "\"sroffset\"") orelse return null; + const tail = continuation[key + "\"sroffset\"".len ..]; + const colon = std.mem.indexOfScalar(u8, tail, ':') orelse return null; + const number = std.mem.trimStart(u8, tail[colon + 1 ..], " \t\r\n"); + var end: usize = 0; + while (end < number.len and std.ascii.isDigit(number[end])) : (end += 1) {} + if (end == 0) return null; + return std.fmt.parseInt(u32, number[0..end], 10) catch null; +} + /// The `query.search[…]` array payload of a search response, or null. Walk it /// with `cj.ObjIter`; each object's display title is `titleField`. pub fn searchArray(json: []const u8) ?[]const u8 { @@ -208,10 +222,10 @@ fn decodeEntity(html: []const u8, i: usize, scratch: *[8]u8) ?Decoded { /// become a paragraph break in the extracted text. fn isParagraphTag(name: []const u8) bool { const blocks = [_][]const u8{ - "p", "div", "li", "ul", "ol", "blockquote", - "h1", "h2", "h3", "h4", "h5", "h6", - "tr", "table", "hr", "section", "article", "dd", - "dt", "dl", "pre", "figure", "header", "footer", + "p", "div", "li", "ul", "ol", "blockquote", + "h1", "h2", "h3", "h4", "h5", "h6", + "tr", "table", "hr", "section", "article", "dd", + "dt", "dl", "pre", "figure", "header", "footer", }; for (blocks) |b| if (std.ascii.eqlIgnoreCase(name, b)) return true; return false; diff --git a/src/services/opds.zig b/src/services/opds.zig index 4564d73c..fa499867 100644 --- a/src/services/opds.zig +++ b/src/services/opds.zig @@ -52,13 +52,77 @@ var entry_covers: [300]components.CoverSlot = [_]components.CoverSlot{.{}} ** 30 // entry_count is written last so a torn read shows fewer rows, never garbage. var parse_mutex: @import("../core/sync.zig").Mutex = .{}; +pub const ConnectionSnapshot = struct { + connected: bool = false, + server: [256]u8 = .{0} ** 256, + server_len: usize = 0, + user: [128]u8 = .{0} ** 128, + user_len: usize = 0, + pass: [128]u8 = .{0} ** 128, + pass_len: usize = 0, + identity: u64 = 0, +}; +var configured_connection: ConnectionSnapshot = .{}; + +/// Credentials are copied from the configured record, never the live form. +pub fn connectionSnapshot() ConnectionSnapshot { + parse_mutex.lock(); + defer parse_mutex.unlock(); + var result = configured_connection; + result.connected = state.app.opds.connected; + return result; +} + +pub fn configureConnection(server: []const u8, user: []const u8, pass: []const u8) void { + var next: ConnectionSnapshot = .{}; + next.server_len = @min(server.len, next.server.len - 1); + next.user_len = @min(user.len, next.user.len - 1); + next.pass_len = @min(pass.len, next.pass.len - 1); + @memcpy(next.server[0..next.server_len], server[0..next.server_len]); + @memcpy(next.user[0..next.user_len], user[0..next.user_len]); + @memcpy(next.pass[0..next.pass_len], pass[0..next.pass_len]); + var hash = std.hash.Wyhash.init(0x4f504453); + hash.update(next.server[0..next.server_len]); + hash.update("\x00"); + hash.update(next.user[0..next.user_len]); + hash.update("\x00"); + hash.update(next.pass[0..next.pass_len]); + next.identity = hash.final(); + parse_mutex.lock(); + configured_connection = next; + state.app.opds.server_url = next.server; + state.app.opds.server_url_len = next.server_len; + state.app.opds.user_buf = next.user; + state.app.opds.pass_buf = next.pass; + state.app.opds.connected = false; + parse_mutex.unlock(); + fetch_request.cancel(&state.app.opds.is_loading); +} +pub fn setConfiguredServer(value: []const u8) void { + const snap = connectionSnapshot(); + configureConnection(value, snap.user[0..snap.user_len], snap.pass[0..snap.pass_len]); +} +pub fn setConfiguredUser(value: []const u8) void { + const snap = connectionSnapshot(); + configureConnection(snap.server[0..snap.server_len], value, snap.pass[0..snap.pass_len]); +} +pub fn setConfiguredPassword(value: []const u8) void { + const snap = connectionSnapshot(); + configureConnection(snap.server[0..snap.server_len], snap.user[0..snap.user_len], value); +} +pub fn setConfiguredConnected(value: bool) void { + parse_mutex.lock(); + state.app.opds.connected = value and configured_connection.server_len > 0; + parse_mutex.unlock(); +} + // ── Infinite-scroll pagination ── // OPDS/Atom feeds carry a `` at the feed level // (opds_pure.feedNextHref extracts + resolves it). `more_available` and // `next_href_buf` are published by the SAME worker + mutex as entries/ // entry_count above, so a UI read under parse_mutex always sees a consistent // triple. `loading_more` serializes append fetches so a single near-bottom -// scroll can't spawn a burst (mirrors comics/drama/youtube). `fetch_gen` is +// scroll can't spawn a burst (mirrors comics/drama/youtube). The request generation is // bumped by every fresh (replace) feed load β€” a load-more worker checks it // before publishing so a stale append can never land on top of a feed the // user has since navigated away from. @@ -68,7 +132,7 @@ var loading_more: std.atomic.Value(bool) = std.atomic.Value(bool).init(false); /// when the current feed has no next page. Guarded by parse_mutex. var next_href_buf: [512]u8 = undefined; var next_href_len: usize = 0; -var fetch_gen: std.atomic.Value(u32) = std.atomic.Value(u32).init(0); +var fetch_request: @import("../core/latest_request.zig").Gate = .{}; // Config restores `connected` after startup, but no connect() call occurs in // that path. This latch lets desktop and companion entry points safely kick // one root fetch without retrying a failed server on every frame/request. @@ -100,6 +164,10 @@ fn opdsAuthHeader(buf: []u8) []const u8 { /// a server the rest of the app talks to fine. 43 other services already fetch /// through curl; this was one of the last holdouts. fn opdsGet(url: []const u8, user: []const u8, pass: []const u8) ?[]u8 { + return opdsGetBounded(url, user, pass, 4 * 1024 * 1024, "15"); +} + +fn opdsGetBounded(url: []const u8, user: []const u8, pass: []const u8, cap: usize, timeout: []const u8) ?[]u8 { var auth_buf: [512]u8 = undefined; // basicAuthHeader yields a full header line. Feed it through curl's stdin // config so the credential is not visible in the subprocess command line. @@ -114,18 +182,20 @@ fn opdsGet(url: []const u8, user: []const u8, pass: []const u8) ?[]u8 { // watchdog did. var child = if (auth != null) io_g.Child.init(&.{ - "curl", "-sL", - "-H", "Accept: application/atom+xml,application/xml", - "--config", "-", - "--max-time", "15", - url, + "curl", "-fsSL", + "-H", "Accept: application/atom+xml,application/xml", + "--config", "-", + "--connect-timeout", "3", + "--max-time", timeout, + "--", url, }, alloc) else io_g.Child.init(&.{ - "curl", "-sL", - "-H", "Accept: application/atom+xml,application/xml", - "--max-time", "15", - url, + "curl", "-fsSL", + "-H", "Accept: application/atom+xml,application/xml", + "--connect-timeout", "3", + "--max-time", timeout, + "--", url, }, alloc); child.stdout_behavior = .Pipe; child.stderr_behavior = .Ignore; @@ -135,13 +205,25 @@ fn opdsGet(url: []const u8, user: []const u8, pass: []const u8) ?[]u8 { child.spawn() catch return null; } - const resp_buf = alloc.alloc(u8, 256 * 1024) catch { - _ = child.wait() catch {}; + const resp_buf = alloc.alloc(u8, cap) catch { + _ = child.kill() catch {}; return null; }; defer alloc.free(resp_buf); const n = if (child.stdout) |*so| io_g.readAll(so, resp_buf) catch 0 else 0; - _ = child.wait() catch {}; + if (n == resp_buf.len) { + var extra: [1]u8 = undefined; + const extra_n = if (child.stdout) |*so| io_g.readAll(so, &extra) catch 1 else 1; + if (extra_n > 0) { + _ = child.kill() catch {}; + return null; + } + } + const term = child.wait() catch return null; + if (switch (term) { + .exited => |code| code != 0, + else => true, + }) return null; if (n == 0) return null; const result = alloc.alloc(u8, n) catch return null; @@ -249,42 +331,56 @@ fn setError(msg: []const u8) void { /// a REPLACE fetch (fresh navigation always wins), but the gen is bumped so /// any in-flight load-more append from the PREVIOUS feed drops its stale /// publish instead of corrupting this one. -fn fetchFeedSync(mark_connected: bool, my_gen: u32) void { - // Snapshot everything the UI thread might edit mid-request into worker locals. - var url_buf: [512]u8 = undefined; - const url_len = @min(state.app.opds.current_url_len, url_buf.len); - @memcpy(url_buf[0..url_len], state.app.opds.current_url[0..url_len]); - const url = url_buf[0..url_len]; - - var user_buf: [128]u8 = undefined; - @memcpy(&user_buf, &state.app.opds.user_buf); - var pass_buf: [128]u8 = undefined; - @memcpy(&pass_buf, &state.app.opds.pass_buf); - const user = user_buf[0 .. std.mem.indexOfScalar(u8, &user_buf, 0) orelse user_buf.len]; - const pass = pass_buf[0 .. std.mem.indexOfScalar(u8, &pass_buf, 0) orelse pass_buf.len]; +const FeedJob = struct { + url: [512]u8 = std.mem.zeroes([512]u8), + url_len: usize = 0, + user: [128]u8 = std.mem.zeroes([128]u8), + pass: [128]u8 = std.mem.zeroes([128]u8), + mark_connected: bool = false, + generation: u32 = 0, +}; + +fn publishFetchError(gen: u32, msg: []const u8) void { + parse_mutex.lock(); + defer parse_mutex.unlock(); + if (!fetch_request.isCurrent(gen)) return; + setError(msg); + state.wakeUi(); +} + +fn fetchFeedSync(job: FeedJob) void { + const my_gen = job.generation; + const mark_connected = job.mark_connected; + const url = job.url[0..job.url_len]; + const user = job.user[0 .. std.mem.indexOfScalar(u8, &job.user, 0) orelse job.user.len]; + const pass = job.pass[0 .. std.mem.indexOfScalar(u8, &job.pass, 0) orelse job.pass.len]; if (url.len == 0) { - setError("Catalog URL is empty"); + publishFetchError(my_gen, "Catalog URL is empty"); state.wakeUi(); return; } const body = opdsGet(url, user, pass) orelse { - setError("Could not reach the OPDS server β€” check the URL and credentials"); + publishFetchError(my_gen, "Could not load the OPDS server β€” check its URL, credentials or response size"); state.wakeUi(); return; }; defer alloc.free(body); + if (!pure.isCompleteFeed(body)) { + publishFetchError(my_gen, "The server did not return a complete OPDS 1.x Atom feed"); + return; + } // A newer navigation (another connect/openFeed/goBack) superseded this // in-flight request β€” drop the stale result rather than clobber the feed // the user has since moved to (mirrors drama.zig's fetch_gen guard). - if (fetch_gen.load(.acquire) != my_gen) return; + if (fetch_request.current() != my_gen) return; // Publish under the mutex: title first, entries, then entry_count LAST so a // concurrent UI read never sees a count ahead of the data. parse_mutex.lock(); - if (fetch_gen.load(.acquire) != my_gen) { + if (fetch_request.current() != my_gen) { parse_mutex.unlock(); return; } // re-check under the lock @@ -304,38 +400,40 @@ fn fetchFeedSync(mark_connected: bool, my_gen: u32) void { next_href_len = 0; more_available = false; } - parse_mutex.unlock(); - state.app.opds.fetch_error = false; if (mark_connected) { state.app.opds.connected = true; state.markConfigDirty(); } + parse_mutex.unlock(); logs.pushLog("info", "opds", "OPDS feed loaded", false); state.wakeUi(); } /// Spawn the detached fetch worker for the current feed URL. fn spawnFetch(mark_connected: bool) void { - if (state.app.opds.is_loading.load(.acquire)) return; - state.app.opds.is_loading.store(true, .release); + // Snapshot before spawning: navigation and credentials may change while + // an older request is still waiting for its response. + var job: FeedJob = .{ .mark_connected = mark_connected }; + parse_mutex.lock(); + job.url_len = @min(state.app.opds.current_url_len, job.url.len); + @memcpy(job.url[0..job.url_len], state.app.opds.current_url[0..job.url_len]); + job.user = state.app.opds.user_buf; + job.pass = state.app.opds.pass_buf; + job.generation = fetch_request.begin(&state.app.opds.is_loading); state.app.opds.fetch_error = false; - // A fresh replace-fetch (connect/openFeed/goBack) is a new generation β€” an - // in-flight load-more append from the feed being navigated away from will - // see its generation is stale and drop its publish (see fetchMoreSync). - const my_gen = fetch_gen.fetchAdd(1, .acq_rel) + 1; - + parse_mutex.unlock(); state.app.opds.thread = @import("../core/workers.zig").spawnLegacy(struct { - fn worker(mc: bool, gen: u32) void { - defer state.app.opds.is_loading.store(false, .release); - fetchFeedSync(mc, gen); + fn worker(request: FeedJob) void { + defer fetch_request.finish(request.generation, &state.app.opds.is_loading); + fetchFeedSync(request); + state.wakeUi(); } - }.worker, .{ mark_connected, my_gen }) catch blk: { - state.app.opds.is_loading.store(false, .release); - setError("Could not start the OPDS request"); + }.worker, .{job}) catch blk: { + fetch_request.finish(job.generation, &state.app.opds.is_loading); + publishFetchError(job.generation, "Could not start the OPDS request"); break :blk null; }; - // Detach: the result is observed via state.app.opds, never joined. if (state.app.opds.thread) |t| @import("../core/workers.zig").release(t); } @@ -347,86 +445,68 @@ fn spawnFetch(mark_connected: bool) void { /// merge its entries onto the existing list. Guarded by `loading_more` + the /// main `is_loading` atomic so a near-bottom scroll can't spawn a burst; /// no-op once `more_available` clears β€” which happens the moment a feed has -/// no rel="next" link (set by fetchFeedSync/fetchMoreSync), a load-more fetch -/// fails, or the fixed 300-entry buffer fills. Runs under the current -/// fetch_gen so a fresh feed navigation (connect/openFeed/goBack) supersedes +/// no rel="next" link (set by fetchFeedSync/fetchMoreSync), the fixed 300-entry buffer fills. Errors pause automatic paging until retry. Runs under the current +/// request generation so a fresh feed navigation (connect/openFeed/goBack) supersedes /// it. Mirrors services/drama.zig's loadMore. pub fn loadMore() void { - if (!more_available) return; - if (state.app.opds.is_loading.load(.acquire)) return; - if (loading_more.load(.acquire)) return; - if (state.app.opds.entry_count == 0) return; - if (state.app.opds.entry_count >= state.app.opds.entries.len) { - parse_mutex.lock(); - more_available = false; + parse_mutex.lock(); + if (!more_available or state.app.opds.is_loading.load(.acquire) or + state.app.opds.fetch_error or state.app.opds.entry_count == 0 or + state.app.opds.entry_count >= state.app.opds.entries.len) + { parse_mutex.unlock(); return; } - if (loading_more.swap(true, .acq_rel)) return; // lost the race β€” another append in flight - - const my_gen = fetch_gen.load(.acquire); // stay within the current generation - - if (@import("../core/workers.zig").spawnLegacy(loadMoreWorker, .{my_gen})) |t| { + if (loading_more.swap(true, .acq_rel)) { + parse_mutex.unlock(); + return; + } + var job: FeedJob = .{ .generation = fetch_request.current() }; + job.url_len = @min(next_href_len, job.url.len); + @memcpy(job.url[0..job.url_len], next_href_buf[0..job.url_len]); + job.user = state.app.opds.user_buf; + job.pass = state.app.opds.pass_buf; + parse_mutex.unlock(); + if (@import("../core/workers.zig").spawnLegacy(loadMoreWorker, .{job})) |t| { @import("../core/workers.zig").release(t); } else |_| { loading_more.store(false, .release); + publishFetchError(job.generation, "Could not start the next OPDS page request"); } } -fn loadMoreWorker(my_gen: u32) void { +fn loadMoreWorker(job: FeedJob) void { defer loading_more.store(false, .release); - fetchMoreSync(my_gen); + fetchMoreSync(job); } -/// Fetch the feed's rel="next" continuation page and APPEND its entries onto -/// state.app.opds.entries starting at the current entry_count (never clears β€” -/// only fetchFeedSync's replace path does that). Uses the SAME Basic-auth -/// header + fetch path (opdsGet) as the main feed fetch. Runs ONLY on the -/// detached load-more worker (never the UI thread). -fn fetchMoreSync(my_gen: u32) void { - var url_buf: [512]u8 = undefined; - parse_mutex.lock(); - const url_len = @min(next_href_len, url_buf.len); - @memcpy(url_buf[0..url_len], next_href_buf[0..url_len]); - parse_mutex.unlock(); - const url = url_buf[0..url_len]; - if (url.len == 0) { - parse_mutex.lock(); - more_available = false; - parse_mutex.unlock(); - return; - } - - // Same credential snapshot as fetchFeedSync β€” the UI thread may edit these - // buffers mid-request, so copy them out before the blocking fetch. - var user_buf: [128]u8 = undefined; - @memcpy(&user_buf, &state.app.opds.user_buf); - var pass_buf: [128]u8 = undefined; - @memcpy(&pass_buf, &state.app.opds.pass_buf); - const user = user_buf[0 .. std.mem.indexOfScalar(u8, &user_buf, 0) orelse user_buf.len]; - const pass = pass_buf[0 .. std.mem.indexOfScalar(u8, &pass_buf, 0) orelse pass_buf.len]; +/// Append a continuation using the URL and credentials captured at launch. +fn fetchMoreSync(job: FeedJob) void { + const my_gen = job.generation; + const url = job.url[0..job.url_len]; + if (url.len == 0 or fetch_request.current() != my_gen) return; + const user = job.user[0 .. std.mem.indexOfScalar(u8, &job.user, 0) orelse job.user.len]; + const pass = job.pass[0 .. std.mem.indexOfScalar(u8, &job.pass, 0) orelse job.pass.len]; const body = opdsGet(url, user, pass) orelse { - // Fail closed (mirrors drama.zig treating a short/failed page as "no - // more") rather than retry-spamming the server on every near-bottom - // frame after a transient failure. logs.pushLog("error", "opds", "Load-more fetch failed", true); - if (fetch_gen.load(.acquire) == my_gen) { - parse_mutex.lock(); - more_available = false; - parse_mutex.unlock(); - } + publishFetchError(my_gen, "Could not load the next OPDS page. Retry to refresh the catalog."); + state.wakeUi(); return; }; defer alloc.free(body); + if (!pure.isCompleteFeed(body)) { + publishFetchError(my_gen, "The next OPDS page was incomplete or invalid. Retry to refresh."); + return; + } // A fresh feed navigation superseded this append β€” drop it rather than // append the old feed's continuation onto the new feed's entries. - if (fetch_gen.load(.acquire) != my_gen) return; + if (fetch_request.current() != my_gen) return; parse_mutex.lock(); defer parse_mutex.unlock(); - if (fetch_gen.load(.acquire) != my_gen) return; // re-check under the lock + if (fetch_request.current() != my_gen) return; // re-check under the lock const base = state.app.opds.entry_count; const cap = state.app.opds.entries.len; @@ -434,12 +514,31 @@ fn fetchMoreSync(my_gen: u32) void { more_available = false; return; } - const n = pure.parseFeed(body, url, state.app.opds.entries[base..]); - state.app.opds.entry_count = base + n; + const staged = alloc.alloc(pure.OpdsEntry, cap) catch { + setError("Not enough memory for the next OPDS page"); + return; + }; + defer alloc.free(staged); + const n = pure.parseFeed(body, url, staged); + var count = base; + for (staged[0..n]) |entry| { + var duplicate = false; + for (state.app.opds.entries[0..count]) |existing| { + if (std.mem.eql(u8, existing.href[0..existing.href_len], entry.href[0..entry.href_len])) { + duplicate = true; + break; + } + } + if (duplicate) continue; + if (count >= cap) break; + state.app.opds.entries[count] = entry; + count += 1; + } + state.app.opds.entry_count = count; if (pure.feedNextHref(body, url, &next_href_buf)) |next| { next_href_len = next.len; - more_available = state.app.opds.entry_count < cap; + more_available = state.app.opds.entry_count < cap and !std.mem.eql(u8, next, url); } else { next_href_len = 0; more_available = false; @@ -456,6 +555,7 @@ fn fetchMoreSync(my_gen: u32) void { /// Connect to the configured catalog root (server_url) and load its feed. pub fn connect() void { if (state.app.opds.is_loading.load(.acquire)) return; + configureConnection(state.app.opds.server_url[0..state.app.opds.server_url_len], state.app.opds.user_buf[0 .. std.mem.indexOfScalar(u8, &state.app.opds.user_buf, 0) orelse state.app.opds.user_buf.len], state.app.opds.pass_buf[0 .. std.mem.indexOfScalar(u8, &state.app.opds.pass_buf, 0) orelse state.app.opds.pass_buf.len]); state.app.opds.nav_depth = 0; // current_url := server_url (catalog root). const n = @min(state.app.opds.server_url_len, state.app.opds.current_url.len); @@ -491,6 +591,45 @@ pub fn retry() void { spawnFetch(false); } +pub const SearchResult = struct { + count: usize = 0, + status: @import("resolver_lifecycle_pure.zig").SourceStatus = .no_results, + connection_identity: u64 = 0, +}; + +/// Query only an advertised search facility; never filter the visible feed. +pub fn searchInto(query: []const u8, out: []pure.OpdsEntry) SearchResult { + const snap = connectionSnapshot(); + if (!snap.connected or snap.server_len == 0) return .{ .status = .unavailable }; + if (query.len == 0 or out.len == 0) return .{}; + const root = snap.server[0..snap.server_len]; + const body = opdsGetBounded(root, snap.user[0..snap.user_len], snap.pass[0..snap.pass_len], 1024 * 1024, "8") orelse return .{ .status = .transport_failed }; + defer alloc.free(body); + if (!pure.isCompleteFeed(body)) return .{ .status = .parse_failed }; + const link = pure.feedSearchLink(body, root) orelse { + logs.pushLog("info", "opds", "This catalog does not advertise a supported OPDS search facility", false); + return .{ .status = .unavailable }; + }; + var url_buf: [3072]u8 = undefined; + const href = link.url[0..link.url_len]; + var search_url: []const u8 = undefined; + if (link.description) { + const with_auth = pure.sameOrigin(root, href); + const description = opdsGetBounded(href, if (with_auth) snap.user[0..snap.user_len] else "", if (with_auth) snap.pass[0..snap.pass_len] else "", 128 * 1024, "8") orelse return .{ .status = .transport_failed }; + defer alloc.free(description); + search_url = pure.openSearchUrl(description, href, query, &url_buf) orelse return .{ .status = .unavailable }; + } else { + search_url = pure.expandSearchTemplate(href, query, &url_buf) orelse return .{ .status = .unavailable }; + } + const with_auth = pure.sameOrigin(root, search_url); + const results = opdsGetBounded(search_url, if (with_auth) snap.user[0..snap.user_len] else "", if (with_auth) snap.pass[0..snap.pass_len] else "", 1024 * 1024, "8") orelse return .{ .status = .transport_failed }; + defer alloc.free(results); + if (!pure.isCompleteFeed(results)) return .{ .status = .parse_failed }; + if (connectionSnapshot().identity != snap.identity) return .{ .status = .unavailable }; + const count = pure.parseFeed(results, search_url, out); + return .{ .count = count, .status = if (count > 0) .done else .no_results, .connection_identity = snap.identity }; +} + pub fn entryCount() usize { parse_mutex.lock(); defer parse_mutex.unlock(); @@ -553,6 +692,15 @@ pub fn goBack() void { /// type (image/comic β†’ in-app comics reader; EPUB/PDF β†’ external; else toast). pub fn openEntry(idx: usize) void { const row = entryRow(idx) orelse return; + openCatalogEntry(row, connectionSnapshot().identity); +} + +pub fn openCatalogEntry(row: pure.OpdsEntry, connection_identity: u64) void { + const snap = connectionSnapshot(); + if (!snap.connected or snap.identity != connection_identity) { + state.showToastTyped("OPDS connection changed. Search this catalog again.", .warning); + return; + } const e = &row; const href = e.hrefSlice(); if (href.len == 0) return; @@ -570,15 +718,20 @@ pub fn openEntry(idx: usize) void { // stored credentials and drive the comics reader with the tested // page-URL template + count. var auth_buf: [512]u8 = undefined; - const auth = opdsAuthHeader(&auth_buf); + const auth = if (pure.sameOrigin(snap.server[0..snap.server_len], e.pseUrlSlice())) + pure.basicAuthHeader(snap.user[0..snap.user_len], snap.pass[0..snap.pass_len], &auth_buf) orelse "" + else + ""; @import("comics.zig").loadPseBook(e.titleSlice(), e.pseUrlSlice(), e.pse_count, auth); state.navigateToTab(.Comics); state.showToast("Streaming pages…"); - } else { - // Plain page-image / archive server β†’ the existing scraper. + } else if (std.mem.startsWith(u8, e.contentTypeSlice(), "image/")) { @import("comics.zig").requestLoad(href); state.navigateToTab(.Comics); state.showToast("Opening in reader"); + } else { + @import("../ui/settings.zig").openExternal(href); + state.showToast("This catalog offers an archive. Opening externally."); } }, .external => { @@ -593,13 +746,13 @@ pub fn openEntry(idx: usize) void { /// Disconnect: forget the connection + clear the loaded feed. UI-thread only. pub fn disconnect() void { - _ = fetch_gen.fetchAdd(1, .acq_rel); // supersede any in-flight fetch/append + fetch_request.cancel(&state.app.opds.is_loading); // supersede any in-flight fetch/append parse_mutex.lock(); state.app.opds.entry_count = 0; more_available = true; next_href_len = 0; - parse_mutex.unlock(); state.app.opds.connected = false; + parse_mutex.unlock(); state.app.opds.nav_depth = 0; state.app.opds.feed_title_len = 0; state.app.opds.current_url_len = 0; @@ -764,7 +917,7 @@ fn renderFeed() void { .padding = .{ .x = 12, .y = 6, .w = 12, .h = 6 }, .margin = .{ .x = 16, .y = 0, .w = 0, .h = 8 }, })) retry(); - return; + if (entryCount() == 0) return; } var scroll = dvui.scrollArea(@src(), .{}, .{ diff --git a/src/services/opds_pure.zig b/src/services/opds_pure.zig index 34e4daa1..0bb66b69 100644 --- a/src/services/opds_pure.zig +++ b/src/services/opds_pure.zig @@ -211,6 +211,10 @@ pub fn resolveHref(base_url: []const u8, href: []const u8, buf: []u8) ?[]const u pub const OpdsEntry = struct { title: [256]u8 = std.mem.zeroes([256]u8), title_len: usize = 0, + author: [160]u8 = std.mem.zeroes([160]u8), + author_len: usize = 0, + summary: [512]u8 = std.mem.zeroes([512]u8), + summary_len: usize = 0, /// Primary click target, resolved to an absolute URL: the subsection feed /// (navigation) or the acquisition download (leaf). href: [512]u8 = std.mem.zeroes([512]u8), @@ -266,30 +270,174 @@ pub fn feedTitle(xml: []const u8) []const u8 { /// slice into `xml`). Used for both the feed heading and each entry title. fn tagText(xml: []const u8, limit: usize) []const u8 { const hay = xml[0..@min(limit, xml.len)]; - const open = std.mem.indexOf(u8, hay, "' that closes the opening tag (attributes tolerated). - const gt = std.mem.indexOfScalarPos(u8, hay, open, '>') orelse return ""; - const start = gt + 1; - const close = std.mem.indexOfPos(u8, hay, start, "") orelse return ""; - if (close < start) return ""; - return hay[start..close]; + return contentOf(hay, "title"); } /// Read the value of an XML attribute `name="…"` inside a single tag slice. fn attr(tag: []const u8, name: []const u8) ?[]const u8 { - var search: [64]u8 = undefined; - if (name.len + 1 > search.len) return null; - @memcpy(search[0..name.len], name); - search[name.len] = '='; - const key = search[0 .. name.len + 1]; - const at = std.mem.indexOf(u8, tag, key) orelse return null; - var p = at + key.len; - if (p >= tag.len) return null; - const quote = tag[p]; - if (quote != '"' and quote != '\'') return null; - p += 1; - const end = std.mem.indexOfScalarPos(u8, tag, p, quote) orelse return null; - return tag[p..end]; + var pos: usize = 0; + while (std.mem.indexOfPos(u8, tag, pos, name)) |at| { + pos = at + name.len; + if (at > 0 and !std.ascii.isWhitespace(tag[at - 1])) continue; + var p = pos; + while (p < tag.len and std.ascii.isWhitespace(tag[p])) : (p += 1) {} + if (p >= tag.len or tag[p] != '=') continue; + p += 1; + while (p < tag.len and std.ascii.isWhitespace(tag[p])) : (p += 1) {} + if (p >= tag.len or (tag[p] != '\'' and tag[p] != '"')) continue; + const quote = tag[p]; + p += 1; + const end = std.mem.indexOfScalarPos(u8, tag, p, quote) orelse return null; + return tag[p..end]; + } + return null; +} + +const XmlTag = struct { start: usize, end: usize, local: []const u8, closing: bool }; +fn nextTag(xml: []const u8, from: usize) ?XmlTag { + var pos = from; + while (std.mem.indexOfScalarPos(u8, xml, pos, '<')) |at| { + if (std.mem.startsWith(u8, xml[at..], "") orelse return null) + 3; + continue; + } + if (std.mem.startsWith(u8, xml[at..], "") orelse return null) + 3; + continue; + } + var p = at + 1; + const closing = p < xml.len and xml[p] == '/'; + if (closing) p += 1; + const start = p; + while (p < xml.len and !std.ascii.isWhitespace(xml[p]) and xml[p] != '>' and xml[p] != '/') : (p += 1) {} + const name = xml[start..p]; + const local = if (std.mem.lastIndexOfScalar(u8, name, ':')) |colon| name[colon + 1 ..] else name; + var quote: u8 = 0; + while (p < xml.len) : (p += 1) { + const ch = xml[p]; + if (quote != 0) { + if (ch == quote) quote = 0; + } else if (ch == '\'' or ch == '"') quote = ch else if (ch == '>') { + return .{ .start = at, .end = p + 1, .local = local, .closing = closing }; + } + } + return null; + } + return null; +} +fn findTag(xml: []const u8, from: usize, name: []const u8, closing: bool) ?XmlTag { + var pos = from; + while (nextTag(xml, pos)) |tag| { + pos = tag.end; + if (tag.closing == closing and std.mem.eql(u8, tag.local, name)) return tag; + } + return null; +} +fn contentOf(xml: []const u8, name: []const u8) []const u8 { + const open = findTag(xml, 0, name, false) orelse return ""; + const close = findTag(xml, open.end, name, true) orelse return ""; + return xml[open.end..close.start]; +} + +pub const SearchLink = struct { url: [1024]u8 = undefined, url_len: usize = 0, description: bool = false }; +/// Only advertised feed-level search links are used; entry links are unrelated. +pub fn feedSearchLink(xml: []const u8, base: []const u8) ?SearchLink { + var depth: usize = 0; + var pos: usize = 0; + while (nextTag(xml, pos)) |tag| { + pos = tag.end; + if (std.mem.eql(u8, tag.local, "entry")) { + if (tag.closing) depth -|= 1 else depth += 1; + } + if (depth != 0 or tag.closing or !std.mem.eql(u8, tag.local, "link")) continue; + const raw = xml[tag.start..tag.end]; + const rel = attr(raw, "rel") orelse continue; + if (!std.mem.eql(u8, rel, "search")) continue; + const kind = attr(raw, "type") orelse ""; + const description = std.mem.startsWith(u8, kind, "application/opensearchdescription+xml"); + if (!description and !std.mem.startsWith(u8, kind, "application/atom+xml")) continue; + var decoded: [1024]u8 = undefined; + const href = attr(raw, "href") orelse continue; + if (href.len > decoded.len) return null; + const n = decodeXmlEntities(href, &decoded); + var result: SearchLink = .{ .description = description }; + const address = resolveHref(base, decoded[0..n], &result.url) orelse return null; + result.url_len = address.len; + return result; + } + return null; +} + +/// Expand documented OpenSearch parameters. Never invent a query endpoint. +pub fn expandSearchTemplate(template: []const u8, query: []const u8, out: []u8) ?[]const u8 { + var encoded: [3072]u8 = undefined; + var e: usize = 0; + for (query) |ch| { + if (std.ascii.isAlphanumeric(ch) or ch == '-' or ch == '_' or ch == '.' or ch == '~') { + if (e == encoded.len) return null; + encoded[e] = ch; + e += 1; + } else { + if (e + 3 > encoded.len) return null; + encoded[e] = '%'; + encoded[e + 1] = "0123456789ABCDEF"[ch >> 4]; + encoded[e + 2] = "0123456789ABCDEF"[ch & 15]; + e += 3; + } + } + var pos: usize = 0; + var written: usize = 0; + var searched = false; + while (pos < template.len) { + if (template[pos] != '{') { + if (written >= out.len) return null; + out[written] = template[pos]; + written += 1; + pos += 1; + continue; + } + const end = std.mem.indexOfScalarPos(u8, template, pos, '}') orelse return null; + var parameter = template[pos + 1 .. end]; + const optional = std.mem.endsWith(u8, parameter, "?"); + if (optional) parameter = parameter[0 .. parameter.len - 1]; + if (std.mem.startsWith(u8, parameter, "opensearch:")) parameter = parameter["opensearch:".len..]; + const value: []const u8 = if (std.mem.eql(u8, parameter, "searchTerms")) blk: { + searched = true; + break :blk encoded[0..e]; + } else if (std.mem.eql(u8, parameter, "count")) "12" else if (std.mem.eql(u8, parameter, "startIndex") or std.mem.eql(u8, parameter, "startPage")) "1" else if (std.mem.eql(u8, parameter, "language")) "*" else if (std.mem.eql(u8, parameter, "inputEncoding") or std.mem.eql(u8, parameter, "outputEncoding")) "UTF-8" else if (optional) "" else return null; + if (written + value.len > out.len) return null; + @memcpy(out[written..][0..value.len], value); + written += value.len; + pos = end + 1; + } + return if (searched and written > 0) out[0..written] else null; +} + +pub fn openSearchUrl(xml: []const u8, base: []const u8, query: []const u8, out: []u8) ?[]const u8 { + var pos: usize = 0; + while (findTag(xml, pos, "Url", false)) |tag| { + pos = tag.end; + const raw = xml[tag.start..tag.end]; + const kind = attr(raw, "type") orelse continue; + if (!std.mem.startsWith(u8, kind, "application/atom+xml")) continue; + const method = attr(raw, "method") orelse "GET"; + if (!std.ascii.eqlIgnoreCase(method, "GET")) continue; + const template = attr(raw, "template") orelse continue; + var decoded: [2048]u8 = undefined; + if (template.len > decoded.len) continue; + const n = decodeXmlEntities(template, &decoded); + var address_buf: [2048]u8 = undefined; + const address = resolveHref(base, decoded[0..n], &address_buf) orelse continue; + if (expandSearchTemplate(address, query, out)) |expanded| return expanded; + } + return null; +} + +pub fn sameOrigin(a: []const u8, b: []const u8) bool { + const ao = originOf(a) orelse return false; + const bo = originOf(b) orelse return false; + if (std.mem.indexOfScalar(u8, ao, '@') != null or std.mem.indexOfScalar(u8, bo, '@') != null) return false; + return std.ascii.eqlIgnoreCase(ao, bo); } /// Decode the minimal XML entity set OPDS titles use into `out`; returns the @@ -334,6 +482,13 @@ fn decodeXmlEntities(in: []const u8, out: []u8) usize { return o; } +/// Reject error documents and truncated responses before catalog publication. +/// This is an envelope check for the unprefixed Atom feeds this parser supports. +pub fn isCompleteFeed(xml: []const u8) bool { + const start = findTag(xml, 0, "feed", false) orelse return false; + return findTag(xml, start.end, "feed", true) != null; +} + /// Extract the feed-level `` (Atom/OPDS pagination /// β€” RFC 5005 style paging that Komga/Kavita/Calibre-Web emit) and resolve it /// against `base_url` into `buf`. Returns null when the feed has no next page @@ -370,7 +525,10 @@ pub fn feedNextHref(xml: []const u8, base_url: []const u8, buf: []u8) ?[]const u const rel = attr(tag, "rel") orelse continue; if (!std.mem.eql(u8, rel, "next")) continue; const href = attr(tag, "href") orelse continue; - return resolveHref(base_url, href, buf); + var decoded: [1024]u8 = undefined; + const n = decodeXmlEntities(href, &decoded); + if (n == decoded.len and href.len > decoded.len) return null; + return resolveHref(base_url, decoded[0..n], buf); } return null; } @@ -384,11 +542,10 @@ pub fn parseFeed(xml: []const u8, base_url: []const u8, out: []OpdsEntry) usize var pos: usize = 0; while (count < out.len) { - const e_rel = std.mem.indexOfPos(u8, xml, pos, ", or (truncated feed) at end-of-buffer. - const e_end = std.mem.indexOfPos(u8, xml, e_rel, "") orelse xml.len; - const block = xml[e_rel..e_end]; - pos = if (e_end < xml.len) e_end + "".len else xml.len; + const entry_tag = findTag(xml, pos, "entry", false) orelse break; + const close_tag = findTag(xml, entry_tag.end, "entry", true) orelse break; + const block = xml[entry_tag.start..close_tag.start]; + pos = close_tag.end; var ent = OpdsEntry{}; @@ -398,15 +555,24 @@ pub fn parseFeed(xml: []const u8, base_url: []const u8, out: []OpdsEntry) usize ent.title_len = decodeXmlEntities(raw_title, &ent.title); } + const author = contentOf(contentOf(block, "author"), "name"); + ent.author_len = decodeXmlEntities(author, &ent.author); + const summary = contentOf(block, "summary"); + var decoded_summary: [2048]u8 = undefined; + const description = if (summary.len > 0) summary else contentOf(block, "content"); + const summary_len = decodeXmlEntities(description, &decoded_summary); + ent.summary_len = @import("novels_pure.zig").htmlToText(decoded_summary[0..summary_len], &ent.summary); // Walk every tag in the block, classifying each. var lp: usize = 0; var have_primary = false; - while (std.mem.indexOfPos(u8, block, lp, "') orelse break; - const tag = block[l_at .. l_close + 1]; - lp = l_close + 1; - - const href = attr(tag, "href") orelse continue; + while (findTag(block, lp, "link", false)) |link_tag| { + const tag = block[link_tag.start..link_tag.end]; + lp = link_tag.end; + const raw_href = attr(tag, "href") orelse continue; + var decoded_href: [1024]u8 = undefined; + if (raw_href.len > decoded_href.len) continue; + const href_n = decodeXmlEntities(raw_href, &decoded_href); + const href = decoded_href[0..href_n]; const rel = attr(tag, "rel") orelse ""; const kind = classifyRel(rel); @@ -431,9 +597,9 @@ pub fn parseFeed(xml: []const u8, base_url: []const u8, out: []OpdsEntry) usize ent.is_navigation = false; have_primary = true; if (attr(tag, "type")) |ty| { - const n = @min(ty.len, ent.content_type.len); - @memcpy(ent.content_type[0..n], ty[0..n]); - ent.content_type_len = n; + const type_len = @min(ty.len, ent.content_type.len); + @memcpy(ent.content_type[0..type_len], ty[0..type_len]); + ent.content_type_len = type_len; } } } diff --git a/src/services/plex.zig b/src/services/plex.zig index a54f5c06..336f7c5e 100644 --- a/src/services/plex.zig +++ b/src/services/plex.zig @@ -145,8 +145,47 @@ fn setStatus(comptime fmt: []const u8, args: anytype) void { const s = std.fmt.bufPrint(&status_msg, fmt, args) catch status_msg[0..0]; status_msg_len = s.len; } +var connection_mutex: @import("../core/sync.zig").Mutex = .{}; +var connection_epoch: u64 = 1; + +pub const ConnectionSnapshot = struct { + account_token: [128]u8, + account_token_len: usize, + server: [256]u8, + server_len: usize, + server_token: [128]u8, + server_token_len: usize, + name: [128]u8, + name_len: usize, + epoch: u64, + + fn accessToken(self: *const @This()) []const u8 { + return if (self.server_token_len > 0) self.server_token[0..self.server_token_len] else self.account_token[0..self.account_token_len]; + } +}; + +pub fn connectionSnapshot() ConnectionSnapshot { + connection_mutex.lock(); + defer connection_mutex.unlock(); + var snap: ConnectionSnapshot = .{ + .account_token = token_buf, + .account_token_len = @min(token_len, token_buf.len), + .server = server_uri, + .server_len = @min(server_uri_len, server_uri.len), + .server_token = server_token, + .server_token_len = @min(server_token_len, server_token.len), + .name = undefined, + .name_len = @min(server_name_len, server_name.len), + .epoch = connection_epoch, + }; + @memset(&snap.name, 0); + @memcpy(snap.name[0..snap.name_len], server_name[0..snap.name_len]); + return snap; +} + pub fn isConnected() bool { - return token_len > 0; + const connection = connectionSnapshot(); + return connection.account_token_len > 0; } fn token() []const u8 { return token_buf[0..token_len]; @@ -161,14 +200,15 @@ fn cfgPath(buf: []u8) []const u8 { return std.fmt.bufPrint(buf, "{s}/plex.json", .{paths.configDir(&c)}) catch ""; } fn save() void { + const connection = connectionSnapshot(); var protected_token: [512]u8 = undefined; defer @memset(&protected_token, 0); var protected_server_token: [512]u8 = undefined; defer @memset(&protected_server_token, 0); - const sealed_token = secret_store.seal(token(), &protected_token) orelse return; - const sealed_server_token = secret_store.seal(serverTok(), &protected_server_token) orelse return; + const sealed_token = secret_store.seal(connection.account_token[0..connection.account_token_len], &protected_token) orelse return; + const sealed_server_token = secret_store.seal(connection.accessToken(), &protected_server_token) orelse return; var b: [3072]u8 = undefined; - const body = std.fmt.bufPrint(&b, "{{\"token\":\"{s}\",\"server\":\"{s}\",\"server_token\":\"{s}\",\"name\":\"{s}\"}}", .{ sealed_token, server_uri[0..server_uri_len], sealed_server_token, server_name[0..server_name_len] }) catch return; + const body = std.fmt.bufPrint(&b, "{{\"token\":\"{s}\",\"server\":\"{s}\",\"server_token\":\"{s}\",\"name\":\"{s}\"}}", .{ sealed_token, connection.server[0..connection.server_len], sealed_server_token, connection.name[0..connection.name_len] }) catch return; var pb: [600]u8 = undefined; @import("../core/secret_file.zig").write(cfgPath(&pb), body) catch {}; } @@ -197,18 +237,26 @@ pub fn init() void { var parsed = std.json.parseFromSlice(Json, alloc, body, .{}) catch return; defer parsed.deinit(); if (parsed.value != .object) return; + connection_mutex.lock(); const legacy_token = loadSecretStr(parsed.value, "token", &token_buf, &token_len); loadStr(parsed.value, "server", &server_uri, &server_uri_len); const legacy_server_token = loadSecretStr(parsed.value, "server_token", &server_token, &server_token_len); loadStr(parsed.value, "name", &server_name, &server_name_len); if (token_len > 0) conn_state.store(.connected, .release); + connection_epoch +%= 1; + connection_mutex.unlock(); if (@import("builtin").os.tag == .windows and (legacy_token or legacy_server_token)) save(); } pub fn disconnect() void { @import("server_progress.zig").clearPlexConnection(); + connection_mutex.lock(); + @memset(&token_buf, 0); + @memset(&server_token, 0); token_len = 0; server_uri_len = 0; server_token_len = 0; + connection_epoch +%= 1; + connection_mutex.unlock(); section_count = 0; item_count = 0; nav_depth = 0; @@ -246,11 +294,24 @@ fn httpGet(url: []const u8, post: bool, tok: []const u8, buf: []u8, status_out: } fn expireAuthSession() void { - @import("server_progress.zig").clearPlexConnection(); + expireAuthSessionFor(null); +} + +fn expireAuthSessionFor(expected_epoch: ?u64) void { + connection_mutex.lock(); + if (expected_epoch) |epoch| { + if (connection_epoch != epoch) { + connection_mutex.unlock(); + return; + } + } @memset(&token_buf, 0); token_len = 0; @memset(&server_token, 0); server_token_len = 0; + connection_epoch +%= 1; + connection_mutex.unlock(); + @import("server_progress.zig").clearPlexConnection(); section_count = 0; item_count = 0; current_start = 0; @@ -274,16 +335,17 @@ pub fn connect() void { if (conn_state.load(.acquire) == .awaiting) return; conn_state.store(.awaiting, .release); setStatus("Requesting PIN…", .{}); - @import("../core/workers.zig").release(@import("../core/workers.zig").spawnLegacy(pinWorker, .{}) catch { + @import("../core/workers.zig").release(@import("../core/workers.zig").spawnLegacy(pinWorker, .{connectionSnapshot().epoch}) catch { conn_state.store(.err, .release); return; }); } -fn pinWorker() void { +fn pinWorker(start_epoch: u64) void { var buf: [16384]u8 = undefined; // Plain pin β†’ a short 4-char code usable at plex.tv/link (strong pins are long). const n = httpGet("https://plex.tv/api/v2/pins", true, "", &buf, null); + if (connectionSnapshot().epoch != start_epoch) return; if (n == 0) { conn_state.store(.err, .release); setStatus("Network error", .{}); @@ -312,13 +374,21 @@ fn pinWorker() void { while (waited < 120) : (waited += 3) { io.sleep(3 * std.time.ns_per_s); const m = httpGet(purl, false, "", &buf, null); + if (connectionSnapshot().epoch != start_epoch) return; if (m == 0) continue; var pp = std.json.parseFromSlice(Json, alloc, buf[0..m], .{}) catch continue; defer pp.deinit(); if (jstr(pp.value, "authToken")) |at| { + connection_mutex.lock(); + if (connection_epoch != start_epoch) { + connection_mutex.unlock(); + return; + } const tl = @min(at.len, token_buf.len); @memcpy(token_buf[0..tl], at[0..tl]); token_len = tl; + connection_epoch +%= 1; + connection_mutex.unlock(); setStatus("Linked β€” finding servers…", .{}); discoverServers(); return; @@ -329,12 +399,15 @@ fn pinWorker() void { } fn discoverServers() void { - var buf: [262144]u8 = undefined; + const connection = connectionSnapshot(); + const buf = alloc.alloc(u8, 256 * 1024) catch return; + defer alloc.free(buf); var status: ?std.http.Status = null; - const n = httpGet("https://plex.tv/api/v2/resources?includeHttps=1&includeRelay=1", false, token(), &buf, &status); + const n = httpGet("https://plex.tv/api/v2/resources?includeHttps=1&includeRelay=1", false, connection.account_token[0..connection.account_token_len], buf, &status); + if (connectionSnapshot().epoch != connection.epoch) return; if (n == 0) { if (status) |s| if (plex_pure.authRejected(@intFromEnum(s))) { - expireAuthSession(); + expireAuthSessionFor(connection.epoch); return; }; conn_state.store(.err, .release); @@ -370,9 +443,16 @@ fn discoverServers() void { if (chosen == null) chosen = uri; } const uri = chosen orelse continue; + connection_mutex.lock(); + if (connection_epoch != connection.epoch) { + connection_mutex.unlock(); + return; + } const ul = @min(uri.len, server_uri.len); @memcpy(server_uri[0..ul], uri[0..ul]); server_uri_len = ul; + @memset(&server_token, 0); + server_token_len = 0; if (jstr(res, "accessToken")) |st| { const sl = @min(st.len, server_token.len); @memcpy(server_token[0..sl], st[0..sl]); @@ -383,6 +463,8 @@ fn discoverServers() void { @memcpy(server_name[0..nl], nm[0..nl]); server_name_len = nl; } + connection_epoch +%= 1; + connection_mutex.unlock(); conn_state.store(.connected, .release); setStatus("Connected: {s}", .{server_name[0..server_name_len]}); save(); @@ -690,27 +772,39 @@ fn fetchWindow(request: BrowseRequest, start: usize, gen: u64) void { /// Search the connected Plex server without touching the Plex tab's live /// section/items buffers. This makes the omnibox a real cross-library search /// while keeping worker ownership isolated from browse pagination. +pub const SearchResult = struct { + count: usize = 0, + status: @import("resolver_lifecycle_pure.zig").SourceStatus = .no_results, +}; + pub fn searchInto(query: []const u8, out: []SearchItem) usize { - if (query.len == 0 or out.len == 0 or !isConnected() or server_uri_len == 0 or serverTok().len == 0) return 0; + return searchIntoResult(query, out).count; +} +pub fn searchIntoResult(query: []const u8, out: []SearchItem) SearchResult { + const connection = connectionSnapshot(); + if (query.len == 0 or out.len == 0) return .{}; + if (connection.server_len == 0 or connection.accessToken().len == 0) return .{ .status = .unavailable }; var enc_buf: [768]u8 = undefined; const encoded = @import("../core/http.zig").urlEncode(query, &enc_buf); var url_buf: [1200]u8 = undefined; const url = std.fmt.bufPrint(&url_buf, "{s}/search?query={s}&limit={d}", .{ - server_uri[0..server_uri_len], encoded, @min(out.len, 24), - }) catch return 0; - - const body = alloc.alloc(u8, 512 * 1024) catch return 0; + connection.server[0..connection.server_len], encoded, @min(out.len, 24), + }) catch return .{ .status = .failed }; + const body = alloc.alloc(u8, 512 * 1024) catch return .{ .status = .failed }; defer alloc.free(body); var status: ?std.http.Status = null; @import("../core/rate_limit.zig").acquire("plex", 5.0); - const n = httpGet(url, false, serverTok(), body, &status); + const n = httpGet(url, false, connection.accessToken(), body, &status); if (n == 0) { - if (status) |s| if (plex_pure.authRejected(@intFromEnum(s))) expireAuthSession(); - return 0; + if (status) |code| if (plex_pure.authRejected(@intFromEnum(code))) { + expireAuthSessionFor(connection.epoch); + return .{ .status = .unavailable }; + }; + return .{ .status = .transport_failed }; } - - return plex_pure.parseSearchItems(alloc, body[0..n], out) orelse 0; + const count = plex_pure.parseSearchItems(alloc, body[0..n], out) orelse return .{ .status = .parse_failed }; + return .{ .count = count, .status = if (count > 0) .done else .no_results }; } /// Infinite-scroll appender: fetch the NEXT Container-Start/Size window for the diff --git a/src/services/plugins.zig b/src/services/plugins.zig index 2a352152..9d3a2d79 100644 --- a/src/services/plugins.zig +++ b/src/services/plugins.zig @@ -532,8 +532,9 @@ pub fn searchInstalledInto(query: []const u8, out: []UniversalResult) usize { var query_buf: [256]u8 = std.mem.zeroes([256]u8); const query_len = @min(query.len, query_buf.len); @memcpy(query_buf[0..query_len], query[0..query_len]); - var slots: [UNIVERSAL_PLUGIN_CAP][UNIVERSAL_ROWS_PER_PLUGIN]UniversalResult = - [_][UNIVERSAL_ROWS_PER_PLUGIN]UniversalResult{[_]UniversalResult{.{}} ** UNIVERSAL_ROWS_PER_PLUGIN} ** UNIVERSAL_PLUGIN_CAP; + const slots = c_alloc.alloc([UNIVERSAL_ROWS_PER_PLUGIN]UniversalResult, UNIVERSAL_PLUGIN_CAP) catch return 0; + defer c_alloc.free(slots); + @memset(slots, [_]UniversalResult{.{}} ** UNIVERSAL_ROWS_PER_PLUGIN); var counts: [UNIVERSAL_PLUGIN_CAP]usize = [_]usize{0} ** UNIVERSAL_PLUGIN_CAP; var threads: [UNIVERSAL_PLUGIN_CAP]?std.Thread = [_]?std.Thread{null} ** UNIVERSAL_PLUGIN_CAP; const Task = struct { diff --git a/src/services/plugins_pure.zig b/src/services/plugins_pure.zig index 16f3c721..092ae41d 100644 --- a/src/services/plugins_pure.zig +++ b/src/services/plugins_pure.zig @@ -53,6 +53,10 @@ pub const SearchRow = struct { media_type: [16]u8 = std.mem.zeroes([16]u8), media_type_len: usize = 0, episodes: u16 = 0, + poster_url: [256]u8 = std.mem.zeroes([256]u8), + poster_url_len: usize = 0, + overview: [512]u8 = std.mem.zeroes([512]u8), + overview_len: usize = 0, }; fn copyJsonString(value: std.json.Value, key: []const u8, dst: []u8, len: *usize) void { @@ -81,6 +85,8 @@ pub fn parseSearchRows(allocator: std.mem.Allocator, input: []const u8, out: []S copyJsonString(value, "stream_url", &row.stream_url, &row.stream_url_len); copyJsonString(value, "year", &row.year, &row.year_len); copyJsonString(value, "type", &row.media_type, &row.media_type_len); + copyJsonString(value, "poster", &row.poster_url, &row.poster_url_len); + copyJsonString(value, "overview", &row.overview, &row.overview_len); if (value.object.get("episodes")) |episodes| { if (episodes == .integer and episodes.integer > 0) row.episodes = @intCast(@min(episodes.integer, std.math.maxInt(u16))); diff --git a/src/services/podcasts.zig b/src/services/podcasts.zig index 2ee76b56..7c6c4015 100644 --- a/src/services/podcasts.zig +++ b/src/services/podcasts.zig @@ -37,7 +37,7 @@ const agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/2010010 var parse_mutex: @import("../core/sync.zig").Mutex = .{}; var search_request: LatestRequest = .{}; var publication_gen: u64 = 0; -var episode_gen: std.atomic.Value(u32) = .init(0); +var episode_request: @import("../core/latest_request.zig").Gate = .{}; var episode_failed: std.atomic.Value(bool) = .init(false); var episode_retry_count: std.atomic.Value(u8) = .init(0); var episode_retry_at_ms: std.atomic.Value(i64) = .init(0); @@ -62,23 +62,27 @@ pub const Snapshot = struct { }; pub fn snapshot() Snapshot { + var view: Snapshot = undefined; + copySnapshot(&view); + return view; +} + +pub fn copySnapshot(out: *Snapshot) void { parse_mutex.lock(); defer parse_mutex.unlock(); - return .{ - .generation = publication_gen, - .results = state.app.podcasts.results, - .result_count = @min(state.app.podcasts.result_count, state.app.podcasts.results.len), - .episodes = state.app.podcasts.episodes, - .episode_count = @min(state.app.podcasts.episode_count, state.app.podcasts.episodes.len), - .selected_idx = state.app.podcasts.selected_idx, - .selected_name = state.app.podcasts.selected_name, - .selected_name_len = @min(state.app.podcasts.selected_name_len, state.app.podcasts.selected_name.len), - .fetch_error = state.app.podcasts.fetch_error, - .showing_popular = state.app.podcasts.showing_popular, - .loading = state.app.podcasts.is_loading.load(.acquire), - .episodes_loading = state.app.podcasts.episodes_loading.load(.acquire), - .episodes_failed = episode_failed.load(.acquire), - }; + out.generation = publication_gen; + out.results = state.app.podcasts.results; + out.result_count = @min(state.app.podcasts.result_count, state.app.podcasts.results.len); + out.episodes = state.app.podcasts.episodes; + out.episode_count = @min(state.app.podcasts.episode_count, state.app.podcasts.episodes.len); + out.selected_idx = state.app.podcasts.selected_idx; + out.selected_name = state.app.podcasts.selected_name; + out.selected_name_len = @min(state.app.podcasts.selected_name_len, state.app.podcasts.selected_name.len); + out.fetch_error = state.app.podcasts.fetch_error; + out.showing_popular = state.app.podcasts.showing_popular; + out.loading = state.app.podcasts.is_loading.load(.acquire); + out.episodes_loading = state.app.podcasts.episodes_loading.load(.acquire); + out.episodes_failed = episode_failed.load(.acquire); } pub fn copyArtwork(idx: usize, out: []u8) usize { @@ -92,13 +96,12 @@ pub fn copyArtwork(idx: usize, out: []u8) usize { } pub fn closeEpisodes() void { - _ = episode_gen.fetchAdd(1, .acq_rel); + parse_mutex.lock(); + defer parse_mutex.unlock(); + episode_request.cancel(&state.app.podcasts.episodes_loading); episode_failed.store(false, .release); episode_retry_count.store(0, .release); episode_retry_at_ms.store(0, .release); - state.app.podcasts.episodes_loading.store(false, .release); - parse_mutex.lock(); - defer parse_mutex.unlock(); state.app.podcasts.selected_idx = null; state.app.podcasts.episode_count = 0; publication_gen +%= 1; @@ -321,7 +324,7 @@ fn fetchApplePopular(rows: []pure.Podcast) usize { var lookup_url_buf: [640]u8 = undefined; const body = fetchBody(pure.buildLookupUrl(ids, &lookup_url_buf), 512 * 1024) orelse return 0; defer alloc.free(body); - return pure.parseItunes(body, rows); + return parseItunesBody(body, rows) orelse 0; } // ══════════════════════════════════════════════════════════ @@ -385,9 +388,10 @@ fn searchWorker(job: SearchJob) void { const url = std.fmt.bufPrint(&url_buf, "https://itunes.apple.com/search?media=podcast&limit=40&term={s}", .{encoded}) catch return; if (fetchBody(url, 512 * 1024)) |body| { defer alloc.free(body); - const n = pure.parseItunes(body, extra); - count = pure.appendUnique(rows, count, extra[0..n]); - succeeded = succeeded or std.mem.indexOf(u8, body, "\"results\"") != null; + if (parseItunesBody(body, extra)) |n| { + count = pure.appendUnique(rows, count, extra[0..n]); + succeeded = true; + } } publishShows(job.generation, rows[0..count], !succeeded); } @@ -396,6 +400,12 @@ fn searchWorker(job: SearchJob) void { // Episodes β€” a show's RSS feed // ══════════════════════════════════════════════════════════ +fn parseItunesBody(body: []const u8, out: []pure.Podcast) ?usize { + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch return null; + defer parsed.deinit(); + return pure.parseItunesValue(parsed.value, out); +} + const PODCAST_EPISODE_CACHE_TTL_S: i64 = 6 * 60 * 60; const PODCAST_FEED_CAP: usize = 4 * 1024 * 1024; @@ -419,8 +429,16 @@ fn clearEpisodeFailure() void { episode_retry_at_ms.store(0, .release); } +fn clearEpisodeFailureFor(generation: u32) void { + parse_mutex.lock(); + defer parse_mutex.unlock(); + if (episode_request.current() == generation) clearEpisodeFailure(); +} + fn markEpisodeFailure(generation: u32) void { - if (episode_gen.load(.acquire) != generation) return; + parse_mutex.lock(); + defer parse_mutex.unlock(); + if (episode_request.current() != generation) return; const old = episode_retry_count.load(.acquire); const attempt = route_resilience.nextAttempt(old); episode_retry_count.store(attempt, .release); @@ -434,8 +452,6 @@ pub fn loadEpisodes(idx: usize) void { } fn startEpisodes(idx: usize, reset_retry: bool) void { - if (state.app.podcasts.episodes_loading.load(.acquire)) return; - parse_mutex.lock(); if (idx >= state.app.podcasts.result_count) { parse_mutex.unlock(); @@ -447,7 +463,6 @@ fn startEpisodes(idx: usize, reset_retry: bool) void { state.app.podcasts.fetch_error = false; clearEpisodeFailure(); } - state.app.podcasts.episodes_loading.store(true, .release); // Copy the selected show's name (episode-view header) + feed url for the // worker so it never reads results[] as it may be reordered by a new search. @@ -456,7 +471,7 @@ fn startEpisodes(idx: usize, reset_retry: bool) void { @memcpy(state.app.podcasts.selected_name[0..nlen], p.name[0..nlen]); state.app.podcasts.selected_name_len = nlen; - var job = EpisodeJob{ .generation = episode_gen.fetchAdd(1, .acq_rel) +% 1 }; + var job = EpisodeJob{ .generation = episode_request.begin(&state.app.podcasts.episodes_loading) }; const flen = @min(p.feed_url_len, job.feed.len); @memcpy(job.feed[0..flen], p.feed_url[0..flen]); job.feed_len = flen; @@ -464,7 +479,7 @@ fn startEpisodes(idx: usize, reset_retry: bool) void { parse_mutex.unlock(); workers.spawn(episodeWorker, .{job}) catch { - state.app.podcasts.episodes_loading.store(false, .release); + episode_request.finish(job.generation, &state.app.podcasts.episodes_loading); markEpisodeFailure(job.generation); }; } @@ -481,10 +496,10 @@ pub fn retryEpisodesIfDue() bool { } fn episodeWorker(job: EpisodeJob) void { - defer if (episode_gen.load(.acquire) == job.generation) { - state.app.podcasts.episodes_loading.store(false, .release); + defer { + episode_request.finish(job.generation, &state.app.podcasts.episodes_loading); state.wakeUi(); - }; + } const url = job.feed[0..job.feed_len]; if (url.len == 0) { markEpisodeFailure(job.generation); @@ -501,7 +516,7 @@ fn episodeWorker(job: EpisodeJob) void { if (cache_key) |key| { if (content_cache.get(key, cache_buf)) |hit| { if (publishEpisodes(job.generation, hit.bytes)) { - clearEpisodeFailure(); + clearEpisodeFailureFor(job.generation); if (hit.staleness == .fresh) return; } } @@ -516,7 +531,7 @@ fn episodeWorker(job: EpisodeJob) void { defer alloc.free(body); if (publishEpisodes(job.generation, body)) { if (cache_key) |key| content_cache.put(key, body, PODCAST_EPISODE_CACHE_TTL_S); - clearEpisodeFailure(); + clearEpisodeFailureFor(job.generation); return; } if (attempt < 2) io.sleep((250 + @as(u64, attempt) * 500) * std.time.ns_per_ms); @@ -525,13 +540,13 @@ fn episodeWorker(job: EpisodeJob) void { } fn publishEpisodes(generation: u32, body: []const u8) bool { - if (episode_gen.load(.acquire) != generation or !rssBodyValid(body)) return false; + if (episode_request.current() != generation or !rssBodyValid(body)) return false; const parsed = alloc.alloc(pure.Episode, state.app.podcasts.episodes.len) catch return false; defer alloc.free(parsed); const n = pure.parseRssEpisodes(body, parsed); parse_mutex.lock(); defer parse_mutex.unlock(); - if (episode_gen.load(.acquire) != generation or state.app.podcasts.selected_idx == null) return false; + if (episode_request.current() != generation or state.app.podcasts.selected_idx == null) return false; @memcpy(state.app.podcasts.episodes[0..n], parsed[0..n]); state.app.podcasts.episode_count = n; publication_gen +%= 1; @@ -548,7 +563,9 @@ fn publishEpisodes(generation: u32, body: []const u8) bool { /// direct audio stream, so loadContentDirect (no content-type routing) is used /// β€” creating a player if none exists and revealing the player page. pub fn playEpisode(idx: usize) void { - const view = snapshot(); + const view = alloc.create(Snapshot) catch return; + defer alloc.destroy(view); + copySnapshot(view); if (idx >= view.episode_count) return; const e = &view.episodes[idx]; if (e.audio_url_len == 0) return; diff --git a/src/services/podcasts_pure.zig b/src/services/podcasts_pure.zig index 9857f9be..4a5cd1a9 100644 --- a/src/services/podcasts_pure.zig +++ b/src/services/podcasts_pure.zig @@ -37,8 +37,49 @@ pub const Episode = struct { date_len: usize = 0, duration: [16]u8 = std.mem.zeroes([16]u8), duration_len: usize = 0, + summary: [512]u8 = std.mem.zeroes([512]u8), + summary_len: usize = 0, }; +/// Project decoded Apple rows without relying on key order or JSON spacing. +/// Invalid envelopes are distinct from a valid search with no matches. +pub fn parseItunesValue(root: std.json.Value, out: []Podcast) ?usize { + if (root != .object) return null; + const rows = root.object.get("results") orelse return null; + if (rows != .array) return null; + var count: usize = 0; + for (rows.array.items) |entry| { + if (count == out.len) break; + if (entry != .object) continue; + const feed = valueText(entry, "feedUrl"); + if (feed.len == 0 or feed.len > 300 or (!std.mem.startsWith(u8, feed, "https://") and !std.mem.startsWith(u8, feed, "http://"))) continue; + var title = valueText(entry, "collectionName"); + if (title.len == 0) title = valueText(entry, "trackName"); + if (title.len == 0) continue; + var row: Podcast = .{}; + copyValueText(title, &row.name, &row.name_len); + copyValueText(feed, &row.feed_url, &row.feed_url_len); + var art = valueText(entry, "artworkUrl600"); + if (art.len == 0) art = valueText(entry, "artworkUrl100"); + if (art.len <= row.artwork.len) copyValueText(art, &row.artwork, &row.artwork_len); + copyValueText(valueText(entry, "artistName"), &row.artist, &row.artist_len); + out[count] = row; + count += 1; + } + return count; +} + +fn valueText(value: std.json.Value, key: []const u8) []const u8 { + const field = value.object.get(key) orelse return ""; + return if (field == .string) field.string else ""; +} + +fn copyValueText(src: []const u8, out: []u8, len: *usize) void { + len.* = @min(src.len, out.len); + while (len.* > 0 and !std.unicode.utf8ValidateSlice(src[0..len.*])) len.* -= 1; + @memcpy(out[0..len.*], src[0..len.*]); +} + // ══════════════════════════════════════════════════════════ // Shared helpers // ══════════════════════════════════════════════════════════ @@ -326,24 +367,36 @@ pub fn buildLookupUrl(ids_csv: []const u8, dst: []u8) []const u8 { /// keeps rows that have both a title and an audio enclosure URL. Returns the /// number of episodes written (≀ out.len). pub fn parseRssEpisodes(xml: []const u8, out: []Episode) usize { + return parseRssEpisodePage(xml, out, 0).count; +} + +pub const EpisodePage = struct { count: usize, total: usize }; + +/// A bounded window over all playable enclosures in the feed. The cursor counts +/// usable episodes, so missing enclosures do not create gaps or repeated pages. +pub fn parseRssEpisodePage(xml: []const u8, out: []Episode, offset: usize) EpisodePage { var count: usize = 0; + var total: usize = 0; var pos: usize = 0; - while (pos < xml.len and count < out.len) { + while (pos < xml.len) { const item_start = std.mem.indexOfPos(u8, xml, pos, "") orelse break; const block = xml[item_start..item_end]; pos = item_end + "".len; - var e = &out[count]; - e.* = .{}; + var episode: Episode = .{}; + const e = &episode; // Audio enclosure URL is the load-bearing field. if (std.mem.indexOf(u8, block, "') orelse block.len; const enc = block[enc_at..@min(enc_end + 1, block.len)]; - if (xmlAttr(enc, "url=\"")) |u| e.audio_url_len = xmlText(&e.audio_url, u); + if (xmlAttr(enc, "url=\"")) |u| { + if (u.len > e.audio_url.len) continue; + e.audio_url_len = xmlText(&e.audio_url, u); + } } - if (e.audio_url_len == 0) continue; + if (!isFeedUrl(e.audio_url[0..e.audio_url_len])) continue; if (xmlTag(block, "", "")) |t| e.title_len = xmlText(&e.title, t); if (e.title_len == 0) continue; @@ -351,10 +404,40 @@ pub fn parseRssEpisodes(xml: []const u8, out: []Episode) usize { if (xmlTag(block, "", "")) |d| e.date_len = xmlText(&e.date, d); if (xmlTag(block, "", "")) |d| e.duration_len = xmlText(&e.duration, d); + if (xmlTag(block, "", "") orelse xmlTag(block, "", "")) |description| + e.summary_len = plainXmlText(&e.summary, description); + if (total >= offset and count < out.len) { + out[count] = episode; + count += 1; + } + total += 1; + } + return .{ .count = count, .total = total }; +} +fn plainXmlText(out: []u8, html: []const u8) usize { + var stripped: [1024]u8 = undefined; + var count: usize = 0; + var inside = false; + for (html) |ch| { + if (ch == '<') { + inside = true; + continue; + } + if (ch == '>') { + inside = false; + if (count > 0 and count < stripped.len and stripped[count - 1] != ' ') { + stripped[count] = ' '; + count += 1; + } + continue; + } + if (inside) continue; + if (count == stripped.len) break; + stripped[count] = ch; count += 1; } - return count; + return xmlText(out, std.mem.trim(u8, stripped[0..count], " \t\r\n")); } // ══════════════════════════════════════════════════════════ diff --git a/src/services/public_books_pure.zig b/src/services/public_books_pure.zig new file mode 100644 index 00000000..7372c520 --- /dev/null +++ b/src/services/public_books_pure.zig @@ -0,0 +1,236 @@ +//! Primary-provider public book discovery. No guessed download links: +//! Gutenberg resolves its advertised UTF-8 download; OpenLibrary supplies IA IDs. +const std = @import("std"); +const cj = @import("comics_pure.zig"); +const html = @import("content_html_pure.zig"); + +pub const Item = struct { + title: []const u8, + author: []const u8 = "", + id: []const u8, + cover_id: u64 = 0, + cover: []const u8 = "", + year: u16 = 0, +}; + +pub fn openLibrarySearch(out: []u8, query: []const u8, page: u32) ?[]const u8 { + if (query.len == 0 or page == 0) return null; + var encoded: [1536]u8 = undefined; + const n = cj.percentEncodeStrict(query, &encoded); + return std.fmt.bufPrint(out, "https://openlibrary.org/search.json?q={s}%20AND%20ebook_access%3Apublic&fields=key,title,author_name,first_publish_year,cover_i,ia,ebook_access&limit=20&page={d}", .{ encoded[0..n], page }) catch null; +} + +pub fn firstString(value: ?std.json.Value) []const u8 { + const v = value orelse return ""; + if (v == .string) return v.string; + if (v == .array) for (v.array.items) |entry| { + if (entry == .string and entry.string.len > 0) return entry.string; + }; + return ""; +} + +pub fn openLibraryItem(value: std.json.Value) ?Item { + if (value != .object) return null; + const access = firstString(value.object.get("ebook_access")); + if (!std.mem.eql(u8, access, "public")) return null; + const id = firstString(value.object.get("ia")); + const title = firstString(value.object.get("title")); + if (!archiveId(id) or title.len == 0) return null; + var item: Item = .{ .id = id, .title = title, .author = firstString(value.object.get("author_name")) }; + if (value.object.get("cover_i")) |cover| if (cover == .integer and cover.integer > 0) { + item.cover_id = @intCast(cover.integer); + }; + if (value.object.get("first_publish_year")) |year| if (year == .integer and year.integer > 0 and year.integer <= 9999) { + item.year = @intCast(year.integer); + }; + return item; +} + +pub fn archiveId(id: []const u8) bool { + if (id.len == 0 or id.len > 512) return false; + for (id) |ch| if (!std.ascii.isAlphanumeric(ch) and ch != '-' and ch != '_' and ch != '.') return false; + return true; +} + +pub fn openLibraryCover(out: []u8, cover: u64) ?[]const u8 { + if (cover == 0) return null; + return std.fmt.bufPrint(out, "https://covers.openlibrary.org/b/id/{d}-M.jpg?default=false", .{cover}) catch null; +} + +pub fn gutenbergSearch(out: []u8, query: []const u8, start: u32) ?[]const u8 { + if (query.len == 0 or start == 0) return null; + var encoded: [1536]u8 = undefined; + const n = cj.percentEncodeStrict(query, &encoded); + return std.fmt.bufPrint(out, "https://www.gutenberg.org/ebooks/search/?query={s}&start_index={d}", .{ encoded[0..n], start }) catch null; +} + +pub fn gutenbergId(route: []const u8) ?[]const u8 { + const prefix = "/ebooks/"; + if (!std.mem.startsWith(u8, route, prefix)) return null; + const id = route[prefix.len..]; + if (id.len == 0 or id.len > 10) return null; + for (id) |ch| if (!std.ascii.isDigit(ch)) return null; + return id; +} + +fn span(block: []const u8, class: []const u8) []const u8 { + var marker: [64]u8 = undefined; + const open = std.fmt.bufPrint(&marker, "", .{class}) catch return ""; + const at = std.mem.indexOf(u8, block, open) orelse return ""; + const start = at + open.len; + const end = std.mem.indexOfPos(u8, block, start, "") orelse return ""; + return block[start..end]; +} + +pub const GutenbergIter = struct { + body: []const u8, + pos: usize = 0, + pub fn next(self: *GutenbergIter) ?Item { + while (std.mem.indexOfPos(u8, self.body, self.pos, "
  • ")) |start| { + const end = std.mem.indexOfPos(u8, self.body, start, "
    • ") orelse return null; + self.pos = end + 5; + const block = self.body[start..end]; + const a = std.mem.indexOf(u8, block, "') orelse continue; + const route = html.attr(block[a..a_end], "href") orelse continue; + const id = gutenbergId(route) orelse continue; + const title = span(block, "title"); + if (title.len == 0) continue; + var cover: []const u8 = ""; + if (std.mem.indexOf(u8, block, "')) |im_end| cover = html.attr(block[im..im_end], "src") orelse ""; + } + return .{ .id = id, .title = title, .author = span(block, "subtitle"), .cover = cover }; + } + return null; + } +}; + +/// The actual next-results link carries query= as well as start_index. Header +/// rel=next can omit the query; read its cursor and rebuild our original query. +pub fn gutenbergNext(body: []const u8) ?u32 { + const marker = "title=\"Next Page\""; + const at = std.mem.indexOf(u8, body, marker) orelse return null; + const end = std.mem.indexOfScalarPos(u8, body, at, '>') orelse return null; + const route = html.attr(body[at..end], "href") orelse return null; + const cursor = std.mem.indexOf(u8, route, "start_index=") orelse return null; + const tail = route[cursor + "start_index=".len ..]; + var n: usize = 0; + while (n < tail.len and std.ascii.isDigit(tail[n])) : (n += 1) {} + if (n == 0) return null; + const value = std.fmt.parseInt(u32, tail[0..n], 10) catch return null; + return if (value > 0) value else null; +} + +pub fn gutenbergDetail(out: []u8, id: []const u8) ?[]const u8 { + var route: [64]u8 = undefined; + const path = std.fmt.bufPrint(&route, "/ebooks/{s}", .{id}) catch return null; + if (gutenbergId(path) == null) return null; + return std.fmt.bufPrint(out, "https://www.gutenberg.org{s}", .{path}) catch null; +} + +pub fn gutenbergTextUrl(out: []u8, body: []const u8) ?[]const u8 { + var p: usize = 0; + while (std.mem.indexOfPos(u8, body, p, "') orelse return null; + p = end + 1; + const tag = body[at..end]; + const mime = html.attr(tag, "type") orelse continue; + if (!std.mem.startsWith(u8, mime, "text/plain") or std.mem.indexOf(u8, mime, "utf-8") == null) continue; + const route = html.attr(tag, "href") orelse continue; + return html.sourceUrl(out, "https://www.gutenberg.org", route); + } + return null; +} + +test "OpenLibrary public book metadata excludes restricted and unavailable books" { + const json = "{\"title\":\"Frankenstein\",\"author_name\":[\"Mary Shelley\"],\"first_publish_year\":1818,\"cover_i\":123,\"ia\":[\"frankenstein_scan\"],\"ebook_access\":\"public\"}"; + const p = try std.json.parseFromSlice(std.json.Value, std.testing.allocator, json, .{}); + defer p.deinit(); + const item = openLibraryItem(p.value).?; + try std.testing.expectEqualStrings("Mary Shelley", item.author); + try std.testing.expectEqual(@as(u16, 1818), item.year); + const restricted = try std.json.parseFromSlice(std.json.Value, std.testing.allocator, "{\"title\":\"Book\",\"ia\":[\"scan\"],\"ebook_access\":\"borrowable\"}", .{}); + defer restricted.deinit(); + try std.testing.expect(openLibraryItem(restricted.value) == null); + try std.testing.expect(!archiveId("../unsafe")); +} + +test "Gutenberg parser keeps author cover and authoritative cursor and rejects invented text URLs" { + const body = "
  • FrankensteinMary Shelley
    • "; + var it = GutenbergIter{ .body = body }; + const item = it.next().?; + try std.testing.expectEqualStrings("84", item.id); + try std.testing.expectEqualStrings("Mary Shelley", item.author); + try std.testing.expectEqualStrings("/cover.jpg", item.cover); + try std.testing.expectEqual(@as(?u32, 26), gutenbergNext(body)); + var out: [512]u8 = undefined; + try std.testing.expectEqualStrings("https://www.gutenberg.org/ebooks/84.txt.utf-8", gutenbergTextUrl(&out, "Plain Text").?); + try std.testing.expect(gutenbergTextUrl(&out, "Plain Text") == null); + try std.testing.expect(gutenbergId("/ebooks/84?bad") == null); +} + +/// Refuse dark/restricted items and private derivatives before claiming a +/// catalog work has a readable Archive text. Metadata availability alone is +/// insufficient: borrowed editions can advertise OCR files without public URLs. +pub fn publicArchiveText(root: std.json.Value) ?[]const u8 { + if (root != .object) return null; + if (truth(root.object.get("is_dark"))) return null; + if (root.object.get("metadata")) |metadata| { + if (metadata == .object and truth(metadata.object.get("access-restricted-item"))) return null; + } + const files = root.object.get("files") orelse return null; + if (files != .array) return null; + var best: ?[]const u8 = null; + var best_tier: u8 = 255; + var best_size: u64 = 0; + for (files.array.items) |file| { + if (file != .object or truth(file.object.get("private"))) continue; + const name = firstString(file.object.get("name")); + if (!std.mem.endsWith(u8, name, ".txt") or std.mem.endsWith(u8, name, "_meta.txt") or std.mem.indexOf(u8, name, "searchtext") != null) continue; + const tier: u8 = if (std.mem.endsWith(u8, name, "_djvu.txt")) 0 else 1; + const size_value = file.object.get("size"); + const size: u64 = if (size_value != null and size_value.? == .integer and size_value.?.integer > 0) + @intCast(size_value.?.integer) + else + std.fmt.parseInt(u64, firstString(size_value), 10) catch 0; + if (tier < best_tier or (tier == best_tier and size > best_size)) { + best = name; + best_tier = tier; + best_size = size; + } + } + return best; +} + +fn truth(value: ?std.json.Value) bool { + const v = value orelse return false; + return switch (v) { + .bool => v.bool, + .string => std.ascii.eqlIgnoreCase(v.string, "true"), + else => false, + }; +} + +pub fn plainTextPrefix(body: []const u8, capacity: usize) []const u8 { + if (!std.unicode.utf8ValidateSlice(body)) return ""; + var n = @min(body.len, capacity); + while (n > 0 and n < body.len and body[n] & 0xC0 == 0x80) : (n -= 1) {} + return body[0..n]; +} + +test "public Archive full text excludes private OCR and restricted editions" { + const json = "{\"files\":[{\"name\":\"private_djvu.txt\",\"private\":\"true\",\"size\":\"9000\"},{\"name\":\"book.txt\",\"size\":\"1000\"},{\"name\":\"public_djvu.txt\",\"size\":\"2000\"}]}"; + const p = try std.json.parseFromSlice(std.json.Value, std.testing.allocator, json, .{}); + defer p.deinit(); + try std.testing.expectEqualStrings("public_djvu.txt", publicArchiveText(p.value).?); + const r = try std.json.parseFromSlice(std.json.Value, std.testing.allocator, "{\"metadata\":{\"access-restricted-item\":\"true\"},\"files\":[{\"name\":\"book.txt\"}]}", .{}); + defer r.deinit(); + try std.testing.expect(publicArchiveText(r.value) == null); +} + +test "Gutenberg plain text preserves comparison signs and UTF8 boundaries" { + try std.testing.expectEqualStrings("1 < 2\nA > B", plainTextPrefix("1 < 2\nA > B", 100)); + try std.testing.expectEqualStrings("A", plainTextPrefix("A東京", 2)); + try std.testing.expectEqualStrings("", plainTextPrefix("\xff", 1)); +} diff --git a/src/services/radio.zig b/src/services/radio.zig index bd756bf4..7af95478 100644 --- a/src/services/radio.zig +++ b/src/services/radio.zig @@ -206,9 +206,8 @@ fn seedPopularFromCache() void { const POPULAR_LIMIT: usize = 30; // ── Infinite-scroll pagination ── -// `current_offset` is the RadioBrowser `offset` last fetched (bookkeeping, -// mirrors drama.zig's current_page); the next loadMore() window starts at the -// current (deduped) result_count. `more_available` clears when a window comes +// `current_offset` is the next provider offset, independent of filtered or +// duplicate station rows in the displayed results. `more_available` clears when a window comes // back shorter than RADIO_PAGE_SIZE or the fixed results[] buffer fills. // `loading_more` serializes append fetches so a single near-bottom scroll // can't spawn a burst (mirrors comics/drama/youtube). All three are read on @@ -293,7 +292,13 @@ fn popularWorker(my_gen: u32) void { defer parse_mutex.unlock(); if (!search_request.isCurrent(my_gen)) return; // re-check under lock - const count = pure.parseStations(body, &state.app.radio.results); + const page = pure.parsePage(alloc, body, &state.app.radio.results) orelse { + state.app.radio.fetch_error = true; + return; + }; + const count = page.count; + more_available = page.consumed >= RADIO_PAGE_SIZE; + current_offset = page.consumed; state.app.radio.result_count = count; if (count == 0) { state.app.radio.fetch_error = true; @@ -368,7 +373,13 @@ fn searchWorker(job: SearchJob) void { defer parse_mutex.unlock(); if (!search_request.isCurrent(my_gen)) return; // re-check under lock - const count = pure.parseStations(body, &state.app.radio.results); + const page = pure.parsePage(alloc, body, &state.app.radio.results) orelse { + state.app.radio.fetch_error = true; + return; + }; + const count = page.count; + more_available = page.consumed >= RADIO_PAGE_SIZE; + current_offset = page.consumed; state.app.radio.result_count = count; if (count == 0) logs.pushLog("info", "radio", "Search returned no stations", false) else logs.pushLog("info", "radio", "Radio search done (RadioBrowser)", false); } @@ -377,8 +388,7 @@ fn searchWorker(job: SearchJob) void { // Infinite scroll β€” fetch + append the NEXT window (same query/popular mode) // ══════════════════════════════════════════════════════════ -/// Fetch the next window of stations (offset = the current, already-deduped -/// result_count) and append it onto the existing grid. Guarded by +/// Fetch the next provider window of stations and append playable unique rows. Guarded by /// `loading_more` + the main `is_loading` so a near-bottom scroll can't spawn /// a burst; runs under the current `search_request` so a fresh search/popular /// reload supersedes it. No-op once `more_available` clears (a short window or @@ -387,7 +397,7 @@ pub fn loadMore() void { if (!more_available) return; if (state.app.radio.is_loading.load(.acquire)) return; if (loading_more.load(.acquire)) return; - if (state.app.radio.result_count == 0) return; + if (state.app.radio.result_count >= state.app.radio.results.len) { more_available = false; return; @@ -395,7 +405,7 @@ pub fn loadMore() void { if (loading_more.swap(true, .acq_rel)) return; // lost the race β€” another append in flight const my_gen = search_request.current(); // stay within the current generation - const offset = current_offset + RADIO_PAGE_SIZE; + const offset = current_offset; const popular = state.app.radio.showing_popular; var job: SearchJob = .{ .generation = my_gen }; parse_mutex.lock(); @@ -431,10 +441,14 @@ fn loadMoreWorker(job: SearchJob, offset: usize, popular: bool) void { // as the initial fetches. rate_limit.acquire("radiobrowser", 1.0); - // Best-effort: a failed append just leaves more_available as-is (retried on - // the next near-bottom scroll) rather than surfacing the page-level - // fetch_error banner over an already-populated grid. - const body = fetchBody(url, 512 * 1024) orelse return; + // Keep loaded rows on failure and surface it; do not retry every UI frame. + const body = fetchBody(url, 512 * 1024) orelse { + if (search_request.isCurrent(my_gen)) { + state.app.radio.fetch_error = true; + more_available = false; + } + return; + }; defer alloc.free(body); if (!search_request.isCurrent(my_gen)) return; // superseded by a fresh search/popular reload @@ -443,19 +457,22 @@ fn loadMoreWorker(job: SearchJob, offset: usize, popular: bool) void { // thread (CLAUDE.md). const staged = alloc.alloc(pure.Station, RADIO_PAGE_SIZE) catch return; defer alloc.free(staged); - const n = pure.parseStations(body, staged); + const page = pure.parsePage(alloc, body, staged) orelse { + if (search_request.isCurrent(my_gen)) { + state.app.radio.fetch_error = true; + more_available = false; + } + return; + }; + const n = page.count; parse_mutex.lock(); defer parse_mutex.unlock(); if (!search_request.isCurrent(my_gen)) return; // re-check under lock - current_offset = offset; - // A window shorter than the page size means RadioBrowser has nothing left - // to page (parseStations already drops unplayable rows, so this is a - // conservative approximation β€” a window that was all-filtered would also - // read as "short" here, which just ends paging a little early rather than - // ever looping forever). - if (n < RADIO_PAGE_SIZE) more_available = false; + current_offset = offset + page.consumed; + more_available = page.consumed >= RADIO_PAGE_SIZE; + state.app.radio.fetch_error = false; const base = state.app.radio.result_count; var appended: usize = 0; @@ -624,6 +641,11 @@ fn fetchBody(url: []const u8, cap: usize) ?[]u8 { }) orelse continue; const trimmed = std.mem.trim(u8, body, " \t\r\n"); if (trimmed.len == 0 or trimmed[0] != '[') continue; + // A mirror returning truncated/malformed JSON is a failed host too. + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch continue; + const valid = parsed.value == .array; + parsed.deinit(); + if (!valid) continue; return alloc.realloc(buf, body.len) catch { alloc.free(buf); return null; @@ -716,21 +738,16 @@ fn renderLogo(i: usize, s: *const pure.Station) void { const slot = &station_posters[i]; const fav = s.favicon[0..s.favicon_len]; - if (fav.len > 0) { - // Pin the slot to whatever station is at index i now β€” a re-search (or - // the popular list landing) can replace results[], so a URL-hash change - // means "different station here": free the stale texture/pixels and - // refetch. Only when not mid-fetch, so we never spawn a second worker - // onto the same slot. - const h = std.hash.Fnv1a_64.hash(fav); - if (slot.url_hash != h and !slot.fetching) { - poster.deinitPoster(&slot.pixels, &slot.tex); - slot.w = 0; - slot.h = 0; - slot.attempted = false; - slot.failed = false; - slot.url_hash = h; - } + const h = std.hash.Fnv1a_64.hash(fav); + if (slot.url_hash != h and !slot.fetching) { + poster.deinitPoster(&slot.pixels, &slot.tex); + slot.w = 0; + slot.h = 0; + slot.attempted = false; + slot.failed = false; + slot.url_hash = h; + } + if (fav.len > 0 and slot.url_hash == h) { _ = poster.uploadIfReady(&slot.pixels, slot.w, slot.h, &slot.tex); if (slot.fetching) slot.attempted = true else if (slot.attempted and slot.pixels == null and slot.tex == null) slot.failed = true; if (!slot.failed and slot.tex == null and !slot.fetching and slot.pixels == null) { @@ -739,8 +756,9 @@ fn renderLogo(i: usize, s: *const pure.Station) void { } } - if (slot.tex) |*tex| { - _ = dvui.image(@src(), .{ .source = .{ .texture = tex.* } }, .{ + const visible_texture = if (fav.len > 0 and slot.url_hash == h) slot.tex else null; + if (visible_texture) |tex| { + _ = dvui.image(@src(), .{ .source = .{ .texture = tex } }, .{ .id_extra = i + 1000, .expand = .both, .corner_radius = dvui.Rect.all(8), @@ -871,7 +889,8 @@ fn renderResults() void { const count = @min(state.app.radio.result_count, state.app.radio.results.len); const showing_popular = state.app.radio.showing_popular; parse_mutex.unlock(); - if (count == 0 and !state.app.radio.is_loading.load(.acquire)) { + if (count == 0 and more_available and !state.app.radio.is_loading.load(.acquire)) loadMore(); + if (count == 0 and !state.app.radio.is_loading.load(.acquire) and !loading_more.load(.acquire)) { components.emptyState(icons.tvg.lucide.radio, "Find a station", "Search by station, genre, or country."); return; } diff --git a/src/services/radio_pure.zig b/src/services/radio_pure.zig index 2312fcf8..9f316683 100644 --- a/src/services/radio_pure.zig +++ b/src/services/radio_pure.zig @@ -256,6 +256,50 @@ pub fn parseStations(json: []const u8, out: []Station) usize { return count; } +/// Fully validate a directory page and preserve the provider's cursor even +/// when entries are not playable. JSON whitespace/key ordering is irrelevant. +pub const StationPage = struct { count: usize, consumed: usize }; +pub fn parsePage(a: std.mem.Allocator, json: []const u8, out: []Station) ?StationPage { + var parsed = std.json.parseFromSlice(std.json.Value, a, json, .{}) catch return null; + defer parsed.deinit(); + if (parsed.value != .array) return null; + var count: usize = 0; + for (parsed.value.array.items) |obj| { + if (count == out.len) break; + if (obj != .object) continue; + var station: Station = .{}; + jsonCopy(obj, "name", &station.name, &station.name_len); + jsonCopy(obj, "url_resolved", &station.url_resolved, &station.url_resolved_len); + jsonCopy(obj, "url", &station.url, &station.url_len); + if (station.name_len == 0 or (station.url_len == 0 and station.url_resolved_len == 0)) continue; + jsonCopy(obj, "stationuuid", &station.stationuuid, &station.stationuuid_len); + jsonCopy(obj, "favicon", &station.favicon, &station.favicon_len); + jsonCopy(obj, "tags", &station.tags, &station.tags_len); + jsonCopy(obj, "country", &station.country, &station.country_len); + jsonCopy(obj, "codec", &station.codec, &station.codec_len); + station.votes = jsonNumber(obj, "votes"); + station.bitrate = jsonNumber(obj, "bitrate"); + out[count] = station; + count += 1; + } + return .{ .count = count, .consumed = parsed.value.array.items.len }; +} + +fn jsonCopy(obj: std.json.Value, key: []const u8, dst: []u8, len: *usize) void { + const value = obj.object.get(key) orelse return; + if (value != .string) return; + len.* = @min(value.string.len, dst.len); + while (len.* > 0 and !std.unicode.utf8ValidateSlice(value.string[0..len.*])) len.* -= 1; + @memcpy(dst[0..len.*], value.string[0..len.*]); +} +fn jsonNumber(obj: std.json.Value, key: []const u8) u32 { + const value = obj.object.get(key) orelse return 0; + if (value == .integer and value.integer > 0) + return @intCast(@min(value.integer, std.math.maxInt(u32))); + if (value == .string) return std.fmt.parseInt(u32, value.string, 10) catch 0; + return 0; +} + // ══════════════════════════════════════════════════════════ // Tests // ══════════════════════════════════════════════════════════ diff --git a/src/services/remote.zig b/src/services/remote.zig index db730554..cfd84718 100644 --- a/src/services/remote.zig +++ b/src/services/remote.zig @@ -3450,9 +3450,14 @@ fn apiPodcasts(stream: std.Io.net.Stream, api_path: []const u8, query: []const u } // GET /podcasts β†’ results + episodes for the current show. podcasts_svc.loadPopularOnce(); - const view = podcasts_svc.snapshot(); const allocator = @import("../core/alloc.zig").allocator; - const json_buf = allocator.alloc(u8, 192 * 1024) catch { + const view = allocator.create(podcasts_svc.Snapshot) catch { + sendJsonStatus(stream, "503 Service Unavailable", "{\"error\":\"podcast snapshot unavailable\"}"); + return; + }; + defer allocator.destroy(view); + podcasts_svc.copySnapshot(view); + const json_buf = allocator.alloc(u8, @sizeOf(podcasts_svc.Snapshot) * 6 + 8192) catch { sendJsonStatus(stream, "503 Service Unavailable", "{\"error\":\"podcast view unavailable\"}"); return; }; @@ -3679,7 +3684,16 @@ fn apiDrama(stream: std.Io.net.Stream, api_path: []const u8, query: []const u8) /// whichever view is live, so one poll drives the whole flow. fn apiNovels(stream: std.Io.net.Stream, api_path: []const u8, query: []const u8) void { const nov = @import("novels.zig"); - const n = &state.app.novels; + if (std.mem.eql(u8, api_path, "/novels/back")) { + nov.back(); + sendJson(stream, "{\"ok\":true}"); + return; + } + if (std.mem.eql(u8, api_path, "/novels/more")) { + nov.loadMore(); + sendJson(stream, "{\"ok\":true}"); + return; + } if (std.mem.eql(u8, api_path, "/novels/search")) { if (getQueryParam(query, "q")) |q| { @@ -3721,15 +3735,25 @@ fn apiNovels(stream: std.Io.net.Stream, api_path: []const u8, query: []const u8) } const a = @import("../core/alloc.zig").allocator; - // text_buf alone is 128KB, and JSON escaping can nearly double it. - const json_buf = a.alloc(u8, 384 * 1024) catch return; + const n = a.create(nov.ReaderSnapshot) catch return; + defer a.destroy(n); + nov.copyReaderSnapshot(n); + // Size from the immutable view, including worst-case JSON escapes. Large + // enriched catalogs must not silently cut off chapter lists or reader text. + var json_cap: usize = 8192 + (n.text_len + n.work_title_len + n.chapter_label_len) * 6; + for (n.results[0..n.result_count]) |*row| { + json_cap += (row.title().len + row.url().len + row.metadata.author().len + + row.metadata.overview().len + row.metadata.cover().len) * 6 + 128; + } + for (n.chapter_title_lens[0..n.chapter_count]) |length| json_cap += length * 6 + 32; + const json_buf = a.alloc(u8, json_cap) catch return; defer a.free(json_buf); var w = std.Io.Writer.fixed(json_buf); w.print("{{\"view\":\"{s}\",\"loading\":{s},\"chapters_loading\":{s},\"text_loading\":{s},\"error\":{s},\"title\":\"", .{ @tagName(n.view), - if (n.is_loading.load(.acquire)) "true" else "false", - if (n.chapters_loading.load(.acquire)) "true" else "false", - if (n.text_loading.load(.acquire)) "true" else "false", + if (n.loading) "true" else "false", + if (n.chapters_loading) "true" else "false", + if (n.text_loading) "true" else "false", if (n.fetch_error) "true" else "false", }) catch return; escJsonWrite(&w, txt.safeUtf8(n.work_title[0..@min(n.work_title_len, n.work_title.len)])); @@ -3741,27 +3765,41 @@ fn apiNovels(stream: std.Io.net.Stream, api_path: []const u8, query: []const u8) }) catch return; var i: usize = 0; - const rn = nov.resultCount(); + const rn = n.result_count; while (i < rn) : (i += 1) { - const row = nov.resultRow(i) orelse break; + const row = &n.results[i]; if (i > 0) w.writeAll(",") catch return; w.writeAll("{\"title\":\"") catch return; escJsonWrite(&w, txt.safeUtf8(row.title())); - w.print("\",\"source\":{d}}}", .{row.source}) catch return; + w.writeAll("\",\"url\":\"") catch return; + escJsonWrite(&w, row.url()); + w.writeAll("\",\"author\":\"") catch return; + escJsonWrite(&w, txt.safeUtf8(row.metadata.author())); + w.writeAll("\",\"overview\":\"") catch return; + escJsonWrite(&w, txt.safeUtf8(row.metadata.overview())); + w.writeAll("\",\"cover\":\"") catch return; + escJsonWrite(&w, row.metadata.cover()); + w.print("\",\"year\":{d},\"source\":{d}}}", .{ row.metadata.year, row.source }) catch return; } w.writeAll("],\"chapters\":[") catch return; i = 0; - const cn = nov.chapterCount(); + const cn = n.chapter_count; while (i < cn) : (i += 1) { - const row = nov.chapterRow(i) orelse break; + const chapter_title = n.chapter_titles[i][0..n.chapter_title_lens[i]]; if (i > 0) w.writeAll(",") catch return; w.writeAll("{\"title\":\"") catch return; - escJsonWrite(&w, txt.safeUtf8(row.title())); + escJsonWrite(&w, txt.safeUtf8(chapter_title)); w.writeAll("\"}") catch return; } w.writeAll("],\"text\":\"") catch return; escJsonWrite(&w, txt.safeUtf8(n.text_buf[0..@min(n.text_len, n.text_buf.len)])); - w.writeAll("\"}") catch return; + w.print("\",\"has_more\":{s},\"loading_more\":{s},\"search_generation\":{d},\"chapter_generation\":{d},\"text_generation\":{d}}}", .{ + if (n.has_more) "true" else "false", + if (n.loading_more) "true" else "false", + n.search_generation, + n.chapter_generation, + n.text_generation, + }) catch return; sendJson(stream, json_buf[0..w.end]); } @@ -3782,9 +3820,7 @@ fn apiAbs(stream: std.Io.net.Stream, method: []const u8, api_path: []const u8, q defer @memset(&dec, 0); if (credParam(body, "", "server", &dec)) |v| { const sv = txt.safeUtf8(v); - const n = @min(sv.len, s.server_url.len); - @memcpy(s.server_url[0..n], sv[0..n]); - s.server_url_len = n; + abs.setServerUrl(sv); } if (credParam(body, "", "user", &dec)) |v| { const uv = txt.safeUtf8(v); @@ -3850,46 +3886,101 @@ fn apiAbs(stream: std.Io.net.Stream, method: []const u8, api_path: []const u8, q return; } + if (std.mem.eql(u8, api_path, "/abs/audio")) { + if (!requireMethod(stream, method, "POST")) return; + const idx = std.fmt.parseInt(usize, getQueryParam(query, "idx") orelse "", 10) catch { + sendJsonStatus(stream, "400 Bad Request", "{\"error\":\"invalid track index\"}"); + return; + }; + const generation = std.fmt.parseInt(u32, getQueryParam(query, "generation") orelse "", 10) catch { + sendJsonStatus(stream, "400 Bad Request", "{\"error\":\"invalid audio generation\"}"); + return; + }; + if (!abs.playAudioTrack(idx, generation)) { + sendJsonStatus(stream, "409 Conflict", "{\"error\":\"audio selection changed; refresh and retry\"}"); + return; + } + sendJson(stream, "{\"ok\":true,\"action\":\"abs_audio\"}"); + return; + } + if (std.mem.eql(u8, api_path, "/abs/audio/book")) { + if (!requireMethod(stream, method, "POST")) return; + const generation = std.fmt.parseInt(u32, getQueryParam(query, "generation") orelse "", 10) catch { + sendJsonStatus(stream, "400 Bad Request", "{\"error\":\"invalid audio generation\"}"); + return; + }; + if (!abs.playWholeBook(generation)) { + sendJsonStatus(stream, "409 Conflict", "{\"error\":\"complete book timeline is unavailable; refresh selection\"}"); + return; + } + sendJson(stream, "{\"ok\":true}"); + return; + } + if (std.mem.eql(u8, api_path, "/abs/audio/close")) { + if (!requireMethod(stream, method, "POST")) return; + abs.clearAudioSelection(); + sendJson(stream, "{\"ok\":true}"); + return; + } + if (!std.mem.eql(u8, api_path, "/abs")) { sendJsonStatus(stream, "404 Not Found", "{\"error\":\"unknown Audiobookshelf action\"}"); return; } if (!requireMethod(stream, method, "GET")) return; const a = @import("../core/alloc.zig").allocator; - const json_buf = a.alloc(u8, 192 * 1024) catch return; + const pure = @import("audiobookshelf_pure.zig"); + const books = a.alloc(pure.Book, 320) catch return; + defer a.free(books); + var libraries: [16]pure.Library = undefined; + const catalog = abs.catalogSnapshot(books, &libraries); + const tracks = a.alloc(pure.AudioTrack, 128) catch return; + defer a.free(tracks); + const audio = abs.audioSnapshot(tracks); + const json_buf = a.alloc(u8, 1280 * 1024) catch return; defer a.free(json_buf); var w = std.Io.Writer.fixed(json_buf); - w.print("{{\"connected\":{s},\"loading\":{s},\"view\":\"{s}\",\"server\":\"", .{ - if (s.connected) "true" else "false", - if (s.is_loading.load(.acquire)) "true" else "false", - @tagName(s.view), + w.print("{{\"connected\":{s},\"loading\":{s},\"loading_more\":{s},\"has_more\":{s},\"view\":\"{s}\",\"server\":\"", .{ + if (catalog.connected) "true" else "false", if (catalog.loading) "true" else "false", + if (catalog.loading_more) "true" else "false", if (catalog.has_more) "true" else "false", + catalog.view, }) catch return; - escJsonWrite(&w, s.server_url[0..@min(s.server_url_len, s.server_url.len)]); + escJsonWrite(&w, catalog.server[0..catalog.server_len]); w.writeAll("\",\"error\":\"") catch return; - escJsonWrite(&w, txt.safeUtf8(s.login_error[0..@min(s.login_error_len, s.login_error.len)])); + escJsonWrite(&w, txt.safeUtf8(catalog.error_text[0..catalog.error_len])); w.writeAll("\",\"library\":\"") catch return; - escJsonWrite(&w, txt.safeUtf8(s.selected_lib_name[0..@min(s.selected_lib_name_len, s.selected_lib_name.len)])); + escJsonWrite(&w, txt.safeUtf8(catalog.library[0..catalog.library_len])); w.writeAll("\",\"libraries\":[") catch return; - const ln = @min(s.library_count, s.libraries.len); - for (s.libraries[0..ln], 0..) |*l, i| { - if (i > 0) w.writeAll(",") catch return; + for (libraries[0..catalog.library_count], 0..) |*library, idx| { + if (idx > 0) w.writeAll(",") catch return; w.writeAll("{\"name\":\"") catch return; - escJsonWrite(&w, txt.safeUtf8(l.name[0..@min(l.name_len, l.name.len)])); + escJsonWrite(&w, txt.safeUtf8(library.name[0..@min(library.name_len, library.name.len)])); w.writeAll("\",\"media_type\":\"") catch return; - escJsonWrite(&w, txt.safeUtf8(l.media_type[0..@min(l.media_type_len, l.media_type.len)])); + escJsonWrite(&w, txt.safeUtf8(library.media_type[0..@min(library.media_type_len, library.media_type.len)])); w.writeAll("\"}") catch return; } w.writeAll("],\"books\":[") catch return; - const bn = @min(s.book_count, s.books.len); - for (s.books[0..bn], 0..) |*b, i| { - if (i > 0) w.writeAll(",") catch return; + for (books[0..catalog.book_count], 0..) |*book, idx| { + if (idx > 0) w.writeAll(",") catch return; w.writeAll("{\"title\":\"") catch return; - escJsonWrite(&w, txt.safeUtf8(b.title[0..@min(b.title_len, b.title.len)])); + escJsonWrite(&w, txt.safeUtf8(book.title[0..@min(book.title_len, book.title.len)])); w.writeAll("\",\"author\":\"") catch return; - escJsonWrite(&w, txt.safeUtf8(b.author[0..@min(b.author_len, b.author.len)])); - w.print("\",\"duration\":{d}}}", .{b.duration_secs}) catch return; + escJsonWrite(&w, txt.safeUtf8(book.author[0..@min(book.author_len, book.author.len)])); + w.print("\",\"duration\":{d}}}", .{book.duration_secs}) catch return; } - w.writeAll("]}") catch return; + w.print("],\"audio\":{{\"visible\":{s},\"complete_book\":{s},\"loading\":{s},\"generation\":{d},\"returned\":{d},\"total\":{d},\"truncated\":{s},\"title\":\"", .{ + if (audio.visible) "true" else "false", if (audio.complete_book) "true" else "false", if (audio.loading) "true" else "false", audio.generation, + audio.count, audio.total, if (audio.count < audio.total) "true" else "false", + }) catch return; + escJsonWrite(&w, txt.safeUtf8(audio.title[0..audio.title_len])); + w.writeAll("\",\"tracks\":[") catch return; + for (tracks[0..audio.count], 0..) |*track, idx| { + if (idx > 0) w.writeAll(",") catch return; + w.print("{{\"index\":{d},\"episode\":{s},\"title\":\"", .{ idx, if (track.episode) "true" else "false" }) catch return; + escJsonWrite(&w, txt.safeUtf8(track.title[0..track.title_len])); + w.writeAll("\"}") catch return; + } + w.writeAll("]}}") catch return; sendJson(stream, json_buf[0..w.end]); } @@ -4222,10 +4313,13 @@ fn apiComics(stream: std.Io.net.Stream, api_path: []const u8, query: []const u8) // Return current state. `downloaded` vs `pages` is what the reader polls to // know which /api/comics/page?i= indices will answer 200 rather than 404. const cm = &state.app.comic; - var json_buf: [2048]u8 = undefined; - var w = std.Io.Writer.fixed(&json_buf); + const reader_loading = cm.is_loading.load(.acquire) or cm.pending_load.load(.acquire) or cm.pending_close.load(.acquire); + const a = @import("../core/alloc.zig").allocator; + const json_buf = a.alloc(u8, 96 * 1024) catch return; + defer a.free(json_buf); + var w = std.Io.Writer.fixed(json_buf); w.print("{{\"loading\":{s},\"pages\":{d},\"downloaded\":{d},\"current\":{d},\"has_next\":{s},\"has_prev\":{s},\"title\":\"", .{ - if (cm.is_loading.load(.acquire)) "true" else "false", + if (reader_loading) "true" else "false", cm.page_count, cm.dl_progress.load(.acquire), cm.current_page, @@ -4235,7 +4329,32 @@ fn apiComics(stream: std.Io.net.Stream, api_path: []const u8, query: []const u8) escJsonWrite(&w, txt.safeUtf8(cm.title[0..@min(cm.title_len, cm.title.len)])); w.writeAll("\",\"url\":\"") catch return; escJsonWrite(&w, cm.url_buf[0..cm.url_len]); - w.writeAll("\"}") catch return; + w.writeAll("\",\"next_url\":\"") catch return; + escJsonWrite(&w, cm.next_url[0..@min(cm.next_url_len, cm.next_url.len)]); + w.writeAll("\",\"prev_url\":\"") catch return; + escJsonWrite(&w, cm.prev_url[0..@min(cm.prev_url_len, cm.prev_url.len)]); + w.writeAll("\",\"chapters\":[") catch return; + const comics_svc = @import("comics.zig"); + if (@import("comics_pure.zig").mangaIdFromRoute(cm.url_buf[0..@min(cm.url_len, cm.url_buf.len)]) != null and !reader_loading) { + var i: usize = 0; + while (comics_svc.mangaDexChapterRow(i)) |row| : (i += 1) { + if (i > 0) w.writeAll(",") catch return; + w.writeAll("{\"title\":\"") catch return; + escJsonWrite(&w, txt.safeUtf8(row.title())); + w.writeAll("\",\"url\":\"") catch return; + escJsonWrite(&w, row.route()); + w.print("\",\"selected\":{s}}}", .{if (row.selected) "true" else "false"}) catch return; + } + } + w.writeAll("],\"ready_pages\":[") catch return; + var ready_count: usize = 0; + for (0..@min(cm.page_count, cm.page_pixels.len)) |idx| { + if (!comics_svc.pageReady(idx)) continue; + if (ready_count > 0) w.writeAll(",") catch return; + w.print("{d}", .{idx}) catch return; + ready_count += 1; + } + w.writeAll("]}") catch return; sendJson(stream, json_buf[0..w.end]); } @@ -4382,46 +4501,81 @@ fn apiUnifiedResolverSearch(stream: std.Io.net.Stream, query: []const u8) void { resolver.resolve(dq[0..@min(dq.len, 255)], "auto"); } - // Snapshot under the resolver transaction, then release it before JSON - // encoding and socket I/O so a slow web client cannot stall search workers. const allocator = @import("../core/alloc.zig").allocator; - const snapshot = allocator.alloc(resolver.ResolvedItem, 80) catch { + const snapshot = allocator.alloc(resolver.ResolvedItem, resolver.MAX_RESULTS) catch { sendJsonStatus(stream, "503 Service Unavailable", "{\"error\":\"search snapshot unavailable\"}"); return; }; defer allocator.free(snapshot); const generation = resolver.lockRemoteSnapshot(); const loading = resolver.isResolving(); - const count = @min(resolver.result_count, snapshot.len); + const total = @min(resolver.result_count, resolver.results.len); + const count = @min(total, snapshot.len); @memcpy(snapshot[0..count], resolver.results[0..count]); + const SourceSnapshot = struct { name: []const u8, enabled: bool, status: resolver.SourceStatus }; + const sources = [_]SourceSnapshot{ + .{ .name = "local", .enabled = resolver.sourceOn(.local), .status = resolver.status_local.load(.acquire) }, + .{ .name = "torrent", .enabled = resolver.sourceOn(.torrent), .status = resolver.combineManySourceStatuses(&.{ resolver.status_torrent.load(.acquire), resolver.status_yts.load(.acquire), resolver.status_torznab.load(.acquire), resolver.status_eztv.load(.acquire) }) }, + .{ .name = "jellyfin", .enabled = resolver.sourceOn(.jellyfin), .status = resolver.status_jf.load(.acquire) }, + .{ .name = "youtube", .enabled = resolver.sourceOn(.youtube), .status = resolver.status_yt.load(.acquire) }, + .{ .name = "anime", .enabled = resolver.sourceOn(.anime), .status = resolver.status_anime.load(.acquire) }, + .{ .name = "comics", .enabled = resolver.sourceOn(.comics), .status = resolver.status_comics.load(.acquire) }, + .{ .name = "stremio", .enabled = resolver.sourceOn(.stremio), .status = resolver.combineManySourceStatuses(&.{ resolver.status_stremio.load(.acquire), resolver.status_archive.load(.acquire), resolver.status_nasa.load(.acquire), resolver.status_commons.load(.acquire) }) }, + .{ .name = "rss", .enabled = resolver.sourceOn(.rss), .status = resolver.status_rss.load(.acquire) }, + .{ .name = "livetv", .enabled = resolver.sourceOn(.livetv), .status = resolver.status_livetv.load(.acquire) }, + .{ .name = "music", .enabled = resolver.sourceOn(.music), .status = resolver.status_music.load(.acquire) }, + .{ .name = "radio", .enabled = resolver.sourceOn(.radio), .status = resolver.status_radio.load(.acquire) }, + .{ .name = "podcast", .enabled = resolver.sourceOn(.podcast), .status = resolver.status_podcast.load(.acquire) }, + .{ .name = "novels", .enabled = resolver.sourceOn(.novels), .status = resolver.combineSourceStatuses(resolver.status_novels.load(.acquire), resolver.status_novel_archive.load(.acquire)) }, + .{ .name = "audiobooks", .enabled = resolver.sourceOn(.audiobooks), .status = resolver.status_audiobooks.load(.acquire) }, + .{ .name = "vndb", .enabled = resolver.sourceOn(.vndb), .status = resolver.status_vndb.load(.acquire) }, + .{ .name = "tmdb", .enabled = true, .status = resolver.status_catalog.load(.acquire) }, + .{ .name = "plex", .enabled = true, .status = resolver.status_plex.load(.acquire) }, + .{ .name = "plugin", .enabled = true, .status = resolver.status_plugins.load(.acquire) }, + }; resolver.unlockRemoteSnapshot(); - // Opaque action keys preserve native playback semantics while keeping - // local paths, magnets, signed URLs and request headers out of web JSON. - var json_buf: [65536]u8 = undefined; - var w = std.Io.Writer.fixed(&json_buf); - w.print("{{\"loading\":{s},\"generation\":{d},\"results\":[", .{ - if (loading) "true" else "false", - generation, + // Each bounded row fits even if every string byte becomes a six-byte JSON + // escape. Encoding stays off the connection stack and never drops rows. + const json_buf = allocator.alloc(u8, @as(usize, count) * 12000 + 8192) catch { + sendJsonStatus(stream, "503 Service Unavailable", "{\"error\":\"search response unavailable\"}"); + return; + }; + defer allocator.free(json_buf); + var w = std.Io.Writer.fixed(json_buf); + w.print("{{\"loading\":{s},\"generation\":{d},\"returned\":{d},\"total\":{d},\"truncated\":{s},\"sources\":[", .{ + if (loading) "true" else "false", generation, count, total, if (count < total) "true" else "false", }) catch return; + for (sources, 0..) |source, idx| { + if (idx > 0) w.writeAll(",") catch return; + w.print("{{\"source\":\"{s}\",\"enabled\":{s},\"status\":\"{s}\"}}", .{ source.name, if (source.enabled) "true" else "false", @tagName(source.status) }) catch return; + } + w.writeAll("],\"results\":[") catch return; for (snapshot[0..count], 0..) |*item, idx| { - if (w.end + 3072 > json_buf.len) break; if (idx > 0) w.writeAll(",") catch return; w.writeAll("{\"source\":\"") catch return; escJsonWrite(&w, @tagName(item.source)); w.writeAll("\",\"title\":\"") catch return; - escJsonWrite(&w, item.name[0..item.name_len]); + escJsonWrite(&w, txt.safeUtf8(item.name[0..@min(item.name_len, item.name.len)])); w.writeAll("\",\"detail\":\"") catch return; - escJsonWrite(&w, item.detail[0..item.detail_len]); + escJsonWrite(&w, txt.safeUtf8(item.detail[0..@min(item.detail_len, item.detail.len)])); + w.writeAll("\",\"author\":\"") catch return; + escJsonWrite(&w, txt.safeUtf8(item.author[0..@min(item.author_len, item.author.len)])); + w.writeAll("\",\"poster_url\":\"") catch return; + escJsonWrite(&w, txt.safeUtf8(item.poster_url[0..@min(item.poster_url_len, item.poster_url.len)])); + w.writeAll("\",\"summary\":\"") catch return; + escJsonWrite(&w, txt.safeUtf8(item.summary[0..@min(item.summary_len, item.summary.len)])); + const rating = if (std.math.isFinite(item.rating) and item.rating > 0 and item.rating <= 10) item.rating else 0; + w.print("\",\"rating\":{d:.1}", .{rating}) catch return; if (item.source == .tmdb and item.catalog_id != 0) { - w.writeAll("\",\"media\":\"") catch return; - escJsonWrite(&w, item.catalog_kind[0..item.catalog_kind_len]); + w.writeAll(",\"media\":\"") catch return; + escJsonWrite(&w, item.catalog_kind[0..@min(item.catalog_kind_len, item.catalog_kind.len)]); w.print("\",\"id\":{d},\"imdb\":\"", .{item.catalog_id}) catch return; - escJsonWrite(&w, item.catalog_imdb[0..item.catalog_imdb_len]); + escJsonWrite(&w, item.catalog_imdb[0..@min(item.catalog_imdb_len, item.catalog_imdb.len)]); + w.writeAll("\"") catch return; } - w.print("\",\"key\":\"{x}\",\"queueable\":{s}}}", .{ - resolver.actionKey(item), - if (resolver.isRemoteQueueable(item)) "true" else "false", + w.print(",\"key\":\"{x}\",\"queueable\":{s}}}", .{ + resolver.actionKey(item), if (resolver.isRemoteQueueable(item)) "true" else "false", }) catch return; } w.writeAll("]}") catch return; diff --git a/src/services/resolver.zig b/src/services/resolver.zig index 617945fb..ecef2c4e 100644 --- a/src/services/resolver.zig +++ b/src/services/resolver.zig @@ -35,6 +35,9 @@ pub const SourceType = enum { podcast, // a podcast show β€” opens the Podcasts tab on that feed plex, // connected Plex library β€” credential-free deep-link playback plugin, // trusted installed executable plugin + novels, // work identity opens the built-in reader + vndb, // visual novel metadata, not downloadable game content + audiobooks, // connected Audiobookshelf item, opened through expanded audio metadata }; pub const ResolvedItem = struct { @@ -42,6 +45,13 @@ pub const ResolvedItem = struct { name_len: usize = 0, detail: [128]u8 = std.mem.zeroes([128]u8), // size, seeds, source addon, etc. detail_len: usize = 0, + poster_url: [512]u8 = std.mem.zeroes([512]u8), + poster_url_len: usize = 0, + summary: [512]u8 = std.mem.zeroes([512]u8), + summary_len: usize = 0, + rating: f32 = 0, // Provider rating out of ten; zero means unknown. + author: [160]u8 = std.mem.zeroes([160]u8), + author_len: usize = 0, url: [2048]u8 = std.mem.zeroes([2048]u8), // magnet/http/jf item id url_len: usize = 0, source: SourceType = .torrent, @@ -102,10 +112,10 @@ pub var results_mutex = @import("../core/sync.zig").Mutex{}; // Encrypted-content-cache SWR state. When a query's results are seeded from // the on-disk cache (instant, no empty view), `results_from_cache` is true; the // first live result of the fresh wave then replaces the placeholder (see -// pushResult). Guarded by results_mutex. Serialized blobs cap at ~160 KB -// (MAX_RESULTS rows Γ— fixed buffers), so 512 KB is a safe scratch size. +// pushResult). Guarded by results_mutex. Reserve enough for every bounded +// result field, including descriptions, cover URLs and playback fallbacks. var results_from_cache: bool = false; -const SEARCH_BLOB_CAP: usize = 768 * 1024; +const SEARCH_BLOB_CAP: usize = MAX_RESULTS * @sizeOf(ResolvedItem); const SEARCH_TTL_S: i64 = @import("browse_cache.zig").TTL_S; // Search state β€” shared across 7 worker threads + UI; access via atomics. @@ -137,6 +147,10 @@ pub var status_podcast = std.atomic.Value(SourceStatus).init(.idle); pub var status_catalog = std.atomic.Value(SourceStatus).init(.idle); pub var status_plex = std.atomic.Value(SourceStatus).init(.idle); pub var status_plugins = std.atomic.Value(SourceStatus).init(.idle); +pub var status_novels = std.atomic.Value(SourceStatus).init(.idle); +pub var status_novel_archive = std.atomic.Value(SourceStatus).init(.idle); +pub var status_vndb = std.atomic.Value(SourceStatus).init(.idle); +pub var status_audiobooks = std.atomic.Value(SourceStatus).init(.idle); // Explicit u8 backing so std.atomic.Value(SourceStatus) is byte-atomic. The // richer terminal values make an empty source distinguishable from a network, @@ -150,7 +164,7 @@ pub const sourceStatusIsFailure = lifecycle.isFailure; /// not spawned by resolve() (their status reads .idle) and their result /// groups are hidden. A mask of 0 is treated as "everything on" so the user /// can never filter themselves into a permanently empty search. -pub const SourceBit = enum(u4) { local, torrent, jellyfin, youtube, anime, comics, stremio, rss, livetv, music, radio, podcast }; +pub const SourceBit = enum(u4) { local, torrent, jellyfin, youtube, anime, comics, stremio, rss, livetv, music, radio, podcast, novels, vndb, audiobooks }; /// Every declared bit set. Kept in sync with SourceBit at comptime so adding a /// vertical can't silently leave its pill off (or make `all on` mis-detect). pub const ALL_SOURCE_BITS: u16 = (1 << @typeInfo(SourceBit).@"enum".fields.len) - 1; @@ -423,6 +437,10 @@ pub fn resolveTracked(query: []const u8, intent: []const u8) u32 { Pre.set(&status_music, .music); Pre.set(&status_radio, .radio); Pre.set(&status_podcast, .podcast); + Pre.set(&status_novels, .novels); + Pre.set(&status_novel_archive, .novels); + Pre.set(&status_vndb, .vndb); + Pre.set(&status_audiobooks, .audiobooks); status_catalog.store(.searching, .release); status_plex.store(.searching, .release); status_plugins.store(.searching, .release); @@ -480,6 +498,10 @@ pub fn resolveTracked(query: []const u8, intent: []const u8) u32 { if (sourceOn(.music)) Spawn.go(resolveMusic, &status_music); if (sourceOn(.radio)) Spawn.go(resolveRadio, &status_radio); if (sourceOn(.podcast)) Spawn.go(resolvePodcasts, &status_podcast); + if (sourceOn(.novels)) Spawn.go(resolveWikiNovels, &status_novels); + if (sourceOn(.novels)) Spawn.go(resolveArchiveNovels, &status_novel_archive); + if (sourceOn(.vndb)) Spawn.go(resolveVisualNovels, &status_vndb); + if (sourceOn(.audiobooks)) Spawn.go(resolveAudiobooks, &status_audiobooks); // If filtering left nothing to spawn, close the resolve immediately β€” // no worker exists to call checkAllDone(). @@ -945,10 +967,23 @@ fn resolveCatalog(query_buf: [256]u8, qlen: usize) void { const kind_len = @min(kind.len, item.catalog_kind.len); @memcpy(item.catalog_kind[0..kind_len], kind[0..kind_len]); item.catalog_kind_len = kind_len; + // Catalog entries have no playback URL yet, but need distinct transport + // identities: otherwise every empty URL deduplicates to the first card. + const identity = std.fmt.bufPrint(&item.url, "opal://catalog/{s}/{d}", .{ kind, entry.id }) catch ""; + item.url_len = identity.len; const imdb_len = @min(entry.imdb_id_len, item.catalog_imdb.len); @memcpy(item.catalog_imdb[0..imdb_len], entry.imdb_id[0..imdb_len]); item.catalog_imdb_len = imdb_len; + copyField(&item.summary, &item.summary_len, entry.overview[0..@min(entry.overview_len, entry.overview.len)]); + item.rating = entry.rating; + const poster = entry.poster_path[0..@min(entry.poster_path_len, entry.poster_path.len)]; + if (std.mem.startsWith(u8, poster, "https://") or std.mem.startsWith(u8, poster, "http://")) { + if (poster.len <= item.poster_url.len) copyField(&item.poster_url, &item.poster_url_len, poster); + } else if (std.mem.startsWith(u8, poster, "/")) { + const address = std.fmt.bufPrint(&item.poster_url, "https://image.tmdb.org/t/p/w185{s}", .{poster}) catch ""; + item.poster_url_len = address.len; + } const year = entry.year[0..@min(entry.year_len, entry.year.len)]; const label = if (std.mem.eql(u8, kind, "tv")) "TV details" else "Movie details"; const detail = if (year.len > 0) @@ -956,6 +991,15 @@ fn resolveCatalog(query_buf: [256]u8, qlen: usize) void { else label; item.detail_len = detail.len; + if (entry.rating > 0) { + const rating = std.fmt.bufPrint(item.detail[item.detail_len..], " Β· {d:.1}/10", .{entry.rating}) catch ""; + item.detail_len += rating.len; + } + const genres = entry.genre_text[0..@min(entry.genre_text_len, entry.genre_text.len)]; + if (genres.len > 0) { + const suffix = std.fmt.bufPrint(item.detail[item.detail_len..], " Β· {s}", .{genres}) catch ""; + item.detail_len += suffix.len; + } _ = pushResult(item); } } @@ -1153,7 +1197,7 @@ fn attrValue(html: []const u8, name: []const u8, limit: usize) ?[]const u8 { // ══════════════════════════════════════════════════════════ fn cacheKey(buf: []u8, query: []const u8) []const u8 { - return std.fmt.bufPrint(buf, "search:v5:{s}", .{query}) catch "search:v5:"; + return std.fmt.bufPrint(buf, "search:v6:{s}", .{query}) catch "search:v6:"; } /// Serialize the current `results` (under caller's lock) into `out`. @@ -1197,6 +1241,10 @@ fn serializeRows(rows: []const ResolvedItem, out: []u8) ?[]u8 { w.blob(it.plugin_id[0..@min(it.plugin_id_len, it.plugin_id.len)]); w.blob(it.plugin_item_id[0..@min(it.plugin_item_id_len, it.plugin_item_id.len)]); w.u16v(it.plugin_episodes); + w.blob(it.poster_url[0..@min(it.poster_url_len, it.poster_url.len)]); + w.blob(it.summary[0..@min(it.summary_len, it.summary.len)]); + w.f32v(it.rating); + w.blob(it.author[0..@min(it.author_len, it.author.len)]); } return w.done(); } @@ -1250,6 +1298,10 @@ fn deserializeInto(bytes: []const u8) usize { copyField(&it.plugin_id, &it.plugin_id_len, r.blob() orelse break); copyField(&it.plugin_item_id, &it.plugin_item_id_len, r.blob() orelse break); it.plugin_episodes = r.u16v() orelse break; + copyField(&it.poster_url, &it.poster_url_len, r.blob() orelse break); + copyField(&it.summary, &it.summary_len, r.blob() orelse break); + it.rating = r.f32v() orelse break; + copyField(&it.author, &it.author_len, r.blob() orelse break); results[count] = it; count += 1; } @@ -1339,7 +1391,9 @@ fn checkAllDoneLocked(run: u32) void { status_music.load(.acquire) != .searching and status_radio.load(.acquire) != .searching and status_podcast.load(.acquire) != .searching and status_livetv.load(.acquire) != .searching and status_catalog.load(.acquire) != .searching and status_plex.load(.acquire) != .searching and - status_plugins.load(.acquire) != .searching) + status_plugins.load(.acquire) != .searching and + status_novels.load(.acquire) != .searching and status_novel_archive.load(.acquire) != .searching and + status_vndb.load(.acquire) != .searching and status_audiobooks.load(.acquire) != .searching) { // Swap so the resolvingβ†’done transition fires exactly once even if two // finishing workers observe "all done" concurrently β€” only the winner @@ -1416,6 +1470,9 @@ fn computeMatchAgainst(item: ResolvedItem, query_override: []const u8) MatchInfo .music => 22, .podcast => 24, .radio => 26, + .novels => 23, + .audiobooks => 23, + .vndb => 29, .tmdb => 30, // catalog stub β€” not directly playable, rank last }; @@ -1458,94 +1515,443 @@ fn computeMatchAgainst(item: ResolvedItem, query_override: []const u8) MatchInfo // Backend: Jellyfin (local library search) // ══════════════════════════════════════════════════════════ -fn resolveJellyfin(query_buf: [256]u8, qlen: usize) void { - defer finishWorker(&status_jf, .done); - - if (!state.app.jf.connected or state.app.jf.server_url_len == 0) { +// Catalog lookups use caller-owned buffers and never replace the Browse tabs. +fn resolveAudiobooks(query_buf: [256]u8, qlen: usize) void { + defer finishWorker(&status_audiobooks, .done); + const abs = @import("audiobookshelf.zig"); + const pure = @import("audiobookshelf_pure.zig"); + var connection = abs.connectionSnapshot(); + if (!connection.connected or connection.server_len == 0 or connection.token_len == 0) { noteWorkerOutcome(.unavailable); return; } - - const query = query_buf[0..qlen]; - const server = state.app.jf.server_url[0..state.app.jf.server_url_len]; - const uid = state.app.jf.user_id[0..state.app.jf.user_id_len]; - const token = state.app.jf.token[0..state.app.jf.token_len]; - - // URL-encode query - var enc_buf: [512]u8 = undefined; - var enc_len: usize = 0; - for (query) |ch| { - if (enc_len + 3 >= enc_buf.len) break; - if (std.ascii.isAlphanumeric(ch) or ch == '-' or ch == '_' or ch == '.') { - enc_buf[enc_len] = ch; - enc_len += 1; - } else { - enc_buf[enc_len] = '%'; - const hex = "0123456789ABCDEF"; - enc_buf[enc_len + 1] = hex[ch >> 4]; - enc_buf[enc_len + 2] = hex[ch & 0xF]; - enc_len += 3; + const server = std.mem.trimEnd(u8, connection.server[0..connection.server_len], "/"); + // A restored connection may not have loaded its library list yet. Fetch + // permitted libraries independently, without replacing the Browse state. + if (connection.library_count == 0) { + var url_buf: [512]u8 = undefined; + const url = std.fmt.bufPrint(&url_buf, "{s}/api/libraries", .{server}) catch { + noteWorkerOutcome(.failed); + return; + }; + const buf = alloc.alloc(u8, 256 * 1024) catch { + noteWorkerOutcome(.failed); + return; + }; + defer alloc.free(buf); + var code: ?std.http.Status = null; + const headers = [_]std.http.Header{.{ .name = "Authorization", .value = blk: { + break :blk std.fmt.allocPrint(alloc, "Bearer {s}", .{connection.token[0..connection.token_len]}) catch { + noteWorkerOutcome(.failed); + return; + }; + } }}; + defer alloc.free(headers[0].value); + const body = @import("../core/http.zig").fetch(url, buf, .{ .timeout_secs = 8, .extra_headers = &headers, .status_out = &code }) orelse { + noteWorkerOutcome(if (code == .unauthorized or code == .forbidden) .unavailable else .transport_failed); + return; + }; + connection.library_count = pure.parseLibraryPage(alloc, body, &connection.libraries) orelse { + noteWorkerOutcome(.parse_failed); + return; + }; + } + if (connection.library_count == 0) return; + const Page = struct { + rows: [6]ResolvedItem = [_]ResolvedItem{.{}} ** 6, + count: usize = 0, + status: SourceStatus = .no_results, + }; + const pages = alloc.alloc(Page, connection.library_count) catch { + noteWorkerOutcome(.failed); + return; + }; + defer alloc.free(pages); + for (pages) |*page| page.* = .{}; + var threads: [16]?std.Thread = .{null} ** 16; + const Fetch = struct { + fn run(snapshot: *const abs.ConnectionSnapshot, query: *const [256]u8, query_len: usize, library: pure.Library, page: *Page, generation: u32) void { + if (run_gen.load(.acquire) != generation) return; + if (!pure.validItemId(library.id[0..library.id_len])) { + page.status = .parse_failed; + return; + } + var encoded_buf: [768]u8 = undefined; + const encoded = @import("../core/http.zig").urlEncode(query[0..query_len], &encoded_buf); + var url_buf: [1300]u8 = undefined; + const url = std.fmt.bufPrint(&url_buf, "{s}/api/libraries/{s}/search?q={s}&limit=6", .{ + std.mem.trimEnd(u8, snapshot.server[0..snapshot.server_len], "/"), library.id[0..library.id_len], encoded, + }) catch { + page.status = .failed; + return; + }; + var auth_buf: [300]u8 = undefined; + const auth = std.fmt.bufPrint(&auth_buf, "Bearer {s}", .{snapshot.token[0..snapshot.token_len]}) catch { + page.status = .failed; + return; + }; + const headers = [_]std.http.Header{.{ .name = "Authorization", .value = auth }}; + const buf = alloc.alloc(u8, 512 * 1024) catch { + page.status = .failed; + return; + }; + defer alloc.free(buf); + var code: ?std.http.Status = null; + const body = @import("../core/http.zig").fetch(url, buf, .{ .timeout_secs = 8, .extra_headers = &headers, .status_out = &code }) orelse { + page.status = if (code == .unauthorized or code == .forbidden) .unavailable else .transport_failed; + return; + }; + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch { + page.status = .parse_failed; + return; + }; + defer parsed.deinit(); + if (parsed.value != .object) { + page.status = .parse_failed; + return; + } + const key = if (std.mem.eql(u8, library.media_type[0..library.media_type_len], "podcast")) "podcast" else "book"; + const records = parsed.value.object.get(key) orelse { + page.status = .parse_failed; + return; + }; + if (records != .array) { + page.status = .parse_failed; + return; + } + for (records.array.items) |match| { + if (page.count >= page.rows.len) break; + if (match != .object) continue; + const record = match.object.get("libraryItem") orelse continue; + if (record != .object) continue; + const id = catalogString(record, "id"); + if (!pure.validItemId(id)) continue; + const media = record.object.get("media") orelse continue; + if (media != .object) continue; + const metadata = media.object.get("metadata") orelse continue; + const title = catalogString(metadata, "title"); + if (title.len == 0) continue; + var item: ResolvedItem = .{ .source = .audiobooks }; + copyField(&item.name, &item.name_len, title); + copyField(&item.summary, &item.summary_len, catalogString(metadata, "description")); + copyField(&item.author, &item.author_len, catalogString(metadata, "authorName")); + if (item.author_len == 0 and metadata == .object) { + if (metadata.object.get("authors")) |authors| { + if (authors == .array and authors.array.items.len > 0) + copyField(&item.author, &item.author_len, catalogString(authors.array.items[0], "name")); + } + } + const identity: []const u8 = std.fmt.bufPrint(&item.url, "abs://item/{s}", .{id}) catch continue; + item.url_len = identity.len; + var detail_buf: [128]u8 = undefined; + const detail: []const u8 = std.fmt.bufPrint(&detail_buf, "Audiobookshelf Β· {s}", .{library.name[0..library.name_len]}) catch "Audiobookshelf"; + copyField(&item.detail, &item.detail_len, detail); + if (media.object.get("duration")) |duration| item.duration_secs = switch (duration) { + .float => @floatCast(@max(0, duration.float)), + .integer => @floatFromInt(@max(0, duration.integer)), + else => 0, + }; + page.rows[page.count] = item; + page.count += 1; + } + page.status = if (page.count > 0) .done else .no_results; + } + }; + for (pages, 0..) |*page, i| { + threads[i] = @import("../core/workers.zig").spawnLegacy(Fetch.run, .{ &connection, &query_buf, qlen, connection.libraries[i], page, worker_gen }) catch { + page.status = .unavailable; + continue; + }; + } + // Every launched task is joined before its snapshot/result storage dies. + for (threads) |thread| if (thread) |running| running.join(); + var published: usize = 0; + for (pages) |page| { + noteWorkerOutcome(page.status); + for (page.rows[0..page.count]) |item| { + if (published >= 12) break; + if (pushResult(item)) published += 1; } } +} - var url_buf: [1024]u8 = undefined; - const url = std.fmt.bufPrint(&url_buf, "{s}/Users/{s}/Items?searchTerm={s}&Limit=10&Recursive=true&Fields=Overview&api_key={s}", .{ - server, uid, enc_buf[0..enc_len], token, - }) catch { +fn resolveVisualNovels(query_buf: [256]u8, qlen: usize) void { + defer finishWorker(&status_vndb, .done); + const pure = @import("vndb_pure.zig"); + var request_buf: [1024]u8 = undefined; + const request = pure.buildSearchBody(query_buf[0..qlen], 1, &request_buf); + if (request.len == 0) { + noteWorkerOutcome(.failed); + return; + } + const buf = alloc.alloc(u8, 512 * 1024) catch { noteWorkerOutcome(.failed); return; }; - - var buf: [64 * 1024]u8 = undefined; - @import("../core/rate_limit.zig").acquire("jellyfin", 5.0); - const body = @import("../core/http.zig").fetch(url, &buf, .{ .timeout_secs = 5 }) orelse { + defer alloc.free(buf); + @import("../core/rate_limit.zig").acquire("vndb", 1.0); + const body = @import("../core/http.zig").fetch("https://api.vndb.org/kana/vn", buf, .{ + .timeout_secs = 8, + .method = .POST, + .content_type = "application/json", + .payload = request, + }) orelse { noteWorkerOutcome(.transport_failed); return; }; - const n = body.len; - - if (n < 10) { + var schema = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch { + noteWorkerOutcome(.parse_failed); + return; + }; + defer schema.deinit(); + if (schema.value != .object) { + noteWorkerOutcome(.parse_failed); + return; + } + const provider_records = schema.value.object.get("results") orelse { + noteWorkerOutcome(.parse_failed); + return; + }; + if (provider_records != .array) { noteWorkerOutcome(.parse_failed); return; } + const records = alloc.alloc(pure.Vn, 12) catch { + noteWorkerOutcome(.failed); + return; + }; + defer alloc.free(records); + const count = pure.parseVns(body, records); + for (records[0..count]) |record| { + var item: ResolvedItem = .{ .source = .vndb }; + copyField(&item.name, &item.name_len, record.title[0..record.title_len]); + copyField(&item.summary, &item.summary_len, record.description[0..record.description_len]); + copyField(&item.poster_url, &item.poster_url_len, record.image_url[0..record.image_url_len]); + if (record.rating > 0 and record.rating <= 100) item.rating = record.rating / 10; + const url = std.fmt.bufPrint(&item.url, "https://vndb.org/{s}", .{record.id[0..record.id_len]}) catch continue; + item.url_len = url.len; + const detail = std.fmt.bufPrint(&item.detail, "VNDB Β· {s} Β· Visual novel details", .{record.released[0..record.released_len]}) catch "VNDB"; + item.detail_len = detail.len; + _ = pushResult(item); + } +} - // Parse items - var pos: usize = 0; - while (pos < n) { - const id_key = "\"Id\":\""; - const next = std.mem.indexOf(u8, buf[pos..], id_key) orelse break; - const abs = pos + next; +fn catalogString(value: std.json.Value, key: []const u8) []const u8 { + if (value != .object) return ""; + const field = value.object.get(key) orelse return ""; + if (field == .string) return field.string; + if (field == .array and field.array.items.len > 0 and field.array.items[0] == .string) + return field.array.items[0].string; + return ""; +} - // Find object boundaries - const obj_end = findObjEnd(buf[0..n], abs); - const obj = buf[abs..obj_end]; +fn resolveWikiNovels(query_buf: [256]u8, qlen: usize) void { + defer finishWorker(&status_novels, .done); + const pure = @import("novels_pure.zig"); + var url_buf: [1400]u8 = undefined; + const url = pure.buildSearchUrl(&url_buf, query_buf[0..qlen], 12, 0) orelse { + noteWorkerOutcome(.failed); + return; + }; + const buf = alloc.alloc(u8, 256 * 1024) catch { + noteWorkerOutcome(.failed); + return; + }; + defer alloc.free(buf); + @import("../core/rate_limit.zig").acquire("wikisource", 2.0); + const body = @import("../core/http.zig").fetch(url, buf, .{ .timeout_secs = 8 }) orelse { + noteWorkerOutcome(.transport_failed); + return; + }; + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch { + noteWorkerOutcome(.parse_failed); + return; + }; + defer parsed.deinit(); + if (parsed.value != .object) { + noteWorkerOutcome(.parse_failed); + return; + } + const query = parsed.value.object.get("query") orelse { + noteWorkerOutcome(.parse_failed); + return; + }; + if (query != .object) { + noteWorkerOutcome(.parse_failed); + return; + } + const records = query.object.get("search") orelse { + noteWorkerOutcome(.parse_failed); + return; + }; + if (records != .array) { + noteWorkerOutcome(.parse_failed); + return; + } + for (records.array.items) |record| { + const title = catalogString(record, "title"); + if (title.len == 0 or title.len > 256) continue; + var item: ResolvedItem = .{ .source = .novels }; + copyField(&item.name, &item.name_len, title); + item.summary_len = pure.htmlToText(catalogString(record, "snippet"), &item.summary); + item.url_len = pure.formatDeepLink(&item.url, "wikisource", "", title).len; + copyField(&item.detail, &item.detail_len, "Wikisource Β· Read work"); + if (item.url_len > 0) _ = pushResult(item); + } +} - var item = std.mem.zeroes(ResolvedItem); - item.source = .jellyfin; +fn resolveArchiveNovels(query_buf: [256]u8, qlen: usize) void { + defer finishWorker(&status_novel_archive, .done); + const pure = @import("novels_pure.zig"); + var raw_buf: [640]u8 = undefined; + const query = std.fmt.bufPrint(&raw_buf, "title:({s}) AND mediatype:(texts) AND language:(eng) AND format:(DjVuTXT)", .{query_buf[0..qlen]}) catch { + noteWorkerOutcome(.failed); + return; + }; + var enc_buf: [2000]u8 = undefined; + const encoded = @import("../core/http.zig").urlEncode(query, &enc_buf); + var url_buf: [2400]u8 = undefined; + const url = std.fmt.bufPrint(&url_buf, "https://archive.org/advancedsearch.php?q={s}&fl[]=identifier&fl[]=title&fl[]=creator&fl[]=description&rows=12&page=1&output=json", .{encoded}) catch { + noteWorkerOutcome(.failed); + return; + }; + const buf = alloc.alloc(u8, 512 * 1024) catch { + noteWorkerOutcome(.failed); + return; + }; + defer alloc.free(buf); + @import("../core/rate_limit.zig").acquire("archive", 1.0); + const body = @import("../core/http.zig").fetch(url, buf, .{ .timeout_secs = 8 }) orelse { + noteWorkerOutcome(.transport_failed); + return; + }; + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch { + noteWorkerOutcome(.parse_failed); + return; + }; + defer parsed.deinit(); + if (parsed.value != .object) { + noteWorkerOutcome(.parse_failed); + return; + } + const response = parsed.value.object.get("response") orelse { + noteWorkerOutcome(.parse_failed); + return; + }; + if (response != .object) { + noteWorkerOutcome(.parse_failed); + return; + } + const records = response.object.get("docs") orelse { + noteWorkerOutcome(.parse_failed); + return; + }; + if (records != .array) { + noteWorkerOutcome(.parse_failed); + return; + } + for (records.array.items) |record| { + const id = catalogString(record, "identifier"); + const title = catalogString(record, "title"); + if (id.len == 0 or id.len > 512 or title.len == 0 or title.len > 256) continue; + var item: ResolvedItem = .{ .source = .novels }; + copyField(&item.name, &item.name_len, title); + item.summary_len = pure.htmlToText(catalogString(record, "description"), &item.summary); + item.url_len = pure.formatDeepLink(&item.url, "internet_archive", id, title).len; + var id_buf: [1536]u8 = undefined; + const encoded_id = @import("../core/http.zig").urlEncode(id, &id_buf); + const cover = std.fmt.bufPrint(&item.poster_url, "https://archive.org/services/img/{s}", .{encoded_id}) catch ""; + item.poster_url_len = cover.len; + const author = catalogString(record, "creator"); + var detail_buf: [128]u8 = undefined; + const detail = if (author.len > 0) std.fmt.bufPrint(&detail_buf, "Internet Archive Β· {s}", .{author}) catch "Internet Archive" else "Internet Archive Β· Read work"; + copyField(&item.detail, &item.detail_len, detail); + if (item.url_len > 0) _ = pushResult(item); + } +} - if (extractStr(obj, "\"Id\":\"")) |id| { - const ilen = @min(id.len, 63); - @memcpy(item.jf_item_id[0..ilen], id[0..ilen]); - item.jf_item_id_len = ilen; - @memcpy(item.url[0..ilen], id[0..ilen]); - item.url_len = ilen; +fn resolveJellyfin(query_buf: [256]u8, qlen: usize) void { + defer finishWorker(&status_jf, .done); + const connection = @import("jellyfin.zig").connectionSnapshot(); + if (!connection.connected or connection.server_len == 0 or connection.user_id_len == 0) { + noteWorkerOutcome(.unavailable); + return; + } + var enc_buf: [768]u8 = undefined; + const encoded = @import("../core/http.zig").urlEncode(query_buf[0..qlen], &enc_buf); + var url_buf: [1200]u8 = undefined; + const url = std.fmt.bufPrint(&url_buf, "{s}/Users/{s}/Items?searchTerm={s}&Limit=10&Recursive=true&IncludeItemTypes=Movie,Series,Episode,Audio&Fields=Overview,UserData", .{ + std.mem.trimEnd(u8, connection.server[0..connection.server_len], "/"), + connection.user_id[0..connection.user_id_len], + encoded, + }) catch { + noteWorkerOutcome(.failed); + return; + }; + const headers = [_]std.http.Header{.{ .name = "X-Emby-Token", .value = connection.token[0..connection.token_len] }}; + const buf = alloc.alloc(u8, 256 * 1024) catch { + noteWorkerOutcome(.failed); + return; + }; + defer alloc.free(buf); + @import("../core/rate_limit.zig").acquire("jellyfin", 5.0); + const body = @import("../core/http.zig").fetch(url, buf, .{ .timeout_secs = 5, .extra_headers = &headers }) orelse { + noteWorkerOutcome(.transport_failed); + return; + }; + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch { + noteWorkerOutcome(.parse_failed); + return; + }; + defer parsed.deinit(); + if (parsed.value != .object) { + noteWorkerOutcome(.parse_failed); + return; + } + const entries = parsed.value.object.get("Items") orelse { + noteWorkerOutcome(.parse_failed); + return; + }; + if (entries != .array) { + noteWorkerOutcome(.parse_failed); + return; + } + const Field = struct { + fn string(value: std.json.Value, key: []const u8) []const u8 { + if (value != .object) return ""; + const field = value.object.get(key) orelse return ""; + return if (field == .string) field.string else ""; } - if (extractStr(obj, "\"Name\":\"")) |name| { - const nlen = @min(name.len, 255); - @memcpy(item.name[0..nlen], name[0..nlen]); - item.name_len = nlen; + fn number(value: std.json.Value, key: []const u8) f32 { + if (value != .object) return 0; + const field = value.object.get(key) orelse return 0; + return switch (field) { + .float => @floatCast(field.float), + .integer => @floatFromInt(field.integer), + else => 0, + }; } - if (extractStr(obj, "\"Type\":\"")) |mt| { - const dstr = std.fmt.bufPrint(&item.detail, "Jellyfin Β· {s}", .{mt}) catch ""; - item.detail_len = dstr.len; - } else { - const dstr = "Jellyfin Β· Local"; - @memcpy(item.detail[0..dstr.len], dstr); - item.detail_len = dstr.len; + }; + for (entries.array.items) |entry| { + const id = Field.string(entry, "Id"); + const name = Field.string(entry, "Name"); + const kind = Field.string(entry, "Type"); + if (id.len == 0 or name.len == 0) continue; + var item: ResolvedItem = .{ .source = .jellyfin }; + copyField(&item.jf_item_id, &item.jf_item_id_len, id); + copyField(&item.url, &item.url_len, id); + copyField(&item.name, &item.name_len, name); + copyField(&item.summary, &item.summary_len, Field.string(entry, "Overview")); + const rating = Field.number(entry, "CommunityRating"); + if (rating > 0 and rating <= 10) item.rating = rating; + item.duration_secs = Field.number(entry, "RunTimeTicks") / 10_000_000; + if (entry == .object) { + if (entry.object.get("UserData")) |user_data| + item.resume_position_secs = Field.number(user_data, "PlaybackPositionTicks") / 10_000_000; } - - if (item.name_len > 0) _ = pushResult(item); - pos = obj_end; + const detail = std.fmt.bufPrint(&item.detail, "Jellyfin Β· {s}", .{kind}) catch "Jellyfin"; + item.detail_len = detail.len; + _ = pushResult(item); } } @@ -1558,9 +1964,10 @@ fn resolvePlex(query_buf: [256]u8, qlen: usize) void { } var found: [20]plex.SearchItem = [_]plex.SearchItem{.{}} ** 20; - const count = plex.searchInto(query_buf[0..qlen], &found); + const response = plex.searchIntoResult(query_buf[0..qlen], &found); + const count = response.count; if (count == 0) { - noteWorkerOutcome(.no_results); + noteWorkerOutcome(response.status); return; } for (found[0..count]) |entry| { @@ -1578,8 +1985,12 @@ fn resolvePlex(query_buf: [256]u8, qlen: usize) void { fn resolveInstalledPlugins(query_buf: [256]u8, qlen: usize) void { defer finishWorker(&status_plugins, .done); const plugins = @import("plugins.zig"); - var found: [24]plugins.UniversalResult = [_]plugins.UniversalResult{.{}} ** 24; - const count = plugins.searchInstalledInto(query_buf[0..qlen], &found); + const found = alloc.alloc(plugins.UniversalResult, 24) catch { + noteWorkerOutcome(.failed); + return; + }; + defer alloc.free(found); + const count = plugins.searchInstalledInto(query_buf[0..qlen], found); if (count == 0) { noteWorkerOutcome(.no_results); return; @@ -1587,6 +1998,9 @@ fn resolveInstalledPlugins(query_buf: [256]u8, qlen: usize) void { for (found[0..count]) |entry| { var item = ResolvedItem{ .source = .plugin, .plugin_episodes = entry.row.episodes }; copyField(&item.name, &item.name_len, entry.row.title[0..entry.row.title_len]); + copyField(&item.poster_url, &item.poster_url_len, entry.row.poster_url[0..entry.row.poster_url_len]); + copyField(&item.summary, &item.summary_len, entry.row.overview[0..entry.row.overview_len]); + // Plugin scores are adapter-defined; do not pretend they share /10 units. const media_type = entry.row.media_type[0..entry.row.media_type_len]; const year = entry.row.year[0..entry.row.year_len]; const detail = if (year.len > 0) @@ -2748,7 +3162,7 @@ fn resolveYouTube(query_buf: [256]u8, qlen: usize) void { const query = query_buf[0..qlen]; var search_arg: [300]u8 = undefined; - const sa = std.fmt.bufPrint(&search_arg, "ytsearch5:{s}", .{query}) catch { + const sa = std.fmt.bufPrint(&search_arg, "ytsearch10:{s}", .{query}) catch { noteWorkerOutcome(.failed); return; }; @@ -2757,8 +3171,12 @@ fn resolveYouTube(query_buf: [256]u8, qlen: usize) void { const argv_policy = @import("ytdlp_argv_pure.zig"); var argv_storage: argv_policy.Argv = undefined; const argv = argv_policy.build(ytdlp_bin, sa, .search_json, "", &argv_storage); - var buf: [64 * 1024]u8 = undefined; - const execution = @import("../core/bounded_process.zig").run(argv, &buf, .{ .timeout_ms = 20_000 }); + const buf = alloc.alloc(u8, 256 * 1024) catch { + noteWorkerOutcome(.failed); + return; + }; + defer alloc.free(buf); + const execution = @import("../core/bounded_process.zig").run(argv, buf, .{ .timeout_ms = 20_000 }); if (!execution.ok()) { noteWorkerOutcome(switch (execution.failure) { .spawn => .unavailable, @@ -2775,40 +3193,61 @@ fn resolveYouTube(query_buf: [256]u8, qlen: usize) void { return; } - // Each line is a JSON object with "title", "url", "id" + // Structural parsing accepts compact JSON and reordered yt-dlp fields. var lines = std.mem.splitScalar(u8, execution.output, '\n'); var found: usize = 0; while (lines.next()) |line| { - if (found >= 5 or line.len < 10) continue; - - var item = std.mem.zeroes(ResolvedItem); - item.source = .youtube; - - if (extractStr(line, "\"title\": \"")) |title| { - // yt-dlp titles carry JSON escapes (’ etc.) β€” decode them - // or the UI shows "You’ve" literally. - var unesc_buf: [256]u8 = undefined; - const clean = @import("json_pure.zig").jsonUnescape(title, &unesc_buf); - const tlen = @min(clean.len, 255); - @memcpy(item.name[0..tlen], clean[0..tlen]); - item.name_len = tlen; - } - if (extractStr(line, "\"url\": \"")) |url| { - const ulen = @min(url.len, 2047); - @memcpy(item.url[0..ulen], url[0..ulen]); - item.url_len = ulen; - } else if (extractStr(line, "\"id\": \"")) |vid_id| { - var yt_url: [128]u8 = undefined; - const yt = std.fmt.bufPrint(&yt_url, "https://www.youtube.com/watch?v={s}", .{vid_id}) catch ""; - const ulen = @min(yt.len, 2047); - @memcpy(item.url[0..ulen], yt[0..ulen]); - item.url_len = ulen; + if (found >= 10) break; + if (std.mem.trim(u8, line, " \r\t").len == 0) continue; + var parsed = std.json.parseFromSlice(std.json.Value, alloc, line, .{}) catch { + noteWorkerOutcome(.parse_failed); + continue; + }; + defer parsed.deinit(); + const entry = parsed.value; + if (entry != .object) continue; + var item: ResolvedItem = .{ .source = .youtube }; + copyField(&item.name, &item.name_len, catalogString(entry, "title")); + copyField(&item.summary, &item.summary_len, catalogString(entry, "description")); + const id = catalogString(entry, "id"); + // A flat extractor may return an ID as `url`; canonical watch URLs + // remain stable, while expiring media URLs do not belong in search. + if (id.len > 0 and id.len <= 64) { + const address: []const u8 = std.fmt.bufPrint(&item.url, "https://www.youtube.com/watch?v={s}", .{id}) catch ""; + item.url_len = address.len; + } else { + const address = catalogString(entry, "webpage_url"); + if (address.len <= item.url.len) copyField(&item.url, &item.url_len, address); } - + const thumbnail = catalogString(entry, "thumbnail"); + if (thumbnail.len > 0 and thumbnail.len <= item.poster_url.len) { + copyField(&item.poster_url, &item.poster_url_len, thumbnail); + } else if (entry.object.get("thumbnails")) |images| { + if (images == .array) { + for (images.array.items) |image| { + const address = catalogString(image, "url"); + if (address.len > 0 and address.len <= item.poster_url.len) + copyField(&item.poster_url, &item.poster_url_len, address); + } + } + } + if (entry.object.get("duration")) |duration| { + item.duration_secs = switch (duration) { + .float => @floatCast(@max(0, duration.float)), + .integer => @floatFromInt(@max(0, duration.integer)), + else => 0, + }; + } + const channel = catalogString(entry, "channel"); + const uploader = catalogString(entry, "uploader"); + const author = if (channel.len > 0) channel else uploader; + var detail_buf: [128]u8 = undefined; + const detail = if (author.len > 0) + std.fmt.bufPrint(&detail_buf, "YouTube Β· {s}", .{author}) catch "YouTube" + else + "YouTube"; + copyField(&item.detail, &item.detail_len, detail); if (item.name_len > 0 and item.url_len > 0) { - const detail = "YouTube"; - @memcpy(item.detail[0..detail.len], detail); - item.detail_len = detail.len; _ = pushResult(item); found += 1; } @@ -2898,23 +3337,28 @@ fn resolveMusic(query_buf: [256]u8, qlen: usize) void { return; }; - var it = jp.SongIter{ .json = body }; + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch { + noteWorkerOutcome(.parse_failed); + return; + }; + defer parsed.deinit(); + const rows = @import("music_subsonic_pure.zig").pageRows(parsed.value, .jiosaavn) orelse { + noteWorkerOutcome(.parse_failed); + return; + }; var found: usize = 0; - while (found < AUDIO_MAX) { - const obj = it.next() orelse break; - var t: [200]u8 = undefined; - var a: [160]u8 = undefined; - var u: [512]u8 = undefined; - var im: [300]u8 = undefined; - const song = jp.parseSong(obj, &t, &a, &u, &im) orelse continue; + for (rows) |obj| { + if (found >= AUDIO_MAX) break; + const song = jp.parseSongValue(obj) orelse continue; var item = std.mem.zeroes(ResolvedItem); item.source = .music; - copyField(&item.name, &item.name_len, song.title); - copyField(&item.url, &item.url_len, song.perma_url); + copyField(&item.name, &item.name_len, song.title[0..song.title_len]); + copyField(&item.url, &item.url_len, song.play_url[0..song.play_url_len]); + copyField(&item.poster_url, &item.poster_url_len, song.cover[0..song.cover_len]); var d: [128]u8 = undefined; - const detail = if (song.artist.len > 0) - std.fmt.bufPrint(&d, "Music - {s}", .{song.artist}) catch "Music" + const detail = if (song.artist_len > 0) + std.fmt.bufPrint(&d, "Music - {s}", .{song.artist[0..song.artist_len]}) catch "Music" else "Music"; copyField(&item.detail, &item.detail_len, detail); @@ -2949,7 +3393,11 @@ fn resolveRadio(query_buf: [256]u8, qlen: usize) void { // Station is ~1.3KB; AUDIO_MAX of them is well under the stack budget. var stations: [AUDIO_MAX]rp.Station = undefined; - const n = rp.parseStations(body, &stations); + const page = rp.parsePage(alloc, body, &stations) orelse { + noteWorkerOutcome(.parse_failed); + return; + }; + const n = page.count; var i: usize = 0; while (i < n) : (i += 1) { const s = stations[i]; @@ -2965,6 +3413,8 @@ fn resolveRadio(query_buf: [256]u8, qlen: usize) void { item.source = .radio; copyField(&item.name, &item.name_len, s.name[0..s.name_len]); copyField(&item.url, &item.url_len, stream); + copyField(&item.poster_url, &item.poster_url_len, s.favicon[0..s.favicon_len]); + copyField(&item.summary, &item.summary_len, s.tags[0..s.tags_len]); var d: [128]u8 = undefined; const detail = if (s.country_len > 0) std.fmt.bufPrint(&d, "Radio - {s}", .{s.country[0..s.country_len]}) catch "Radio" @@ -3003,12 +3453,32 @@ fn resolvePodcasts(query_buf: [256]u8, qlen: usize) void { return; }; + var parsed = std.json.parseFromSlice(std.json.Value, alloc, body, .{}) catch { + noteWorkerOutcome(.parse_failed); + return; + }; + defer parsed.deinit(); + if (parsed.value != .object) { + noteWorkerOutcome(.parse_failed); + return; + } + const entries = parsed.value.object.get("results") orelse { + noteWorkerOutcome(.parse_failed); + return; + }; + if (entries != .array) { + noteWorkerOutcome(.parse_failed); + return; + } const shows = alloc.alloc(pp.Podcast, AUDIO_MAX) catch { noteWorkerOutcome(.failed); return; }; defer alloc.free(shows); - const n = pp.parseItunes(body, shows); + const n = pp.parseItunesValue(parsed.value, shows) orelse { + noteWorkerOutcome(.parse_failed); + return; + }; var i: usize = 0; while (i < n) : (i += 1) { const s = shows[i]; @@ -3016,6 +3486,7 @@ fn resolvePodcasts(query_buf: [256]u8, qlen: usize) void { var item = std.mem.zeroes(ResolvedItem); item.source = .podcast; copyField(&item.name, &item.name_len, s.name[0..s.name_len]); + copyField(&item.poster_url, &item.poster_url_len, s.artwork[0..s.artwork_len]); // The feed URL is the identity β€” playItem opens the Podcasts tab on it. copyField(&item.url, &item.url_len, s.feed_url[0..s.feed_url_len]); var d: [128]u8 = undefined; @@ -3475,6 +3946,19 @@ pub fn playResolvedItem(item: *const ResolvedItem) void { @import("search.zig").submitQuery(item.name[0..item.name_len]); } }, + .novels => @import("novels.zig").openDeepLink(item.url[0..item.url_len]), + .audiobooks => { + const prefix = "abs://item/"; + const url = item.url[0..item.url_len]; + if (std.mem.startsWith(u8, url, prefix)) + @import("audiobookshelf.zig").playBookById(url[prefix.len..], item.name[0..item.name_len], item.author[0..item.author_len]); + }, + .vndb => { + const prefix = "https://vndb.org/"; + const url = item.url[0..item.url_len]; + if (std.mem.startsWith(u8, url, prefix)) + @import("vndb.zig").openCatalogIdentity(url[prefix.len..]); + }, .podcast => { // No open-by-feed entry point exists yet, so hand the show title to // the Podcasts tab's own search and reveal it β€” the user lands on diff --git a/src/services/resolver_dedup_pure.zig b/src/services/resolver_dedup_pure.zig index bdf5c2cf..0b77cb79 100644 --- a/src/services/resolver_dedup_pure.zig +++ b/src/services/resolver_dedup_pure.zig @@ -7,8 +7,9 @@ const std = @import("std"); /// (the same release from 5 trackers differs only in the appended `&tr=` list), /// so we key on `btih:`; otherwise the full URL. Returns a slice into `url`. pub fn dedupKey(url: []const u8) []const u8 { - if (std.mem.indexOf(u8, url, "btih:")) |at| { - const start = at + "btih:".len; + if (!std.mem.startsWith(u8, url, "magnet:?")) return url; + if (std.mem.indexOf(u8, url, "xt=urn:btih:")) |at| { + const start = at + "xt=urn:btih:".len; var end = start; while (end < url.len and url[end] != '&' and url[end] != '.' and url[end] != '/') end += 1; if (end > start) return url[start..end]; @@ -16,11 +17,15 @@ pub fn dedupKey(url: []const u8) []const u8 { return url; } -/// Two result URLs point at the same item (case-insensitive key compare). +/// Magnet hashes ignore case; URL paths and query values preserve case. pub fn sameItem(a: []const u8, b: []const u8) bool { + if (a.len == 0 or b.len == 0) return false; const ka = dedupKey(a); const kb = dedupKey(b); if (ka.len != kb.len) return false; + // Whole URLs include case-sensitive paths, signed tokens, and provider IDs. + // Only an extracted magnet infohash permits case folding. + if (ka.len == a.len or kb.len == b.len) return std.mem.eql(u8, a, b); for (ka, kb) |x, y| { if (std.ascii.toLower(x) != std.ascii.toLower(y)) return false; } diff --git a/src/services/search.zig b/src/services/search.zig index 4c868345..6d0d38f5 100644 --- a/src/services/search.zig +++ b/src/services/search.zig @@ -1890,6 +1890,9 @@ fn renderSourceStatusCluster() void { .{ .icon = icons.tvg.lucide.music, .name = "Music", .bit = .music, .st = resolver.status_music.load(.acquire) }, .{ .icon = icons.tvg.lucide.radio, .name = "Radio", .bit = .radio, .st = resolver.status_radio.load(.acquire) }, .{ .icon = icons.tvg.lucide.podcast, .name = "Podcasts", .bit = .podcast, .st = resolver.status_podcast.load(.acquire) }, + .{ .icon = icons.tvg.lucide.book, .name = "Novels", .bit = .novels, .st = resolver.combineSourceStatuses(resolver.status_novels.load(.acquire), resolver.status_novel_archive.load(.acquire)) }, + .{ .icon = icons.tvg.lucide.book, .name = "Visual novels", .bit = .vndb, .st = resolver.status_vndb.load(.acquire) }, + .{ .icon = icons.tvg.lucide.headphones, .name = "Audiobooks", .bit = .audiobooks, .st = resolver.status_audiobooks.load(.acquire) }, }; for (rows, 0..) |r, i| { const enabled = resolver.sourceOn(r.bit); @@ -2165,6 +2168,9 @@ fn sourceBitOf(item: *const @import("resolver.zig").ResolvedItem) ?@import("reso .music => .music, .radio => .radio, .podcast => .podcast, + .novels => .novels, + .vndb => .vndb, + .audiobooks => .audiobooks, }; } @@ -2195,6 +2201,9 @@ fn renderSourceSummary(source_has: std.EnumSet(@import("resolver.zig").SourceBit .{ .name = "Music", .src = .music, .rss = false, .st = resolver.status_music.load(.acquire), .bit = .music }, .{ .name = "Radio", .src = .radio, .rss = false, .st = resolver.status_radio.load(.acquire), .bit = .radio }, .{ .name = "Podcasts", .src = .podcast, .rss = false, .st = resolver.status_podcast.load(.acquire), .bit = .podcast }, + .{ .name = "Novels", .src = .novels, .rss = false, .st = resolver.combineSourceStatuses(resolver.status_novels.load(.acquire), resolver.status_novel_archive.load(.acquire)), .bit = .novels }, + .{ .name = "Visual novels", .src = .vndb, .rss = false, .st = resolver.status_vndb.load(.acquire), .bit = .vndb }, + .{ .name = "Audiobooks", .src = .audiobooks, .rss = false, .st = resolver.status_audiobooks.load(.acquire), .bit = .audiobooks }, }; const appendName = struct { @@ -2331,6 +2340,8 @@ fn renderCompactRow(idx: usize, item: *const @import("resolver.zig").ResolvedIte .music => dvui.Color{ .r = 255, .g = 170, .b = 90, .a = 255 }, .radio => dvui.Color{ .r = 120, .g = 200, .b = 255, .a = 255 }, .podcast => dvui.Color{ .r = 255, .g = 140, .b = 170, .a = 255 }, + .novels, .vndb => theme.colors.accent, + .audiobooks => theme.colors.accent, }; const chip_text = switch (item.source) { .jellyfin => "Jellyfin", @@ -2347,6 +2358,9 @@ fn renderCompactRow(idx: usize, item: *const @import("resolver.zig").ResolvedIte .music => "Music", .radio => "Radio", .podcast => "Podcast", + .novels => "Novel", + .vndb => "Visual novel", + .audiobooks => "Audiobook", }; var row = dvui.box(@src(), .{ .dir = .horizontal }, .{ @@ -2430,6 +2444,22 @@ fn renderCompactRow(idx: usize, item: *const @import("resolver.zig").ResolvedIte if (meta.len > 0) metadata.addText(meta, .{ .color_text = theme.colors.text_secondary }); metadata.deinit(); } + if (item.summary_len > 0) { + // Keep search scannable; the complete provider synopsis remains in the + // result API and catalog details. Never substitute generated copy. + const n = @min(item.summary_len, 180); + var synopsis = dvui.textLayout(@src(), .{ .break_lines = true }, .{ + .id_extra = idx + 9450, + .expand = .horizontal, + .min_size_content = .{ .w = 0, .h = 0 }, + .max_size_content = dvui.Options.MaxSize.width(text_w), + .background = false, + .padding = dvui.Rect.all(0), + }); + synopsis.addText(safeUtf8(item.summary[0..n]), .{ .color_text = theme.colors.text_secondary }); + if (n < item.summary_len) synopsis.addText("...", .{ .color_text = theme.colors.text_secondary }); + synopsis.deinit(); + } content.deinit(); var actions = dvui.box(@src(), .{ .dir = .horizontal }, .{ .gravity_y = 0.5 }); @@ -2465,7 +2495,9 @@ fn renderCompactRow(idx: usize, item: *const @import("resolver.zig").ResolvedIte // Explicit Play affordance β€” dimmed on blocked rows; the click still // routes to playItem, whose central risk guard toasts the reason. - if (dvui.buttonIcon(@src(), "play", icons.tvg.lucide.play, .{}, .{}, .{ + const action_icon = if (item.source == .novels or item.source == .comics) icons.tvg.lucide.book else if (item.source == .vndb or item.source == .tmdb or item.source == .audiobooks) icons.tvg.lucide.info else icons.tvg.lucide.play; + const action_name = if (item.source == .novels or item.source == .comics) "read" else if (item.source == .audiobooks) "open audio" else if (item.source == .vndb or item.source == .tmdb) "details" else "play"; + if (dvui.buttonIcon(@src(), action_name, action_icon, .{}, .{}, .{ .id_extra = idx + 9700, .color_fill = dvui.Color{ .r = 0, .g = 0, .b = 0, .a = 0 }, .color_text = if (risk.risk == .block) theme.colors.text_tertiary else theme.colors.accent, diff --git a/src/services/tmdb.zig b/src/services/tmdb.zig index 528a5c29..0995e3f2 100644 --- a/src/services/tmdb.zig +++ b/src/services/tmdb.zig @@ -351,7 +351,7 @@ pub fn renderTmdbContent() void { api.fetchCurrentView(false); } else if (state.app.tmdb.view == .Trending and !state.app.tmdb.is_loading.load(.acquire) and state.app.tmdb.results.items.len > 0 and state.app.tmdb.page == 1 and - @import("browse_cache.zig").isStale(state.app.tmdb.last_fetch_s)) + api.shouldRefreshBrowse()) { // Cache aged past the TTL β€” refresh in the background (the current // results keep showing until the new ones arrive). @@ -374,6 +374,15 @@ pub fn renderTmdbContent() void { }); renderToolbar(list.items.len); toolbar_scroll.deinit(); + if (state.app.tmdb.view == .Trending and list.items.len > 0 and api.refreshFailed() and !state.app.tmdb.is_loading.load(.acquire)) { + var retry_row = dvui.box(@src(), .{ .dir = .horizontal }, .{ .expand = .horizontal }); + _ = dvui.label(@src(), "Refresh failed. Showing previously loaded titles.", .{}, .{ .color_text = theme.colors.warning }); + if (@import("../ui/components.zig").actionButton(@src(), "Retry", .secondary, 88001)) { + state.app.tmdb.page = 1; + api.fetchCurrentView(false); + } + retry_row.deinit(); + } // Initial load (nothing to show yet) renders skeleton tiles inside the // gallery; a stale-refresh keeps the current results on screen β€” seamless. const show_load_more = state.app.tmdb.view == .Trending or state.app.tmdb.view == .Search; diff --git a/src/services/tmdb_api.zig b/src/services/tmdb_api.zig index 4ead26d3..61b5b786 100644 --- a/src/services/tmdb_api.zig +++ b/src/services/tmdb_api.zig @@ -23,6 +23,19 @@ const TMDB_MAX_CACHED_ITEMS: usize = 100; // UI-thread-only: coalesce changes while one worker owns the request snapshot. var refresh_queued: std.atomic.Value(bool) = .init(false); +var last_refresh_attempt_s = std.atomic.Value(i64).init(0); // Separate from successful freshness. +var refresh_failed = std.atomic.Value(bool).init(false); + +pub fn refreshFailed() bool { + return refresh_failed.load(.acquire); +} + +pub fn shouldRefreshBrowse() bool { + const cache = @import("browse_cache.zig"); + const attempt = last_refresh_attempt_s.load(.acquire); + return cache.isStale(state.app.tmdb.last_fetch_s) and + (attempt == 0 or cache.now() - attempt >= 30); +} pub fn fetchCurrentView(append: bool) void { if (state.app.tmdb.is_loading.load(.acquire)) { @@ -35,10 +48,10 @@ pub fn fetchCurrentView(append: bool) void { state.app.tmdb.pending_ready = false; state.app.tmdb.results_mutex.unlock(); refresh_queued.store(false, .release); - - // SWR: stamp the cache time on a fresh (non-append) load so revisits within - // the TTL skip the network (see browse_cache + renderTmdbContent). - if (!append) state.app.tmdb.last_fetch_s = @import("browse_cache.zig").now(); + if (!append) { + last_refresh_attempt_s.store(@import("browse_cache.zig").now(), .release); + refresh_failed.store(false, .release); + } // Reserve once to keep scrolling/appending cheap. Poster jobs publish by // identity on the UI thread, so reallocating no longer invalidates workers. @@ -164,9 +177,14 @@ pub fn searchCatalogInto(query: []const u8, out: []state.TmdbItem) usize { var url_buf: [512]u8 = undefined; const url = buildApiUrl(&url_buf, .search, query, .trending, .all, .week, 0, 0, 1) orelse return 0; rate_limit.acquire("tmdb", 3.0); - const body = httpGet(url, key) orelse return 0; - defer alloc.free(body); - parse.parseTmdbResponse(body, &staged); + if (httpGet(url, key)) |body| { + defer alloc.free(body); + parse.parseTmdbResponse(body, &staged); + if (parse.extractJsonInt(body, "\"status_code\":") != 0) + fetchCinemetaInto(&staged, .search, query, .trending, .all, 0, 1); + } else { + fetchCinemetaInto(&staged, .search, query, .trending, .all, 0, 1); + } } else { fetchCinemetaInto(&staged, .search, query, .trending, .all, 0, 1); } @@ -221,6 +239,7 @@ fn fetchTmdb(mode: FetchMode, query: []const u8, append: bool) void { state.app.tmdb.thread = @import("../core/workers.zig").spawnLegacy(struct { fn worker() void { defer { + if (!S.do_append) last_refresh_attempt_s.store(@import("browse_cache.zig").now(), .release); state.app.tmdb.is_loading.store(false, .release); state.wakeUi(); } @@ -232,6 +251,7 @@ fn fetchTmdb(mode: FetchMode, query: []const u8, append: bool) void { // the renderCatalogRail out-of-bounds crash). The UI thread swaps // staged pages in at frame start via applyPendingResults(). var staged: std.ArrayListUnmanaged(state.TmdbItem) = .empty; + defer staged.deinit(alloc); var total_pages: u32 = 100; var attempt: u8 = 0; while (true) : (attempt += 1) { @@ -243,10 +263,19 @@ fn fetchTmdb(mode: FetchMode, query: []const u8, append: bool) void { if (httpGet(url, key)) |body| { total_pages = @intCast(@max(1, parse.extractJsonInt(body, "\"total_pages\":"))); parse.parseTmdbResponse(body, &staged); + const api_failed = parse.extractJsonInt(body, "\"status_code\":") != 0; alloc.free(body); + if (api_failed) { + fetchCinemetaInto(&staged, S.fetch_mode, S.q[0..S.q_len], S.category, S.media_filter, S.genre_idx, S.page); + total_pages = if (staged.items.len == 0) S.page else S.page + 1; + } + } else { + fetchCinemetaInto(&staged, S.fetch_mode, S.q[0..S.q_len], S.category, S.media_filter, S.genre_idx, S.page); + total_pages = if (staged.items.len == 0) S.page else S.page + 1; } } else { fetchCinemetaInto(&staged, S.fetch_mode, S.q[0..S.q_len], S.category, S.media_filter, S.genre_idx, S.page); + total_pages = if (staged.items.len == 0) S.page else S.page + 1; } const retry = @import("tmdb_pure.zig").shouldRetryEmptyCatalog( @@ -266,6 +295,7 @@ fn fetchTmdb(mode: FetchMode, query: []const u8, append: bool) void { // loaded catalog. With no visible cards, fall through so the normal // empty state and manual Retry remain available after exhaustion. if (S.fetch_mode == .browse and !S.do_append and staged.items.len == 0) { + refresh_failed.store(true, .release); state.app.tmdb.results_mutex.lock(); const have_visible = state.app.tmdb.results.items.len > 0; state.app.tmdb.results_mutex.unlock(); @@ -282,11 +312,13 @@ fn fetchTmdb(mode: FetchMode, query: []const u8, append: bool) void { defer state.app.tmdb.results_mutex.unlock(); state.app.tmdb.pending_results.deinit(alloc); state.app.tmdb.pending_results = staged; + staged = .empty; // Ownership moved to the UI publication. state.app.tmdb.pending_append = S.do_append; state.app.tmdb.pending_total_pages = total_pages; state.app.tmdb.pending_ready = true; } }.worker, .{}) catch blk: { + refresh_failed.store(true, .release); state.app.tmdb.is_loading.store(false, .release); break :blk null; }; @@ -372,11 +404,21 @@ pub fn applyPendingResults() void { if (!t.pending_ready) return; t.pending_ready = false; if (!t.pending_append) { + if (t.pending_results.items.len > 0) t.last_fetch_s = @import("browse_cache.zig").now(); for (t.results.items) |*item| @import("../core/poster.zig").deinitPoster(&item.poster_pixels, &item.poster_tex); t.results.clearRetainingCapacity(); } // Poster publication does not retain pointers into this growable list. - t.results.appendSlice(alloc, t.pending_results.items) catch {}; + for (t.pending_results.items) |item| { + var duplicate = false; + for (t.results.items) |existing| { + if (existing.id == item.id and std.mem.eql(u8, existing.media_type[0..existing.media_type_len], item.media_type[0..item.media_type_len])) { + duplicate = true; + break; + } + } + if (!duplicate) t.results.append(alloc, item) catch break; + } t.pending_results.clearRetainingCapacity(); t.total_pages = t.pending_total_pages; state.wakeUi(); diff --git a/src/services/tmdb_parse.zig b/src/services/tmdb_parse.zig index 584f1fca..047f4eb3 100644 --- a/src/services/tmdb_parse.zig +++ b/src/services/tmdb_parse.zig @@ -57,11 +57,10 @@ pub fn extractJsonInt(json: []const u8, key: []const u8) i32 { const after = json[ki..]; var i: usize = 0; while (i < after.len and (after[i] == ' ')) i += 1; - var result: i32 = 0; - while (i < after.len and after[i] >= '0' and after[i] <= '9') : (i += 1) { - result = result * 10 + @as(i32, @intCast(after[i] - '0')); - } - return result; + const start = i; + while (i < after.len and after[i] >= '0' and after[i] <= '9') : (i += 1) {} + // Untrusted provider IDs/counts must not overflow in safety builds. + return std.fmt.parseInt(i32, after[start..i], 10) catch 0; } pub fn extractJsonFloat(json: []const u8, key: []const u8) f32 { @@ -116,9 +115,7 @@ pub fn parseCinemetaResponse(body: []const u8, out: *std.ArrayListUnmanaged(stat } fn copyInto(dst: []u8, len: *usize, src: []const u8) void { - const n = @min(dst.len, src.len); - @memcpy(dst[0..n], src[0..n]); - len.* = n; + len.* = @import("json_pure.zig").jsonUnescape(src, dst).len; } fn parseCinemetaGenres(json: []const u8, item: *state.TmdbItem) void { @@ -160,6 +157,9 @@ fn parseAndAddCinemetaItem(json: []const u8, out: *std.ArrayListUnmanaged(state. if (extractJsonString(json, "\"name\":")) |name| copyInto(&item.title, &item.title_len, name); if (extractJsonString(json, "\"year\":")) |year| copyInto(&item.year, &item.year_len, year); + if (item.year_len == 0) { + if (extractJsonString(json, "\"releaseInfo\":")) |year| copyInto(&item.year, &item.year_len, year); + } if (extractJsonString(json, "\"released\":")) |released| { if (item.year_len == 0 and released.len >= 4) copyInto(&item.year, &item.year_len, released[0..4]); if (released.len >= 10) { @@ -187,18 +187,18 @@ fn parseAndAddCinemetaItem(json: []const u8, out: *std.ArrayListUnmanaged(state. } fn parseAndAddItem(json_obj: []const u8, out: *std.ArrayListUnmanaged(state.TmdbItem)) !void { + // Multi-search includes people; those are not playable catalog titles. + if (extractJsonString(json_obj, "\"media_type\":")) |kind| { + if (!std.mem.eql(u8, kind, "movie") and !std.mem.eql(u8, kind, "tv")) return; + } var item = state.TmdbItem{}; item.id = extractJsonInt(json_obj, "\"id\":"); if (extractJsonString(json_obj, "\"title\":")) |title| { - const tlen = @min(title.len, 127); - @memcpy(item.title[0..tlen], title[0..tlen]); - item.title_len = tlen; + copyInto(&item.title, &item.title_len, title); } else if (extractJsonString(json_obj, "\"name\":")) |name| { - const nlen = @min(name.len, 127); - @memcpy(item.title[0..nlen], name[0..nlen]); - item.title_len = nlen; + copyInto(&item.title, &item.title_len, name); } if (extractJsonString(json_obj, "\"release_date\":")) |date| { @@ -230,9 +230,7 @@ fn parseAndAddItem(json_obj: []const u8, out: *std.ArrayListUnmanaged(state.Tmdb item.rating = extractJsonFloat(json_obj, "\"vote_average\":"); if (extractJsonString(json_obj, "\"overview\":")) |ov| { - const olen = @min(ov.len, 511); - @memcpy(item.overview[0..olen], ov[0..olen]); - item.overview_len = olen; + copyInto(&item.overview, &item.overview_len, ov); } if (extractJsonString(json_obj, "\"media_type\":")) |mt| { diff --git a/src/services/vndb.zig b/src/services/vndb.zig index da1fafb3..6672efbc 100644 --- a/src/services/vndb.zig +++ b/src/services/vndb.zig @@ -71,7 +71,9 @@ const CoverSlot = struct { failed: bool = false, url_hash: u64 = 0, }; -var cover_slots: [180]CoverSlot = [_]CoverSlot{.{}} ** 180; +// Extra slot belongs to independent universal-search detail, not the grid. +const DETAIL_COVER_INDEX = 180; +var cover_slots: [181]CoverSlot = [_]CoverSlot{.{}} ** 181; // ── Thread-safety ── // The detached fetch worker publishes into state.app.vndb.* under `parse_mutex`, @@ -80,6 +82,15 @@ var cover_slots: [180]CoverSlot = [_]CoverSlot{.{}} ** 180; // remote threads, written by the worker). var parse_mutex: @import("../core/sync.zig").Mutex = .{}; var search_request: LatestRequest = .{}; +var identity_request: LatestRequest = .{}; +var identity_loading = std.atomic.Value(bool).init(false); +var identity_mutex: @import("../core/sync.zig").Mutex = .{}; +var identity_record: pure.Vn = .{}; +var identity_error: bool = false; +var identity_ready: bool = false; +var identity_active: bool = false; // UI-thread owned. +var identity_id: [16]u8 = undefined; +var identity_id_len: usize = 0; // Query snapshot handed to the detached search worker (never read the mutable UI // search_buf from the thread). `is_popular` selects which request body is built. @@ -309,12 +320,63 @@ fn fetchPage(job: SearchJob, page: u32, append: bool) void { /// Open the detail overlay for result `idx`. pub fn openDetail(idx: usize) void { + identity_active = false; + identity_request.cancel(&identity_loading); if (idx >= state.app.vndb.result_count) return; state.app.vndb.selected_idx = idx; } +/// Open a canonical VNDB identity without replacing the browse query or grid. +pub fn openCatalogIdentity(id: []const u8) void { + if (id.len < 2 or id.len > identity_id.len or id[0] != 'v' or id[1] == '0') return; + for (id[1..]) |ch| if (!std.ascii.isDigit(ch)) return; + const generation = identity_request.begin(&identity_loading); + identity_active = true; + identity_id_len = id.len; + @memcpy(identity_id[0..id.len], id); + identity_mutex.lock(); + identity_record = .{}; + identity_ready = false; + identity_error = false; + identity_mutex.unlock(); + state.app.vndb.selected_idx = null; + state.navigateToTab(.Vndb); + const Job = struct { + id: [16]u8, + id_len: usize, + generation: u32, + fn run(job: @This()) void { + defer { + identity_request.finish(job.generation, &identity_loading); + state.wakeUi(); + } + var request_buf: [640]u8 = undefined; + const request = std.fmt.bufPrint(&request_buf, "{{\"filters\":[\"id\",\"=\",\"{s}\"],\"fields\":\"{s}\",\"results\":1}}", .{ job.id[0..job.id_len], pure.FIELDS }) catch return; + rate_limit.acquire("vndb", 1.0); + const response = curlPost(API_URL, request, 128 * 1024); + defer if (response) |body| alloc.free(body); + var records: [1]pure.Vn = .{.{}}; + const count = if (response) |body| pure.parseVns(body, &records) else 0; + identity_mutex.lock(); + defer identity_mutex.unlock(); + if (!identity_request.isCurrent(job.generation)) return; + identity_ready = count == 1; + identity_error = count != 1; + if (count == 1) identity_record = records[0]; + } + }; + @import("../core/workers.zig").spawn(Job.run, .{Job{ .id = identity_id, .id_len = identity_id_len, .generation = generation }}) catch { + identity_mutex.lock(); + identity_error = true; + identity_mutex.unlock(); + identity_request.finish(generation, &identity_loading); + }; +} + /// Close the detail overlay, returning to the grid. pub fn closeDetail() void { + identity_active = false; + identity_request.cancel(&identity_loading); state.app.vndb.selected_idx = null; } @@ -342,35 +404,22 @@ fn searchTorrents(idx: usize) void { /// SEGVs on some ISP TLS resets (see comics.zig). fn curlPost(url: []const u8, body: []const u8, cap: usize) ?[]u8 { const argv = [_][]const u8{ - "curl", "-sL", "-X", "POST", - "-H", "Content-Type: application/json", "-A", agent, - "-d", body, "--max-time", "15", - url, - }; - var child = io.Child.init(&argv, alloc); - child.stdout_behavior = .Pipe; - child.stderr_behavior = .Ignore; - _ = child.spawn() catch return null; - - const buf = alloc.alloc(u8, cap) catch { - _ = child.wait() catch {}; - return null; - }; - const n = if (child.stdout) |*so| io.readAll(so, buf) catch 0 else 0; - _ = child.wait() catch {}; - if (n == 0) { - alloc.free(buf); - return null; - } - - // Shrink to what was actually read. The caller frees what we hand back, and - // the global DebugAllocator checks the free size against the allocation size - // β€” returning buf[0..n] out of a cap-sized allocation is an INVALID FREE and - // aborts the process (the radio/podcasts twin of this helper does the same). - return alloc.realloc(buf, n) catch { - alloc.free(buf); - return null; + "curl", "-fsSL", "--connect-timeout", "3", "--max-time", "10", + "-H", "Content-Type: application/json", "-A", agent, "--data", body, + "--", url, }; + const buf = alloc.alloc(u8, cap) catch return null; + defer alloc.free(buf); + const result = @import("../core/bounded_process.zig").run(&argv, buf, .{ .timeout_ms = 11_000 }); + if (!result.ok() or result.output.len == 0) return null; + // Reject HTTP/error documents and malformed schemas before latching an + // empty successful catalog. Valid empty result arrays remain successful. + var parsed = std.json.parseFromSlice(std.json.Value, alloc, result.output, .{}) catch return null; + defer parsed.deinit(); + if (parsed.value != .object) return null; + const records = parsed.value.object.get("results") orelse return null; + if (records != .array) return null; + return alloc.dupe(u8, result.output) catch null; } // ══════════════════════════════════════════════════════════ @@ -397,6 +446,24 @@ pub fn renderContent() void { var pageroot = dvui.box(@src(), .{ .dir = .vertical }, .{ .expand = .both }); defer pageroot.deinit(); + if (identity_active) { + identity_mutex.lock(); + const record = identity_record; + const ready = identity_ready; + const failed = identity_error; + identity_mutex.unlock(); + if (ready) { + renderDetailRecord(&record, DETAIL_COVER_INDEX); + } else { + if (components.actionButton(@src(), "Back", .secondary, 62001)) closeDetail(); + _ = dvui.label(@src(), "{s}", .{if (identity_loading.load(.acquire)) "Loading visual novel details..." else if (failed) "Details unavailable, not found, or excluded by the cover filter." else "No details available."}, .{ .color_text = theme.colors.text_secondary }); + if (failed and components.actionButton(@src(), "Retry", .secondary, 62002)) { + const id = identity_id; + openCatalogIdentity(id[0..identity_id_len]); + } + } + return; + } // Populate the page on first open (no-op after the first fetch). loadPopularOnce(); @@ -469,9 +536,28 @@ const CARD_FOOTER_H: f32 = 52; // title + rating/year lines under the cover fn renderCover(i: usize, v: *const pure.Vn) void { const slot = &cover_slots[i]; const url = v.image_url[0..v.image_url_len]; + if (url.len == 0) { + if (!slot.fetching) { + poster.deinitPoster(&slot.pixels, &slot.tex); + slot.* = .{}; + } + // A delayed previous cover must never decorate a coverless record. + _ = dvui.icon(@src(), "", icons.tvg.lucide.@"gamepad-2", .{}, .{ + .id_extra = i + 1000, + .color_text = theme.colors.text_tertiary, + .gravity_x = 0.5, + .gravity_y = 0.5, + .expand = .both, + }); + return; + } if (url.len > 0) { const h = std.hash.Fnv1a_64.hash(url); + if (slot.url_hash != h and slot.fetching) { + components.coverSkeleton(@src(), i + 1000, 8); + return; // An older cover must not be painted on a different VN. + } if (slot.url_hash != h and !slot.fetching) { poster.deinitPoster(&slot.pixels, &slot.tex); slot.w = 0; @@ -672,8 +758,10 @@ fn renderResults() void { // ── Detail overlay ── fn renderDetail(idx: usize) void { - const v = &state.app.vndb.results[idx]; + renderDetailRecord(&state.app.vndb.results[idx], idx); +} +fn renderDetailRecord(v: *const pure.Vn, idx: usize) void { var scroll = dvui.scrollArea(@src(), .{}, .{ .expand = .both, .background = true, @@ -768,7 +856,9 @@ fn renderDetail(idx: usize) void { .padding = .{ .x = 12, .y = 6, .w = 12, .h = 6 }, .margin = .{ .x = 0, .y = 12, .w = 0, .h = 0 }, })) { - searchTorrents(idx); + state.navigateToTab(.Search); + search.submitQuery(v.title[0..v.title_len]); + state.showToast("Searching all sources..."); } } } diff --git a/src/ui/components.zig b/src/ui/components.zig index bc4e577b..0c35d9c8 100644 --- a/src/ui/components.zig +++ b/src/ui/components.zig @@ -172,6 +172,7 @@ pub const CoverSlot = struct { attempted: bool = false, failed: bool = false, url_hash: u64 = 0, + desired_url_hash: u64 = 0, pub fn reset(self: *CoverSlot) void { if (self.fetching) return; @@ -186,6 +187,7 @@ pub const CoverSlot = struct { pub fn syncCoverSlot(slot: *CoverSlot, key: []const u8) void { const hash = if (key.len > 0) std.hash.Fnv1a_64.hash(key) else 0; + slot.desired_url_hash = hash; if (slot.url_hash != hash and !slot.fetching) { slot.reset(); slot.url_hash = hash; @@ -193,6 +195,12 @@ pub fn syncCoverSlot(slot: *CoverSlot, key: []const u8) void { } pub fn pollCoverSlot(slot: *CoverSlot) void { + if (slot.url_hash != slot.desired_url_hash) { + if (slot.fetching) return; + const desired = slot.desired_url_hash; + slot.reset(); + slot.url_hash = desired; + } _ = poster.uploadIfReady(&slot.pixels, slot.w, slot.h, &slot.tex); if (slot.fetching) { slot.attempted = true; @@ -202,15 +210,15 @@ pub fn pollCoverSlot(slot: *CoverSlot) void { } pub fn renderCoverSlot(src: std.builtin.SourceLocation, id_extra: usize, slot: *CoverSlot, expects_art: bool, fallback_icon: []const u8, radius: f32) void { - if (slot.tex) |tex| { + if (slot.url_hash == slot.desired_url_hash) if (slot.tex) |tex| { _ = dvui.image(src, .{ .source = .{ .texture = tex } }, .{ .id_extra = id_extra, .expand = .both, .corner_radius = dvui.Rect.all(radius), }); return; - } - if (expects_art and !slot.failed) { + }; + if (expects_art and (!slot.failed or slot.url_hash != slot.desired_url_hash)) { coverSkeleton(src, id_extra, radius); return; } diff --git a/src/ui/podcasts_ui.zig b/src/ui/podcasts_ui.zig index 818251c7..1417c536 100644 --- a/src/ui/podcasts_ui.zig +++ b/src/ui/podcasts_ui.zig @@ -19,16 +19,8 @@ const loadEpisodes = podcasts.loadEpisodes; const playEpisode = podcasts.playEpisode; const closeEpisodes = podcasts.closeEpisodes; -const PodPoster = struct { - pixels: ?[]u8 = null, - tex: ?dvui.Texture = null, - w: u32 = 0, - h: u32 = 0, - fetching: bool = false, - attempted: bool = false, - failed: bool = false, - url_hash: u64 = 0, -}; +const PodPoster = components.CoverSlot; + var pod_posters: [50]PodPoster = [_]PodPoster{.{}} ** 50; pub fn renderContent() void { @@ -104,49 +96,7 @@ const CARD_FOOTER_H: f32 = 46; // title + publisher lines under the cover /// daemon. Falls back to the podcast glyph while loading, when the show has no /// artwork URL, or when the image can't be decoded. UI-thread only. fn renderCover(i: usize, p: *const pure.Podcast) void { - const slot = &pod_posters[i]; - const art = p.artwork[0..p.artwork_len]; - - if (art.len > 0) { - // Pin the slot to whatever show is at index i now β€” a re-search (or the - // popular chart landing) can replace results[], so a URL-hash change - // means "different show here": free the stale texture/pixels and - // refetch. Only when not mid-fetch, so we never spawn a second worker - // onto the same slot. - const h = std.hash.Fnv1a_64.hash(art); - if (slot.url_hash != h and !slot.fetching) { - poster.deinitPoster(&slot.pixels, &slot.tex); - slot.w = 0; - slot.h = 0; - slot.attempted = false; - slot.failed = false; - slot.url_hash = h; - } - _ = poster.uploadIfReady(&slot.pixels, slot.w, slot.h, &slot.tex); - if (slot.fetching) slot.attempted = true else if (slot.attempted and slot.pixels == null and slot.tex == null) slot.failed = true; - if (!slot.failed and slot.tex == null and !slot.fetching and slot.pixels == null) { - poster.fetchAsync(art, &slot.pixels, &slot.w, &slot.h, &slot.fetching); - if (slot.fetching) slot.attempted = true; - } - } - - if (slot.tex) |*tex| { - _ = dvui.image(@src(), .{ .source = .{ .texture = tex.* } }, .{ - .id_extra = i + 1000, - .expand = .both, - .corner_radius = dvui.Rect.all(8), - }); - } else if (art.len > 0 and !slot.failed) { - components.coverSkeleton(@src(), i + 1000, 8); - } else { - _ = dvui.icon(@src(), "", icons.tvg.lucide.podcast, .{}, .{ - .id_extra = i + 1000, - .color_text = theme.colors.text_tertiary, - .gravity_x = 0.5, - .gravity_y = 0.5, - .expand = .both, - }); - } + components.coverArt(@src(), i + 1000, &pod_posters[i], p.artwork[0..p.artwork_len], icons.tvg.lucide.podcast, 8); } /// One show card: square cover (clickable) + title + publisher subtitle. diff --git a/tests/test_expanded_torrent_sources.py b/tests/test_expanded_torrent_sources.py index 3cefa844..d22a29f7 100644 --- a/tests/test_expanded_torrent_sources.py +++ b/tests/test_expanded_torrent_sources.py @@ -61,6 +61,27 @@ def test_anime_sources_do_not_emit_for_unsupported_music_category(self): rows, calls = self.run_engine(name, [], 'track', 'music') self.assertEqual((rows, calls), ([], [])) + def test_animetosho_keeps_real_swarm_counts_and_canonical_hash(self): + digest = 'a' * 40 + row = {'title': 'A & B\nEpisode 4', 'info_hash': digest, + 'magnet_uri': 'magnet:?xt=urn:btih:' + digest + '&tr=https%3A%2F%2Ftracker.test%2Fannounce', + 'total_size': '4096', 'seeders': '3', 'leechers': None, + 'link': 'https://animetosho.org/view/release.123', 'timestamp': 12345} + rows, calls = self.run_engine('animetosho', [json.dumps([row, row, + {**row, 'info_hash': 'bad'}, {**row, 'status': 'deleted'}])], 'A%26B%20dub') + self.assertEqual(len(rows), 1) + self.assertIn('q=A%26B+dub', calls[0]) + self.assertEqual(rows[0]['name'], 'A & B Episode 4') + self.assertIn('xt=urn:btih:' + digest, rows[0]['link']) + self.assertEqual((rows[0]['size'], rows[0]['seeds'], rows[0]['leech']), (4096, 3, -1)) + self.assertEqual(rows[0]['pub_date'], 12345) + + def test_animetosho_refuses_unrelated_category_and_error_envelopes(self): + rows, calls = self.run_engine('animetosho', [], 'book', 'books') + self.assertEqual((rows, calls), ([], [])) + rows, _ = self.run_engine('animetosho', [json.dumps({'error': 'offline'})], 'anime') + self.assertEqual(rows, []) + if __name__ == '__main__': unittest.main() diff --git a/web/js/catalog.js b/web/js/catalog.js index 28c1e743..a0e0d29e 100644 --- a/web/js/catalog.js +++ b/web/js/catalog.js @@ -43,50 +43,92 @@ $('open-queue').onclick = async () => { $('open-hint').textContent = 'Added to the Opal queue.'; }; -function runSearch(){ +let unifiedSearchRun = 0; +async function runSearch(){ const q = $('q').value.trim(); if (!q) return; + const request = ++unifiedSearchRun; + clearInterval(searchWatch); $('search-hint').innerHTML = ' Searching all sources…'; $('results').innerHTML = ''; lastHtml.results = ''; - api('/unified_search?q=' + encodeURIComponent(q)).catch(()=>{}); - clearInterval(searchWatch); - let ticks = 0; + let initial; + try { + initial = await api('/unified_search?q=' + encodeURIComponent(q)); + if (request !== unifiedSearchRun) return; + renderUnifiedResults(initial); + $('search-hint').textContent = unifiedResultCount(initial); + if (!initial.loading) return; + } catch (error) { + if (request === unifiedSearchRun) $('search-hint').textContent = error?.message || 'Search could not start. Try again.'; + return; + } searchWatch = settledInterval(async () => { - ticks++; try { const d = await api('/unified_search'); - renderUnifiedResults(d); - if ((!d.loading && ticks > 2) || ticks > 40) { + if (request !== unifiedSearchRun) return; + if (d.generation !== initial.generation) { clearInterval(searchWatch); - const sources = new Set((d.results || []).map(r => r.source).filter(Boolean)); - $('search-hint').textContent = `${(d.results || []).length} results across ${sources.size} source${sources.size === 1 ? '' : 's'}.`; + $('search-hint').textContent = 'Search changed in another Opal view. Search again to refresh.'; + return; } - } catch { clearInterval(searchWatch); } + renderUnifiedResults(d); + $('search-hint').textContent = unifiedResultCount(d); + if (!d.loading) clearInterval(searchWatch); + } catch (error) { + if (request !== unifiedSearchRun) return; + clearInterval(searchWatch); + $('search-hint').textContent = error?.message || 'Could not refresh search. Loaded results remain available.'; + } }, 900); } - +function unifiedResultCount(payload){ + const rows = Array.isArray(payload.results) ? payload.results : []; + const returned = Number.isFinite(payload.returned) ? payload.returned : rows.length; + const total = Number.isFinite(payload.total) ? payload.total : returned; + const sources = new Set(rows.map(r => r.source).filter(Boolean)); + const unavailable = (payload.sources || []).filter(s => s.enabled && + ['failed', 'transport_failed', 'parse_failed', 'timed_out', 'partial', 'unavailable'].includes(s.status)); + let text = payload.truncated || total > returned ? `Showing ${returned} of ${total} results` : `${returned} results`; + text += ` across ${sources.size} source${sources.size === 1 ? '' : 's'}`; + if (payload.loading) text += ' Β· Still searching'; + if (unavailable.length) text += ' Β· ' + unavailable.map(s => `${s.source}: ${String(s.status).replace(/_/g, ' ')}`).join('; '); + return text + '.'; +} function unifiedActionLabel(source){ - if (source === 'comics') return 'Read'; + if (source === 'comics' || source === 'novels') return 'Read'; + if (source === 'vndb') return 'View details'; + if (source === 'audiobooks') return 'Open audio'; if (source === 'podcast' || source === 'anime' || source === 'tmdb') return 'Open'; return 'Play on Opal'; } +function unifiedArtwork(value){ + if (typeof value !== 'string' || !value) return ''; + try { + const url = new URL(value); + return ['https:', 'http:'].includes(url.protocol) && !url.username && !url.password ? url.href : ''; + } catch { return ''; } +} function renderUnifiedResults(payload){ - const rs = payload.results || [], generation = payload.generation || 0; - const shown = rs.slice(0, 80); + const shown = Array.isArray(payload.results) ? payload.results : [], generation = payload.generation || 0; const html = shown.map((r, i) => { - return `
    -
    ${esc(r.title)}
    + const artwork = unifiedArtwork(r.poster_url), rating = Number(r.rating); + return `
    + ${artwork ? `` : ''} +
    ${esc(r.title)}
    ${esc(r.source || '')} + ${r.author ? `${esc(r.author)}` : ''} ${r.detail ? `${esc(r.detail)}` : ''} + ${Number.isFinite(rating) && rating > 0 && rating <= 10 ? `${rating.toFixed(1)}/10` : ''} ${r.queueable ? `` : ''} - + -
    +
    ${r.summary ? `
    ${esc(r.summary)}
    ` : ''}
    `; - }).join('') || '
    No results yet
    '; + }).join('') || `
    ${payload.loading ? 'Searching sources…' : 'No results found. Try another title or check source availability.'}
    `; if (html === lastHtml.results) return; lastHtml.results = html; $('results').innerHTML = html; + $('results').querySelectorAll('.thumb').forEach(image => image.addEventListener('error', () => image.remove(), { once:true })); $('results').querySelectorAll('.play').forEach(b => b.onclick = () => runUnifiedAction(shown[+b.dataset.i], +b.dataset.gen, 'play', b)); $('results').querySelectorAll('.queue-btn').forEach(b => @@ -112,6 +154,7 @@ async function runUnifiedAction(r, generation, action, button){ } function runStreamSearch(title){ if (!title) return; + ++unifiedSearchRun; $('q').value = title; $('search-hint').innerHTML = ' Finding playable streams…'; $('results').innerHTML = ''; lastHtml.results = ''; diff --git a/web/js/media.js b/web/js/media.js index df808773..756a6efd 100644 --- a/web/js/media.js +++ b/web/js/media.js @@ -223,7 +223,7 @@ function renderMusic(songs){ let raWatch = null; // Parity-tier-2 poll handles. `cxPages` is separate from `cxWatch`: the reader // keeps polling page progress while the search listing sits idle behind it. -let comicGeneration = 0, comicSearchGeneration = 0, radioGeneration = 0; +let comicGeneration = 0, comicSearchGeneration = 0, radioGeneration = 0, novelGeneration = 0; let cxWatch = null, cxPages = null, nvWatch = null, drWatch = null, vnWatch = null; let absWatch = null, opWatch = null, plWatch = null; // Which search row the open novel came from β€” /novels has no server-side back. @@ -285,10 +285,34 @@ function renderComics(rows){ }, button); }); } +function renderComicChapters(d){ + let controls = $('cx-chapter-controls'); + if (!controls) { + controls = document.createElement('div'); + controls.id = 'cx-chapter-controls'; + controls.className = 'm'; + $('cx-pages').before(controls); + } + const chapters = d.chapters || []; + const html = `${d.prev_url ? '' : ''} + ${chapters.length ? `` : ''} + ${d.next_url ? '' : ''}`; + if (controls.dataset.signature === html) return; + controls.dataset.signature = html; + controls.innerHTML = html; + const previous = controls.querySelector('[data-comic-prev]'); + const next = controls.querySelector('[data-comic-next]'); + const select = controls.querySelector('[data-comic-chapter]'); + if (previous) previous.onclick = () => openComic(d.prev_url); + if (next) next.onclick = () => openComic(d.next_url); + if (select) select.onchange = () => openComic(select.value); +} + async function openComic(url){ const generation = ++comicGeneration; clearInterval(cxPages); $('cx-pages').innerHTML = ''; + if ($('cx-chapter-controls')) { $('cx-chapter-controls').innerHTML = ''; delete $('cx-chapter-controls').dataset.signature; } try { await api('/comics/load?url=' + encodeURIComponent(url)); } catch { if (generation === comicGeneration) $('cx-progress').textContent = 'Could not open comic. Try again.'; return; } if (generation !== comicGeneration) return; @@ -303,18 +327,32 @@ async function openComic(url){ try { const d = await api('/comics'); if (generation !== comicGeneration) return; - // `downloaded` is what says which indices answer 200 β€” pages arrive out of - // order across 8 workers, so render only the contiguous prefix. - $('cx-progress').textContent = d.pages + // Download workers finish out of order; only request actual ready pages + // and keep their nodes in reading order as earlier pages arrive. + $('cx-progress').textContent = d.loading ? 'Loading chapter…' : d.pages ? `${d.title || 'Reading'} β€” ${d.downloaded}/${d.pages} pages` : 'Loading…'; - if (d.pages) { - $('cx-pages').innerHTML = Array.from({ length: d.downloaded }, (_, i) => - ``).join(''); + if (!d.loading) renderComicChapters(d); + if (d.pages && !d.loading) { + // Preserve already loaded image nodes while later pages arrive. + const pages = $('cx-pages'); + const ready = d.ready_pages || Array.from({length:d.downloaded}, (_, i) => i); + for (const i of ready) { + if (pages.querySelector(`[data-page="${i}"]`)) continue; + const image = document.createElement('img'); + image.className = 'cx-page'; + image.dataset.page = i; + image.loading = 'lazy'; + image.alt = `Page ${i + 1}`; + image.src = `${BASE}/api/comics/page?i=${i}&reader=${generation}`; + const following = [...pages.querySelectorAll('.cx-page')].find(node => Number(node.dataset.page) > i); + pages.insertBefore(image, following || null); + } } - if ((d.pages && d.downloaded >= d.pages) || (!d.loading && !d.pages) || ticks > 90) { + if ((d.pages && !d.loading && d.downloaded >= d.pages) || (!d.loading && !d.pages) || ticks > 90) { clearInterval(cxPages); if (!d.pages) $('cx-progress').textContent = 'No readable pages found. Try another source or issue.'; + else if (d.downloaded < d.pages) $('cx-progress').textContent = `${d.title || 'Reading'} β€” ${d.downloaded}/${d.pages} pages loaded. Some pages could not load; try reopening this chapter.`; } } catch { clearInterval(cxPages); } }, 1200); @@ -334,36 +372,55 @@ function loadNovels(){ pollNovels(); } function runNovels(){ const q = $('nv-q').value.trim(); if (!q) return; $('nv-hint').innerHTML = ' Searching…'; - api('/novels/search?q=' + encodeURIComponent(q)).catch(()=>{}); - pollNovels(); + novelAction('/novels/search?q=' + encodeURIComponent(q)); +} +async function novelAction(path){ + const generation = ++novelGeneration; + clearInterval(nvWatch); + try { + await api(path); + if (generation === novelGeneration) pollNovels(); + } catch { + if (generation === novelGeneration) $('nv-hint').textContent = 'Could not load novels. Try again.'; + } } function pollNovels(){ + const generation = ++novelGeneration; clearInterval(nvWatch); let ticks = 0; nvWatch = settledInterval(async () => { ticks++; try { const d = await api('/novels'); + if (generation !== novelGeneration) return; renderNovels(d); - const busy = d.loading || d.chapters_loading || d.text_loading; - if ((!busy && ticks > 1) || ticks > 60) clearInterval(nvWatch); - } catch { clearInterval(nvWatch); } + const busy = d.loading || d.chapters_loading || d.text_loading || d.loading_more; + if (!busy && ticks > 1) clearInterval(nvWatch); + } catch { + if (generation === novelGeneration) { + clearInterval(nvWatch); + $('nv-hint').textContent = 'Could not load novels. Try again.'; + } + } }, 900); } function renderNovels(d){ - const busy = d.loading || d.chapters_loading || d.text_loading; + const busy = d.loading || d.chapters_loading || d.text_loading || d.loading_more; + let more = $('nv-more'); + if (!more) { + more = document.createElement('button'); + more.id = 'nv-more'; more.className = 'more'; more.textContent = 'Load more'; + $('nv-results').after(more); + more.onclick = () => novelAction('/novels/more'); + } + more.hidden = d.view !== 'search' || !d.has_more; + more.disabled = busy; $('nv-hint').innerHTML = busy ? ' Loading…' - : (d.error ? 'Fetch failed β€” try another source.' : (d.title || 'Search to begin.')); + : (d.error ? 'Fetch failed β€” try another source.' : esc(d.title || 'Search to begin.')); $('nv-crumbs').innerHTML = d.view === 'search' ? '' : ``; const back = $('nv-back'); - if (back) back.onclick = () => { - // No server-side "back": re-entering the previous view is just re-issuing - // the call that produced it. - if (d.view === 'reader') api('/novels/open?idx=' + novelIdx).catch(()=>{}); - else api('/novels/search?q=' + encodeURIComponent($('nv-q').value.trim())).catch(()=>{}); - pollNovels(); - }; + if (back) back.onclick = () => novelAction('/novels/back'); if (d.view === 'reader') { $('nv-results').innerHTML = ''; $('nv-text').textContent = d.text || ''; @@ -376,8 +433,10 @@ function renderNovels(d){ const target = $('nv-results'); const html = rows.map((r, i) => `
    - ${r.cover ? `` : ''} + ${unifiedArtwork(r.cover) ? `` : ''}
    ${esc(r.title)}
    + ${r.author || r.year ? `
    ${esc([r.author, r.year || ''].filter(Boolean).join(' Β· '))}
    ` : ''} + ${r.overview ? `
    ${esc(r.overview.slice(0, 180))}${r.overview.length > 180 ? '…' : ''}
    ` : ''}
    `).join('') || '
    Nothing here
    '; @@ -386,14 +445,13 @@ function renderNovels(d){ b.onclick = () => { const i = +b.dataset.nv; if (b.dataset.kind === 'open') novelIdx = i; - api('/novels/' + (b.dataset.kind === 'open' ? 'open' : 'chapter') + '?idx=' + i).catch(()=>{}); - pollNovels(); + novelAction('/novels/' + (b.dataset.kind === 'open' ? 'open' : 'chapter') + '?idx=' + i); }; }); target.querySelectorAll('.novel-details').forEach(button => { const row = rows[Number(button.dataset.details)] || {}; button.onclick = () => openSourceDetails('Novel', { - ...row, name:row.title, type:button.dataset.kind === 'chapter' ? 'Chapter' : 'Novel', + ...row, name:row.title, artUrl:row.cover || '', author:row.author || '', overview:row.overview || '', year:row.year || 0, type:button.dataset.kind === 'chapter' ? 'Chapter' : 'Novel', kind:button.dataset.kind === 'chapter' ? 'chapter' : 'novel', index:Number(button.dataset.details), }, button); }); @@ -517,8 +575,8 @@ function pollAbs(){ try { const d = await api('/abs'); renderAbs(d); - if ((!d.loading && ticks > 1) || ticks > 40) clearInterval(absWatch); - } catch { clearInterval(absWatch); } + if (!d.loading && !d.loading_more && !d.audio?.loading && ticks > 1) clearInterval(absWatch); + } catch (error) { clearInterval(absWatch); $('abs-hint').textContent = error?.message || 'Could not refresh Audiobookshelf.'; } }; absWatch = settledInterval(tick, 900, true); } @@ -526,8 +584,36 @@ function renderAbs(d){ $('abs-login').style.display = d.connected ? 'none' : ''; $('abs-session').hidden = !d.connected; if (!d.connected && d.server && !$('abs-server').value) $('abs-server').value = d.server; - $('abs-hint').innerHTML = d.loading ? ' Loading…' - : (d.error || (d.connected ? (d.library || 'Pick a library') : 'Sign in to your Audiobookshelf server.')); + $('abs-hint').innerHTML = d.loading || d.audio?.loading ? ' Loading…' + : esc(d.error || (d.connected ? (d.library || 'Pick a library') : 'Sign in to your Audiobookshelf server.')); + if (d.connected && d.audio?.visible) { + const audio = d.audio, tracks = audio.tracks || [], target = $('abs-results'); + $('abs-crumbs').innerHTML = ''; + $('abs-audio-back').onclick = async () => { + try { await apiMutation('/abs/audio/close'); pollAbs(); } + catch (error) { toast(error.message || 'Could not close track selection'); } + }; + const html = `
    ${esc(audio.title || 'Audio files')} Β· ${audio.returned || 0} available of ${audio.total || 0}
    ` + + (audio.complete_book ? '
    Book tracks advance automatically and resume across files.
    ' : '
    Choose an individual audio file or podcast episode.
    ') + + (audio.loading ? '
    Loading audio files…
    ' : tracks.map(track => + `
    ${esc(track.title)}
    ${track.episode ? 'Podcast episode' : 'Audio track'}
    `).join('') + || '
    No playable audio files found.
    '); + const changed = setSafeHtml(target, html); + if ($('abs-play-book')) $('abs-play-book').onclick = async () => { + try { await apiMutation('/abs/audio/book?generation=' + encodeURIComponent(audio.generation)); toast('Playing book from its saved position'); } + catch (error) { toast(error.message || 'Could not play the complete book'); } + }; + if (changed) target.querySelectorAll('button[data-audio]').forEach(button => { + button.onclick = async () => { + button.disabled = true; + try { + await apiMutation('/abs/audio?idx=' + encodeURIComponent(button.dataset.audio) + '&generation=' + encodeURIComponent(audio.generation)); + button.textContent = 'Sent'; toast('Playing selected audio file on Opal'); + } catch (error) { button.disabled = false; toast(error.message || 'Could not play this audio file'); } + }; + }); + return; + } const books = d.view === 'Books'; if (d.connected && !d.loading && !books && !(d.libraries || []).length && !absRestoreRequested) { absRestoreRequested = true; @@ -537,7 +623,7 @@ function renderAbs(d){ if ($('abs-back')) $('abs-back').onclick = () => { apiMutation('/abs/back').catch(()=>{}); pollAbs(); }; const rows = books ? (d.books || []) : (d.libraries || []); const target = $('abs-results'); - const html = rows.map((r, i) => ` + const html = (rows.map((r, i) => `
    ${esc(r.title || r.name)}
    @@ -545,11 +631,19 @@ function renderAbs(d){ ${r.media_type ? `${esc(r.media_type)}` : ''} ${r.duration ? `${fmt(r.duration)}` : ''} ${books ? `` : ''} -
    -
    `).join('') || (d.connected ? '
    Nothing here
    ' : ''); +
    +
    `).join('') || (d.connected ? '
    Nothing here
    ' : '')) + + (books && d.has_more ? `` : ''); if (!setSafeHtml(target, html)) return; + if ($('abs-more')) $('abs-more').onclick = async () => { + try { await apiMutation('/abs/more'); pollAbs(); } + catch (error) { toast(error.message || 'Could not load more books'); } + }; target.querySelectorAll('button[data-i]').forEach(b => { - b.onclick = () => { apiMutation('/abs/' + (books ? 'play' : 'open') + '?idx=' + b.dataset.i).catch(()=>{}); pollAbs(); }; + b.onclick = async () => { + try { await apiMutation('/abs/' + (books ? 'play' : 'open') + '?idx=' + b.dataset.i); pollAbs(); } + catch (error) { toast(error.message || 'Could not open this item'); } + }; }); target.querySelectorAll('.abs-details').forEach(button => { button.onclick = () => {