chore(deps): update dependency @nitpicker/crawler to ^0.21.0 - #1038
Open
renovate[bot] wants to merge 1 commit into
Open
chore(deps): update dependency @nitpicker/crawler to ^0.21.0#1038renovate[bot] wants to merge 1 commit into
renovate[bot] wants to merge 1 commit into
Conversation
renovate
Bot
requested review from
YusukeHirao,
arktds,
dz-furukawa,
teppei-d and
yusasa16
as code owners
August 26, 2026 17:09
renovate
Bot
force-pushed
the
renovate/nitpicker-crawler-0.x
branch
4 times, most recently
from
September 1, 2026 02:59
d95c14d to
171aff4
Compare
renovate
Bot
force-pushed
the
renovate/nitpicker-crawler-0.x
branch
from
September 1, 2026 10:40
171aff4 to
91b382e
Compare
renovate
Bot
force-pushed
the
renovate/nitpicker-crawler-0.x
branch
3 times, most recently
from
September 2, 2026 22:34
f93bd0a to
620ea18
Compare
renovate
Bot
force-pushed
the
renovate/nitpicker-crawler-0.x
branch
from
September 3, 2026 03:28
620ea18 to
deaf643
Compare
renovate
Bot
force-pushed
the
renovate/nitpicker-crawler-0.x
branch
from
September 3, 2026 13:30
deaf643 to
da40a2e
Compare
renovate
Bot
force-pushed
the
renovate/nitpicker-crawler-0.x
branch
3 times, most recently
from
September 7, 2026 20:13
52ddab7 to
637d666
Compare
renovate
Bot
force-pushed
the
renovate/nitpicker-crawler-0.x
branch
5 times, most recently
from
September 8, 2026 10:55
7ee46a1 to
a47a304
Compare
renovate
Bot
force-pushed
the
renovate/nitpicker-crawler-0.x
branch
from
September 8, 2026 11:57
a47a304 to
f57b700
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
^0.18.0→^0.21.0Release Notes
d-zero-dev/nitpicker (@nitpicker/crawler)
v0.21.0Compare Source
Note: Version bump only for package @nitpicker/crawler
v0.20.0Compare Source
Bug Fixes
v0.19.0Compare Source
Bug Fixes
crawler: compute page body hash once per page instead of twice (f2fbda5)
crawler: route self-healing and session notices away from bare console.* (deb5684)
feat(crawler)!: implement Symbol.asyncDispose for archive lifecycle (de60624)
feat(crawler)!: replace Wappalyzer-only page_tags with technology detection (4b81a47)
Features
BREAKING CHANGES
Management (
using/await using) support.ArchiveAccessor (and, via polymorphism, Archive) and CrawlerOrchestrator
now implement
Symbol.asyncDispose, delegating to their existingclose()/garbageCollect() logic. Callers can use
await usinginsteadof a manual try/finally around teardown.
page_tagsis removed outright in favor of two newtables:
technology_signals(raw per-signal evidence, one row persignal) and
page_technologies(noisy-OR confidence-combined roll-up,one row per page x technology). Existing archives are converted via a
read-time migration (
migrate-page-tags-to-page-technologies.tsfor0.13+ archives, a special case in
retarget-legacy-fk-tables.tsfor the0.10->0.13 upgrade path) that drops
page_tagsafter conversion.Page.getTags()and theTagsSummary/TagInventoryEntrytypes areremoved along with it; use
Page.getTechnologySignals()/Page.getPageTechnologies()instead.Also adds a post-crawl JS license-comment scan (
scanJsResourcesForTechnologySignals)that re-fetches the leading bytes of not-yet-scanned internal JS
resources to detect technologies whose signature lives in the JS body
rather than the HTML — a distinct "post-crawl network enrichment"
category from the existing post-hoc-marking and backfill patterns,
since it is the only step that performs network I/O after the crawl
itself finishes. Each resource is scanned at most once (recorded in
technology_js_scan_cache), and a match folds into every page thatreferences the resource by recombining that page's full signal set
under the same Scoped-Replace invariant
insertTechnologiesuses.Also detects Web Components (custom elements, tag names containing a
hyphen and not one of the 8 reserved SVG/MathML names) within the main
content region, at the same grain as the existing button/table/etc.
categories (
page_main_content_custom_elements,main_content_custom_element_count).0.18.1 (2026-08-12)
Bug Fixes
bf75031found (69c3597)v0.18.1Compare Source
Bug Fixes
bf75031found (69c3597)Configuration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.