feat(velocity): tighten F1+F2+F3 cron to 10-min cadence + emit staleness_seconds - #3175
Open
0motionguy wants to merge 1 commit into
Open
0motionguy wants to merge 1 commit into
0motionguy wants to merge 1 commit into
Conversation
…ess_seconds Match TrendShift's live-mentions refresh cadence (2026-06-15 deltas evidence, impact 4.0) so the velocity board never trails third-party leaderboards by more than one tick. - velocity-refresh: */40 → */10 (300 cheap GH calls × 6/hr) - velocity-backfill: 17 2 * → 5,15,25,35,45,55 (interleaved from :05) - star-activity-deltas: 30 5 * → 3,13,23,33,43,53 (interleaved from :03) The three writers of star-activity-deltas now run on staggered :03/:05/:10 marks so they never collide on the shared slug. velocity-backfill's existingCoversWindows skip-guard keeps the PAGE_BUDGET (30k GraphQL points) honest at the new cadence — once a repo's 7d/30d resolves, subsequent ticks just refresh the recent tier instead of re-paging the full history. Also: each fetcher now emits `staleness_seconds` alongside `computedAt` on the star-activity-deltas payload so downstream readers carry the freshness contract on-wire and can drop stale slugs without a side channel. Field is optional on the interface for back-compat — pre-2026-06-15 payloads omit it. Evidence: https://trendshift.io/live-mentions Refs: toolbox PR #241 (deltas evidence pipeline) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tightens the three velocity fetchers from their previous cron cadence (
*/40and two daily slots) to a 10-min cadence to match TrendShift's live-mentions refresh, and emitsstaleness_secondson thestar-activity-deltaspayload so downstream consumers can drop stale slugs without a side channel.velocity-refresh(F1)*/40 * * * **/10 * * * *velocity-backfill(F2)17 2 * * *5,15,25,35,45,55 * * * *star-activity-deltas(F3)30 5 * * *3,13,23,33,43,53 * * * *The three writers of
star-activity-deltasnow run on staggered:03 / :05 / :10marks within each 10-min window, so they never pile up on the same minute — refresh's re-read-before-merge race window stays the same width it always was.Why now
Deltas evidence from the 2026-06-15 TrendingRepo crawl pipeline (toolbox PR #241) shows TrendShift's live-mentions board refreshes at ~10-min cadence (impact 4.0). At our prior
*/40cadence the homepage velocity board could trail it by up to 40 minutes. Source of truth: https://trendshift.io/live-mentionsCost honesty
velocity-refresh(cheap 1-call REST): 300 calls × 6/hr = 1.8k/hr — trivial for the 20-token GH pool's 100k/hr ceiling.velocity-backfill(GraphQL page-back withPAGE_BUDGET=30000): the existingexistingCoversWindowsskip-guard already short-circuits repos whose 7d/30d resolve from prior runs, so the deep-walk budget converges to "refresh recent tier only" after the first full pass. Worst-case first day might draw ~5× the prior daily draw against the GraphQL pool; subsequent days settle.star-activity-deltas(Redis-only recompute): 8-worker fan-out over ~5000 keys × 6/hr = 240k Redis ops/hr. Cheap.staleness_seconds
StarActivityDeltasPayload.staleness_seconds?: number— emitted asMath.round((Date.now() - Date.parse(computedAt)) / 1000)at publish time (≈0). Field is optional for back-compat — pre-2026-06-15 payloads omit it. Downstream readers can recompute the live value fromcomputedAt; this just makes the freshness contract explicit on-wire.Out of scope
engagement-composite/scoring.ts— unchanged per task spec.velocity-seed(the daily expensive walker stays daily).Test plan
cd apps/trendingrepo-worker && npx tsc --noEmit— clean, no errors introduced.vitest runblocked by pre-existing tinypoolminThreads/maxThreadsconflict invitest.config.ts(unrelated to this PR; reproduces on base branch). Pure helpers tested byvelocity-backfill.test.tsare untouched.velocity-refreshpublished at:00,:10,:20,:30,:40,:50,velocity-backfillat:05,:15,:25,:35,:45,:55,star-activity-deltasat:03,:13,:23,:33,:43,:53.star-activity-deltasslug now carriesstaleness_secondsfield (Redis CLI:GET ss:data:v1:star-activity-deltas | jq .staleness_seconds).velocity-backfillcost is bounded byPAGE_BUDGETbut the per-day GraphQL draw will be higher than17 2 * * *baseline until the skip-guard converges.Refs: toolbox PR #241 (deltas evidence pipeline).
Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com