Skip to content

refactor(skills): rewrite and optimize agent skills - #196

Open
chenxin-yan wants to merge 6 commits into
firecrawl:mainfrom
chenxin-yan:skills/optimized-r3
Open

refactor(skills): rewrite and optimize agent skills#196
chenxin-yan wants to merge 6 commits into
firecrawl:mainfrom
chenxin-yan:skills/optimized-r3

Conversation

@chenxin-yan

@chenxin-yan chenxin-yan commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

rewrite agent skills to follow best practices, reducing context bloat, better skills invocation, eliminate duplication among skill files.

problems

  • old descriptions are bloated and included "what" but not much "why" or "when" to hint agent to pull in the skills. additionally, in some harnesses the descriptions appears to be truncated as the descriptions are too long. Solution: i shorten the descriptions to follow the pattern of 1 short sentence explain what and 1-2 sentences explaining why and when
  • some skills have "When to use" section which is redundant as model would not see that section when deciding to use the skills. Solution: remove all the "when to use" sections
  • most skills have a section to include all the options/flags available and this could easily drift from the implementation Solution: hint the agent to use <command> --help to get up to date options
  • some instructions such as auth and routing and so on are scattered/duplicated around different files. Solution: each topic now has one canonical file (auth → firecrawl/rules/install.md, monitor detail → the monitor skill, search feedback → the search skill) and the router points to it
  • skills didn't tell the agent when a task is actually done, which can cause premature completion or over-work. Solution: add a one-line "Done when" completion criterion to each skill
  • there are some behavior issues that are observed when running evals. Solutions: three targeted rules
  1. agents sometimes fell back to default web tools for ordinary research tasks
  2. on 429 rate limits back off once then stop and clean up (monitor skill: agents were retrying in a loop)
  3. auth/credit errors are terminal: verify config once, report, stop

token impact (~4 chars/token estimate)

Skill bodies (loaded on invocation):

file baseline tok optimized tok Δ %
firecrawl-monitor/SKILL.md 4,925 1,286 -3,639 -74%
firecrawl/SKILL.md (router) 4,709 2,250 -2,459 -52%
firecrawl-search/SKILL.md 2,821 1,986 -835 -30%
firecrawl-scrape/SKILL.md 1,140 648 -492 -43%
firecrawl-parse/SKILL.md 864 395 -469 -54%
firecrawl-agent/SKILL.md 951 542 -409 -43%
firecrawl-download/SKILL.md 854 463 -391 -46%
firecrawl-interact/SKILL.md 1,047 684 -363 -35%
firecrawl-crawl/SKILL.md 743 422 -321 -43%
firecrawl-map/SKILL.md 574 355 -219 -38%
firecrawl/rules/security.md 349 349 0 0%
firecrawl/rules/install.md 558 702 +144 +26%
firecrawl-monitor/goals.md 1,181 +1,181 new
firecrawl-monitor/json-tracking.md 638 +638 new
TOTAL 19,535 11,901 -7,634 -39%

Frontmatter descriptions (always loaded in every agent's skill catalog):

skill baseline tok optimized tok Δ %
firecrawl-monitor 285 70 -215 -75%
firecrawl-interact 201 61 -140 -70%
firecrawl-parse 187 55 -132 -71%
firecrawl-search 170 63 -107 -63%
firecrawl (router) 192 92 -100 -52%
firecrawl-agent 142 61 -81 -57%
firecrawl-crawl 118 49 -69 -58%
firecrawl-download 120 52 -68 -57%
firecrawl-scrape 118 56 -62 -53%
firecrawl-map 112 52 -60 -54%
TOTAL 1,645 611 -1,034 -63%

eval results

metric baseline optimized Δ
task success 30/45 (66.7%) 31/45 (68.9%) +2.2pp
mean cost/trace (clean) $0.292 $0.263 -10%
cost per success $0.528 $0.470 -11%
input tokens / tool calls 326k / 10.96 299k / 10.62 -8% / -3%

Descriptions are always-loaded context; trigger-first wording cuts token load while keeping every distinct trigger branch.
… content

- Router SKILL.md 329 -> 141 lines: pure routing; monitor details,
  install/auth, and search-feedback specifics now live only in their
  sub-skills, with routing pointers left behind.
- Moved two unique monitor facts into firecrawl-monitor (markdown-only
  diff.json parse-diff AST note; goal bullet about user-named noise).
- Resolved keyless contradiction: keyless free tier is real for
  search/scrape/interact (src/utils/client.ts isKeylessMode; scrape and
  search absent from AUTH_REQUIRED_COMMANDS in src/index.ts). Single
  correct statement kept in rules/install.md, including which commands
  prompt interactive login.
…positive phrasing, completion criteria

- Replace cached CLI option tables with 'firecrawl <cmd> --help' pointers,
  keeping only options the recipes actually use (all verified against src/)
- Delete 'When to use' sections that restated frontmatter descriptions
- Split firecrawl-monitor: lean recipe in SKILL.md, goal/query authoring in
  goals.md, JSON change tracking in json-tracking.md, each behind a one-line
  'read when' pointer
- Rewrite never/don't phrasing as positive instructions; keep genuine
  opt-out and data guardrails
- Add exactly one 'Done when' completion criterion per skill

skills/**/*.md drops from 1045 to 837 lines (SKILL.md files: 1045 -> 723).
…d auth terminal rules

EXP-046 forensics: claude SKOP-NEU-02 skipped Firecrawl in optimized arms
(lost web-research trigger); monitor traces looped on 429 to 600s timeouts;
bad-key traces retried identical failing calls.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 13 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread skills/firecrawl-map/SKILL.md Outdated
Comment thread skills/firecrawl-scrape/SKILL.md Outdated
Comment thread skills/firecrawl-interact/SKILL.md Outdated
Comment thread skills/firecrawl-monitor/json-tracking.md
@chenxin-yan chenxin-yan changed the title refactor(skills): rewrite and optomize agent skills refactor(skills): rewrite and optimize agent skills Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant