Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 7 additions & 27 deletions skills/firecrawl-agent/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: firecrawl-agent
description: |
AI-powered autonomous data extraction that navigates complex sites and returns structured JSON. Use this skill when the user wants structured data from websites, needs to extract pricing tiers, product listings, directory entries, or any data as JSON with a schema. Triggers on "extract structured data", "get all the products", "pull pricing info", "extract as JSON", or when the user provides a JSON schema for website data. More powerful than simple scraping for multi-page structured extraction.
Autonomous multi-page extraction into structured JSON. Use when the user wants website data matching a schema — pricing tiers, product listings — beyond a single-page scrape.
allowed-tools:
- Bash(firecrawl *)
- Bash(npx firecrawl-cli *)
Expand All @@ -11,12 +11,6 @@ allowed-tools:

AI-powered autonomous extraction. The agent navigates sites and extracts structured data (takes 2-5 minutes).

## When to use

- You need structured data from complex multi-page sites
- Manual scraping would require navigating many pages
- You want the AI to figure out where the data lives

## Quick start

```bash
Expand All @@ -30,9 +24,13 @@ firecrawl agent "extract products" --schema '{"type":"object","properties":{"nam
firecrawl agent "get feature list" --urls "<url>" --wait --json -o .firecrawl/features.json
```

Run `firecrawl agent --help` for the full option list.

**Done when:** the output file contains valid JSON answering the request — or a job ID was intentionally returned for later polling.

## Job IDs

Without `--wait`, the command returns a job ID. A UUID positional argument is auto-detected as a status check:
Omitting `--wait` returns a job ID. A UUID positional argument is auto-detected as a status check:

```bash
# Check once (equivalent to adding --status)
Expand All @@ -45,27 +43,9 @@ firecrawl agent "<job-id>" --wait --poll-interval 10 --timeout 300
firecrawl agent "<job-id>" --cancel
```

## Options

| Option | Description |
| --------------------------- | ----------------------------------------------------- |
| `--urls <urls>` | Starting URLs for the agent |
| `--model <model>` | Model to use: spark-1-mini or spark-1-pro |
| `--schema <json>` | JSON schema for structured output |
| `--schema-file <path>` | Path to JSON schema file |
| `--max-credits <n>` | Credit limit for this agent run |
| `--status` | Check a job ID's status |
| `--cancel` | Cancel an active job ID |
| `--wait` | Wait for agent to complete |
| `--poll-interval <seconds>` | Polling interval while waiting (default: 5 seconds) |
| `--timeout <seconds>` | Stop waiting after this duration (default: none) |
| `--json` | Output as JSON |
| `--pretty` | Pretty print JSON (`--wait` results require `--json`) |
| `-o, --output <path>` | Output file path |

## Tips

- Use `--wait` for inline results; without it you get a job ID (see [Job IDs](#job-ids)).
- Use `--wait` for inline results; omit it only when you want a job ID to poll later (see [Job IDs](#job-ids)).
- Use `--schema` for predictable, structured output — otherwise the agent returns freeform data.
- Agent runs consume more credits than simple scrapes. Use `--max-credits` to cap spending.
- For simple single-page extraction, prefer `scrape` — it's faster and cheaper.
Expand Down
29 changes: 6 additions & 23 deletions skills/firecrawl-crawl/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: firecrawl-crawl
description: |
Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.
Bulk-extract many pages from one site or section. Use for "crawl", "everything under /docs", or content spanning linked pages.
allowed-tools:
- Bash(firecrawl *)
- Bash(npx firecrawl-cli *)
Expand All @@ -13,12 +13,6 @@ Bulk extract content from a website. Crawls pages following links up to a depth/

**Prerequisite:** `crawl` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.

## When to use

- You need content from many pages on a site (e.g., all `/docs/`)
- You want to extract an entire site section
- Step 4 in the [workflow escalation pattern](../firecrawl/SKILL.md): search → scrape → map + scrape → **crawl** → monitor → interact

## Quick start

```bash
Expand All @@ -32,25 +26,14 @@ firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json
firecrawl crawl <job-id>
```

## Options

| Option | Description |
| ------------------------- | ------------------------------------------- |
| `--wait` | Wait for crawl to complete before returning |
| `--progress` | Show progress while waiting |
| `--limit <n>` | Max pages to crawl |
| `--max-depth <n>` | Max link depth to follow |
| `--include-paths <paths>` | Only crawl URLs matching these paths |
| `--exclude-paths <paths>` | Skip URLs matching these paths |
| `--delay <ms>` | Delay between requests |
| `--max-concurrency <n>` | Max parallel crawl workers |
| `--pretty` | Pretty print JSON output |
| `-o, --output <path>` | Output file path |
Run `firecrawl crawl --help` for the full option list.

**Done when:** the crawl reaches a terminal status and the saved output under `.firecrawl/` contains the expected pages.

## Tips

- Always use `--wait` when you need the results immediately. It has no default timeout; use `--timeout <seconds>` to bound polling. Without `--wait`, crawl returns a job ID for async polling.
- Use `--include-paths` to scope the crawl — don't crawl an entire site when you only need one section.
- Use `--wait` when you need the results immediately. It has no default timeout; use `--timeout <seconds>` to bound polling. Without `--wait`, crawl returns a job ID for async polling.
- **Scope crawls with `--include-paths`** whenever the request names a section — crawl only the pages you need.
- Crawl consumes credits per page. Check `firecrawl credit-usage` before large crawls (`credit-usage` requires authentication).

## See also
Expand Down
38 changes: 4 additions & 34 deletions skills/firecrawl-download/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: firecrawl-download
description: |
Download an entire website as local files markdown, screenshots, or multiple formats per page. Use this skill when the user wants to save a site locally, download documentation for offline use, bulk-save pages as files, or says "download the site", "save as local files", "offline copy", "download all the docs", or "save for reference". Combines site mapping and scraping into organized local directories.
Save a site or section as local files (markdown, screenshots). Use for "download the site", offline docs, or a local copy for reference.
allowed-tools:
- Bash(firecrawl *)
- Bash(npx firecrawl-cli *)
Expand All @@ -13,20 +13,11 @@ allowed-tools:

**Prerequisite:** `download` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.

Maps the site origin first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. Use `--include-paths` to scope a non-root URL to one section. Supported scrape options are listed below. Always pass `-y` to skip the confirmation prompt.

## When to use

- You want to save an entire site (or section) to local files
- You need offline access to documentation or content
- Bulk content extraction with organized file structure
Maps the site origin first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. Use `--include-paths` to scope a non-root URL to one section. Automated runs always pass `-y` — without it the command opens an interactive wizard that blocks on a prompt.

## Quick start

```bash
# Interactive wizard (humans at a TTY only — agents must pass -y or the command blocks on a prompt)
firecrawl x download https://docs.example.com

# With screenshots
firecrawl x download https://docs.example.com --screenshot --limit 20 -y

Expand All @@ -39,32 +30,11 @@ firecrawl x download https://docs.example.com --include-paths "/features,/sdks"

# Skip translations
firecrawl x download https://docs.example.com --exclude-paths "/zh,/ja,/fr,/es,/pt-BR" -y

# Full combo
firecrawl x download https://docs.example.com \
--include-paths "/features,/sdks" \
--exclude-paths "/zh,/ja" \
--only-main-content \
--screenshot \
-y
```

## Download options

| Option | Description |
| ------------------------- | -------------------------------------------------------- |
| `--limit <n>` | Max pages to download |
| `--search <query>` | Filter URLs by search query |
| `--include-paths <paths>` | Only download matching paths |
| `--exclude-paths <paths>` | Skip matching paths |
| `--allow-subdomains` | Include subdomain pages |
| `-y` | Skip confirmation prompt (always use in automated flows) |

## Supported scrape options

Only the options listed below are supported:
Run `firecrawl x download --help` for the full option list, including which scrape options download supports.

`-f <formats>`, `-H`, `-S`, `--lockdown`, `--screenshot`, `--full-page-screenshot`, `--only-main-content`, `--include-tags`, `--exclude-tags`, `--wait-for`, `--max-age`, `--country`, `--languages`
**Done when:** the command exits successfully and the expected files exist under `.firecrawl/`.

## See also

Expand Down
27 changes: 6 additions & 21 deletions skills/firecrawl-interact/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,15 @@
---
name: firecrawl-interact
description: |
Control and interact with a live browser session on any scraped pageclick buttons, fill forms, navigate flows, and extract data using natural language prompts or code. Use when the user needs to interact with a webpage beyond simple scraping: logging into a site, submitting forms, clicking through pagination, handling infinite scroll, navigating multi-step checkout or wizard flows, or when a regular scrape failed because content is behind JavaScript interaction. Also useful for authenticated scraping via profiles. Triggers on "interact", "click", "fill out the form", "log in to", "sign in", "submit", "paginated", "next page", "infinite scroll", "interact with the page", "navigate to", "open a session", or "scrape failed".
Drive a live browser on a scraped page: click, fill forms, log in, paginate, infinite-scroll. Use when content requires interaction or a scrape failed or returned incomplete content.
allowed-tools:
- Bash(firecrawl *)
- Bash(npx firecrawl-cli *)
---

# firecrawl interact

Interact with scraped pages in a live browser session. Scrape a page first, then use natural language prompts or code to click, fill forms, navigate, and extract data.

## When to use

- Content requires interaction: clicks, form fills, pagination, login
- `scrape` failed because content is behind JavaScript interaction
- You need to navigate a multi-step flow
- Last resort in the [workflow escalation pattern](../firecrawl/SKILL.md): search → scrape → map + scrape → crawl → monitor → **interact**
- **Never use interact for web searches** — use `search` instead
Interact with scraped pages in a live browser session. Scrape a page first, then use natural language prompts or code to click, fill forms, navigate, and extract data. For web searches, use `search` — interact is for acting on a specific page.

## Quick start

Expand All @@ -41,16 +33,9 @@ firecrawl interact --code "agent-browser snapshot -i" --bash
firecrawl interact stop
```

## Options
Run `firecrawl interact --help` for the full option list.

| Option | Description |
| -------------------------------- | ------------------------------------------------- |
| `--prompt <text>` | Natural language instruction (use this OR --code) |
| `--code <code>` | Code to execute in the browser session |
| `--node` / `--python` / `--bash` | Language for `--code` (default: node) |
| `--timeout <seconds>` | Execution timeout (default: 30, max: 300) |
| `--scrape-id <id>` | Target a specific scrape (default: last scrape) |
| `-o, --output <path>` | Output file path |
**Done when:** the requested content or action result is captured and the session is stopped with `firecrawl interact stop`.

## Profiles

Expand All @@ -75,12 +60,12 @@ firecrawl scrape "https://app.example.com" --profile my-app --no-save-changes
## Tips

- Always scrape first — `interact` requires a scrape ID from a previous `firecrawl scrape` call
- The scrape ID is saved automatically, so you don't need `--scrape-id` for subsequent interact calls. Saved sessions may expire after about 10 minutes; re-scrape if the CLI warns that the session is stale
- The scrape ID is saved automatically, so you can omit `--scrape-id` for subsequent interact calls. Saved sessions may expire after about 10 minutes; re-scrape if the CLI warns that the session is stale
- Use `firecrawl interact stop` to free resources when done
- For parallel work, scrape multiple pages and interact with each using `--scrape-id`

## See also

- [firecrawl-scrape](../firecrawl-scrape/SKILL.md) — try scrape first, escalate to interact only when needed
- [firecrawl-search](../firecrawl-search/SKILL.md) — for web searches (never use interact for searching)
- [firecrawl-search](../firecrawl-search/SKILL.md) — use `search` for web searches
- [firecrawl-agent](../firecrawl-agent/SKILL.md) — AI-powered extraction (less manual control)
19 changes: 3 additions & 16 deletions skills/firecrawl-map/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: firecrawl-map
description: |
Discover and list all URLs on a website, with optional search filtering. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where something is on a domain, or says "map the site", "find the URL for", "what pages are on", or "list all pages". Essential when the user knows which site but not which exact page.
Discover and list a site's URLs, with search filtering. Use for "map the site" or "find the URL for" requests — when the user knows the site but not the exact page, or wants site structure.
allowed-tools:
- Bash(firecrawl *)
- Bash(npx firecrawl-cli *)
Expand All @@ -13,12 +13,6 @@ Discover URLs on a site. Use `--search` to find a specific page within a large s

**Prerequisite:** `map` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.

## When to use

- You need to find a specific subpage on a large site
- You want a list of all URLs on a site before scraping or crawling
- Step 3 in the [workflow escalation pattern](../firecrawl/SKILL.md): search → scrape → **map** + scrape → crawl → monitor → interact

## Quick start

```bash
Expand All @@ -29,16 +23,9 @@ firecrawl map "<url>" --search "authentication" -o .firecrawl/filtered.txt
firecrawl map "<url>" --limit 500 --json -o .firecrawl/urls.json
```

## Options
Run `firecrawl map --help` for the full option list (sitemap handling, subdomains, etc.).

| Option | Description |
| --------------------------------- | ---------------------------- |
| `--limit <n>` | Max number of URLs to return |
| `--search <query>` | Filter URLs by search query |
| `--sitemap <include\|skip\|only>` | Sitemap handling strategy |
| `--include-subdomains` | Include subdomain URLs |
| `--json` | Output as JSON |
| `-o, --output <path>` | Output file path |
**Done when:** the URL list is saved under `.firecrawl/` and you have selected the URLs to scrape or crawl next.

## Tips

Expand Down
Loading
Loading