curl -X POST 'https://api.firecrawl.dev/v1/crawl' \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
  "url": "https://example.com",
  "excludePaths": [
    "string"
  ],
  "includePaths": [
    "string"
  ],
  "maxDepth": 10,
  "maxDiscoveryDepth": 0,
  "ignoreSitemap": false,
  "ignoreQueryParameters": false,
  "limit": 10000,
  "allowBackwardLinks": false,
  "allowExternalLinks": false,
  "delay": 0,
  "webhook": {
    "url": "string",
    "headers": {},
    "metadata": {},
    "events": [
      "completed"
    ]
  },
  "scrapeOptions": {
    "formats": [
      "markdown"
    ],
    "onlyMainContent": true,
    "includeTags": [
      "string"
    ],
    "excludeTags": [
      "string"
    ],
    "maxAge": 0,
    "headers": {},
    "waitFor": 0,
    "mobile": false,
    "skipTlsVerification": false,
    "timeout": 30000,
    "parsePDF": true,
    "jsonOptions": {
      "schema": {},
      "systemPrompt": "string",
      "prompt": "string"
    },
    "actions": [
      {
        "type": "wait",
        "milliseconds": 0,
        "selector": "#my-element"
      }
    ],
    "location": {
      "country": "US",
      "languages": [
        "en-US"
      ]
    },
    "removeBase64Images": true,
    "blockAds": true,
    "proxy": "basic",
    "changeTrackingOptions": {
      "modes": [
        "git-diff"
      ],
      "schema": {},
      "prompt": "string",
      "tag": null
    },
    "storeInCache": true
  }
}'

# POST /crawl (Firecrawl API)
Base URL: https://api.firecrawl.dev/v1
Auth: Authorization: Bearer <FIRECRAWL_API_KEY>
Last verified: 2026-09-03

> Crawl multiple URLs based on options

## Critical Gotchas
* The `formats` field controls the output shape and defaults to `["markdown"]`. Omitting it does not return HTML — request `["markdown","html"]` explicitly.
* `/crawl` is asynchronous: it returns a job `id`, and results are collected by polling `/crawl/{id}` until `status` is `completed`.

## cURL
curl -X POST 'https://api.firecrawl.dev/v1/crawl' \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com","excludePaths":["string"]}'

## Parameters
| Name | In | Type | Required | Description |
| :--- | :--- | :--- | :--- | :--- |
| url | body | string | Yes | The base URL to start crawling from |
| excludePaths | body | string[] | No | URL pathname regex patterns that excl... |
| includePaths | body | string[] | No | URL pathname regex patterns that incl... |

## Response (200 OK)
{"success":true,"id":"string","url":"https://example.com"}

---
Need an API key? Get one at https://www.firecrawl.dev/signin/signup

POST

/crawl

Firecrawl API

Crawl multiple URLs based on options

Base URL: https://api.firecrawl.dev/v1
Auth: Authorization: Bearer <FIRECRAWL_API_KEY>
Last verified: 2026-09-03 · upstream hash matched

Actions

Agents: curl -H "Accept: text/markdown" this URL
→ 224 tokens · Vary: Accept

Critical gotchas

The formats field controls the output shape and defaults to ["markdown"]. Omitting it does not return HTML — request ["markdown","html"] explicitly.

/crawl is asynchronous: it returns a job id, and results are collected by polling /crawl/{id} until status is completed.

cURL

curl -X POST 'https://api.firecrawl.dev/v1/crawl' \  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \  -H 'Content-Type: application/json' \  -d '{  "url": "https://example.com",  "excludePaths": [    "string"  ],  "includePaths": [    "string"  ],  "maxDepth": 10,  "maxDiscoveryDepth": 0,  "ignoreSitemap": false,  "ignoreQueryParameters": false,  "limit": 10000,  "allowBackwardLinks": false,  "allowExternalLinks": false,  "delay": 0,  "webhook": {    "url": "string",    "headers": {},    "metadata": {},    "events": [      "completed"    ]  },  "scrapeOptions": {    "formats": [      "markdown"    ],    "onlyMainContent": true,    "includeTags": [      "string"    ],    "excludeTags": [      "string"    ],    "maxAge": 0,    "headers": {},    "waitFor": 0,    "mobile": false,    "skipTlsVerification": false,    "timeout": 30000,    "parsePDF": true,    "jsonOptions": {      "schema": {},      "systemPrompt": "string",      "prompt": "string"    },    "actions": [      {        "type": "wait",        "milliseconds": 0,        "selector": "#my-element"      }    ],    "location": {      "country": "US",      "languages": [        "en-US"      ]    },    "removeBase64Images": true,    "blockAds": true,    "proxy": "basic",    "changeTrackingOptions": {      "modes": [        "git-diff"      ],      "schema": {},      "prompt": "string",      "tag": null    },    "storeInCache": true  }}'

Get a free Firecrawl API key → sponsored

Parameters

Name	In	Type	Required	Description
url	body	string	Yes	The base URL to start crawling from
excludePaths	body	string[]	No	URL pathname regex patterns that exclude matching URLs from the crawl. For example, if you set "excludePaths": ["blog/.*"] for the base URL firecrawl.dev, any results matching that pattern will be excluded, such as https://www.firecrawl.dev/blog/firecrawl-launch-week-1-recap.
includePaths	body	string[]	No	URL pathname regex patterns that include matching URLs in the crawl. Only the paths that match the specified patterns will be included in the response. For example, if you set "includePaths": ["blog/.*"] for the base URL firecrawl.dev, only results matching that pattern will be included, such as https://www.firecrawl.dev/blog/firecrawl-launch-week-1-recap.
maxDepth	body	integer	No	Maximum depth to crawl relative to the base URL. Basically, the max number of slashes the pathname of a scraped URL may contain.
maxDiscoveryDepth	body	integer	No	Maximum depth to crawl based on discovery order. The root site and sitemapped pages has a discovery depth of 0. For example, if you set it to 1, and you set ignoreSitemap, you will only crawl the entered URL and all URLs that are linked on that page.
ignoreSitemap	body	boolean	No	Ignore the website sitemap when crawling
ignoreQueryParameters	body	boolean	No	Do not re-scrape the same path with different (or none) query parameters
limit	body	integer	No	Maximum number of pages to crawl. Default limit is 10000.
allowBackwardLinks	body	boolean	No	Allows the crawler to follow internal links to sibling or parent URLs, not just child paths. false: Only crawls deeper (child) URLs. → e.g. /features/feature-1 → /features/feature-1/tips ✅ → Won't follow /pricing or / ❌ true: Crawls any internal links, including siblings and parents. → e.g. /features/feature-1 → /pricing, /, etc. ✅ Use true for broader internal coverage beyond nested paths.
allowExternalLinks	body	boolean	No	Allows the crawler to follow links to external websites.
delay	body	number	No	Delay in seconds between scrapes. This helps respect website rate limits.
webhook	body	object	No	A webhook specification object.
scrapeOptions	body	object	No	Request body field scrapeOptions.

Response 200 OK

{
  "success": true,
  "id": "string",
  "url": "https://example.com"
}