ai web scraping

Add Firecrawl to your AI agent

Firecrawl automates web crawling and data extraction, enabling organizations to gather content, index sites, and gain insights from online sources at scale

Things your agent can do in Firecrawl
29

Things your AI can do in Firecrawl

Your FormWise AI can handle any of these in Firecrawl — on its own, without you needing to click a button.

  • Batch scrape multiple URLs

    Popular

    Tool to scrape multiple URLs in batch with concurrent processing. Use when you need to scrape multiple web pages efficiently with customizable formats and content filtering.

  • Get batch scrape status

    Popular

    Retrieves the current status and results of a batch scrape job using the job ID. Use this to check batch scrape progress and retrieve scraped data.

  • Cancel a crawl job

    Popular

    Cancels an active or queued web crawl job using its ID; attempting to cancel completed, failed, or previously canceled jobs will not change their state.

  • Get crawl job status

    Popular

    Tool to retrieve the status and results of a Firecrawl crawl job. Use when you need to check the progress or get data from an ongoing or completed crawl operation. Returns crawl status, progress metrics, credits used, and the crawled page data.

  • Get team credit usage

    Popular

    Tool to get current team credit usage information. Use when you need to check remaining credits or billing period details.

  • Get extract job status

    Popular

    Tool to retrieve the status and results of a previously submitted extract job. Use when you need to check the progress or get the final results of an extraction operation.

  • Get the status of a crawl job

    Popular

    Retrieves the current status, progress, and details of a web crawl job, using the job ID obtained when the crawl was initiated.

  • Map multiple URLs

    Popular

    Maps a website by discovering URLs from a starting base URL, with options to customize the crawl via search query, subdomain inclusion, sitemap handling, and result limits; search effectiveness is site-dependent.

  • Start an agent job

    Popular

    Tool to start an agent job for agentic web extraction with multi-page navigation and interaction capabilities. Use when you need to autonomously gather data from the web with complex navigation requirements. The agent can search, navigate, and extract information across multiple pages based on your natural language prompt.

  • Get team token usage

    Popular

    Tool to retrieve the current team's token usage and balance information for Firecrawl's Extract feature. Use when you need to check remaining token credits, plan allocation, or billing period details.

  • Cancel an agent job

    Tool to cancel an in-progress agent job by its ID. Use when you need to terminate an active agent operation. The API returns a success boolean upon cancellation.

  • Cancel a batch scrape job

    Tool to cancel a running batch scrape job using its unique identifier. Use when you need to terminate an in-progress batch scrape operation.

Show 17 more things it can do
  • Get errors from batch scrape job

    Tool to retrieve error details from a batch scrape job, including failed URLs and URLs blocked by robots.txt. Use when you need to debug or understand why certain pages failed to scrape in a batch operation.

  • Start a web crawl

    Initiates a Firecrawl web crawl from a given URL, applying various filtering and content extraction rules, and polls until the job is complete; ensure the URL is accessible and any regex patterns for paths are valid.

  • Cancel a crawl job

    Tool to cancel a running crawl job by its ID. Use when you need to stop an active crawl operation. The API returns a status of 'cancelled' upon successful cancellation.

  • Get errors from a crawl job

    Tool to retrieve errors from a Firecrawl crawl job. Use when you need to understand why certain pages failed to scrape or which URLs were blocked by robots.txt during a crawl operation.

  • Get all active crawl jobs

    Tool to retrieve all active crawl jobs for the authenticated team. Use when you need to see which crawl operations are currently running.

  • Preview crawl parameters

    Preview crawl parameters before starting a crawl by generating optimal configuration from natural language instructions. Use this tool to understand what crawl settings will be applied based on your requirements before executing a full crawl operation. The endpoint intelligently interprets natural language prompts to configure crawl parameters like include/exclude paths, depth limits, and domain scope.

  • Start a web crawl (v2) [NEW]

    [NEW v2 API] Initiates a Firecrawl v2 web crawl with enhanced features over v1: natural language prompts for automatic crawler configuration, crawlEntireDomain for sibling/parent page discovery, better depth control with maxDiscoveryDepth, subdomain support, and full webhook configuration. Polls until crawl is complete.

  • Get historical team credit usage

    Tool to retrieve historical team credit usage on a monthly basis. Use when you need to analyze credit consumption patterns over time, optionally segmented by API key.

  • Extract structured data

    Extracts structured data from web pages by initiating an extraction job and polling for completion; requires a natural language `prompt` or a JSON `schema` (one must be provided).

  • Get agent job status

    Tool to get the status and results of an agent job. Use when you need to check if an agent job has completed and retrieve the collected data. Agent jobs autonomously search, navigate, and extract data from the web.

  • Get deep research status

    Retrieves the status and results of a deep research job by its ID. Use when you need to check the progress or retrieve the final analysis of a deep research operation.

  • Generate LLMs.txt for a website

    Initiates an async job to generate an LLMs.txt file for a website, converting web content into LLM-friendly format. Returns a job ID to check status and retrieve results. Use when you need to create a standardized, machine-readable representation of website content for language models.

  • Get LLMs.txt generation job status

    Tool to get the status and results of an LLMs.txt generation job. Use when you need to check if a job has completed and retrieve the generated content.

  • Get team queue status

    Tool to retrieve metrics about the team's scrape queue. Use when you need to check queue status, job counts, or concurrency limits.

  • Scrape URL

    Scrapes a publicly accessible URL, optionally performing pre-scrape browser actions or extracting structured JSON using an LLM, to retrieve content in specified formats.

  • Search

    Performs a web search for a query, scrapes content from the top search results using Firecrawl, and returns details in specified formats.

  • Get historical team token usage

    Tool to retrieve historical team token usage on a monthly basis. Use when you need to analyze token consumption patterns over time, optionally segmented by API key.

Built for the way you actually work

No code, no engineers

Connect in a few clicks and start using your AI right away. If you can use the apps you already love, you can do this.

Sounds like you, not like a bot

Train your AI on your offer, your voice, and the way you actually talk to clients — it picks up where you leave off.

Plugs into the stack you already use

No need to rip out your CRM, your scheduler, or your email tool. Your AI works inside the apps running your business today.

Your data stays yours

Client conversations, lists, and credentials are scoped to your workspace. Nothing gets shared, sold, or used to train anyone else’s AI.

What you can do once Firecrawl is connected

Onboard new clients without lifting a finger

When someone signs up, your AI greets them, sets up their next step in Firecrawl, and keeps the momentum going — even at 2am.

Reply to leads in seconds, not days

Your AI watches Firecrawl for new inquiries, answers questions in your voice, and books the call before they cool off.

Get your weekends back

Hand off the repetitive Firecrawl work — follow-ups, reminders, updates, simple admin — and stop being the bottleneck in your own business.

How to connect Firecrawl to your agent

  1. 1

    Start using FormWise for free

    Sign up in under a minute.

  2. 2

    Find Firecrawl in your connected apps

    Inside FormWise, go to Settings → Connected Services and look for Firecrawl.

  3. 3

    Hit "Connect"

    Click "Connect Firecrawl" and log in like you normally would. We only ask for the permissions your AI actually needs.

  4. 4

    Tell your AI what to do in Firecrawl

    Build a quick AI Agent (it's like writing instructions, not code), and pick the Firecrawl actions you want your AI to handle.

  5. 5

    Watch it run

    Send it a real-world task and watch your AI take care of Firecrawl for you. Tweak the wording until it feels just right.

Powered by the AI you already trust

Your Firecrawl workflow runs on the same world-class AI used by ChatGPT, Claude, and Gemini. We handle the technical part — you just tell your AI what to do, in plain English.

  • OpenAI
  • Anthropic Claude
  • Google Gemini

Questions other Firecrawl users had before signing up

Do I need to know how to code?

Not even a little. If you can use Firecrawl today, you can set this up. You'll connect Firecrawl, tell your AI what it should help with, and you're live. The whole thing is built for coaches, course creators, agencies, and operators — not engineers.

Will my clients know they're talking to an AI?

Only if you want them to. Your FormWise AI is trained on your voice, your offer, and the way you actually communicate — so it sounds like a thoughtful team member, not a clunky chatbot. Most people can't tell.

Can I make it sound like me?

Yes. You can give your AI examples of how you write, the questions you usually ask new clients, the way you handle objections, and the language you use about your offer. The more you give it, the more it sounds like you wrote it.

What if I want to take over a conversation?

You're always in control. You can step in at any point, take over from your AI, or stop it from sending messages until you've reviewed them. Think of it as a teammate, not a replacement.

Is my client data safe?

Yes. Your Firecrawl login and client information stay scoped to your workspace. We don't sell it, we don't share it, and we don't use your data to train anyone else's AI. You can disconnect Firecrawl or delete everything any time.

What happens when Firecrawl updates their app?

We handle it. When Firecrawl adds new features or changes how something works, we update the integration in the background. You don't have to do anything — your AI just keeps working.

How fast can I actually get this running?

Most people are up and running the same day they sign up. Connecting Firecrawl takes a few minutes. Teaching your AI to handle the work the way you'd want it done is the part worth slowing down for.

What is FormWise?

FormWise is the AI agent builder for coaches, course creators, and agencies. You connect the apps you already use — like Firecrawl — train your AI on your offer and voice, and let it handle the work you've been doing manually: onboarding, follow-ups, support, admin. No code, no developers, no engineering team required.

Can I sell AI agents to my clients?

Yes. A lot of FormWise users build AI assistants and sell them as part of their offer — done-for-you AI setups, white-label assistants for client businesses, custom GPTs trained on a client's brand. You can charge a setup fee, a monthly retainer, or both.

Can I embed my AI on my website or in my course?

Yes. Every AI you build in FormWise can be embedded on your site, dropped into your course platform, or shared as a standalone link. Your clients and students just open it like any other web page — no app to download, no login hoops.

Do I need to know anything about AI to use this?

No. If you can describe what you want your AI to do in plain English — "greet new clients, ask these 3 questions, book them a call" — FormWise can build it. We handle the models, the prompts, the connections. You handle the part you already know: what your business actually needs.

About Firecrawl on FormWise

Firecrawl automates web crawling and data extraction, enabling organizations to gather content, index sites, and gain insights from online sources at scale

Connect Firecrawl once and your FormWise AI gets 29 things it can do inside it. That means your AI can handle the Firecrawl work in your business — the messages, the updates, the follow-ups, the admin — the way you would, just faster and around the clock.

You don't need to know how any of it works under the hood. We take care of keeping the connection alive, retrying if something fails, and making sure your data only ever moves where you tell it to.

New to Firecrawl? Take a look at www.firecrawl.dev.

Stop doing the Firecrawl work yourself

Hand it off to an AI assistant that sounds like you, works around the clock, and never forgets a follow-up. Free to start.