Autonomous Web Scraping & Research for AI Coding Agents

{
  "mcpServers": {
    "flycrawl": {
      "command": "npx",
      "args": ["-y", "flycrawl-mcp"],
      "env": {
        "FLYCRAWL_API_KEY": "fly_live_your_api_key_here"
      }
    }
  }
}

Single-Page Scrape & Fit-Markdown

Fetches any URL, handles JavaScript rendering, strips cookie and navigation noise, and returns clean Markdown with preserved tables and headers.

Live Web Search + Extraction

Searches the public web for developer queries and returns relevant pages pre-extracted into clean Markdown in one single action.

Domain Mapping & Sitemap Crawl

Discovers and returns all linked documentation endpoints and sub-paths from a domain in under 2 seconds.

High-Res Logo & SVG Extractor

Extracts authoritative SVG brand marks, dark/light variants, and Apple Touch Icons directly from any company domain.

Multilingual PDF Extraction

Parses complex whitepapers and catalogs, including bidirectional Persian/Arabic and Asian CJK text without corruption.

Recursive Website Crawling

Recursively traverses internal links across documentation or catalogs with strict depth and page boundaries. 1 credit per crawled page (0 credits if failed).

LLM Structured Schema Extraction

Transforms dynamic web pages into typed JSON conforming to a JSON Schema or natural language instructions. Read-only, idempotent. 0 credits charged on failure.

Direct Vector Embedding

Generates 1536-dim or 3072-dim embeddings on the fly, enabling agents to compute semantic cosine distance immediately.