Top/web-crawler

The most inspiring discoveries in web crawler

github.com

Firecrawl MCP Server facilitates powerful web scraping and search capabilities for LLM clients, enhancing data collection and interaction.

batch-processingclaudecontent-extractiondata-collectionfirecrawl

wire

github.com

Crawlee is a Node.js library for web scraping and automation, enabling efficient data extraction and crawler management.

apifyautomationbrowser-automationcrawleecrawler

relay

github.com

Maxun is an open-source no-code platform for web scraping and data extraction, enabling users to convert websites into structured data easily.

agentsapiautomationbrowser-automationcrawler

relay

firecrawl.com

Firecrawl provides a powerful API for searching, scraping, and interacting with the web, enabling AI systems to access clean, structured data.

automationdata-extractionfirecrawlweb-crawler

relay

github.com

Scrapling is a Python framework for adaptive web scraping, enabling efficient data extraction with advanced features.

aiai-scrapingautomationcrawlercrawling

relay