QueryList

spatie/crawler2.6K

Crawler

A powerful web crawler written in PHP that can execute JavaScript and crawl multiple URLs concurrently.

feng19/spider_man23

Crawler library

A high-level web crawling and scraping framework for Elixir.

veliovgroup/spiderable-middleware39

crawler proxy

intercepts requests from web crawlers and proxies them to a prerendering service for rendering HTML

chenjiandongx/github-spider264

Crawler

A Python-based web crawler for scraping Github user and repository data.

zhuyingda/webster518

Web scraper

A framework for automating web scraping and crawling tasks using Node.js

apify/crawlee16.1K

Web scraper

A tool for building reliable web scraping and browser automation pipelines in Node.js.

benibela/xidel690

Web scraper

A tool to extract data from web pages using various query languages and selectors.

kiddyuchina/beanbun1.2K

crawler framework

A PHP framework for building distributed web crawlers with modular design and extensibility

technosophos/querypath823

XML/HTML parser library

A PHP library for manipulating XML and HTML documents, supporting various input formats and offering robust functionality through chaining.

joncanning/skyscraper59

Web scraper framework

A framework for building asynchronous web scrapers and crawlers using async/await and Reactive Extensions.

mvdbos/php-spider1.3K

Web Crawler

A flexible PHP web crawler with configurable traversal algorithms and filters.

howie6879/ruia1.8K

Crawler framework

An async web scraping micro-framework built with asyncio and aiohttp to simplify URL crawling

joseconstela/webparsy44

Website scraper

A Node.js library and CLI for scraping websites using Puppeteer and YAML definitions

xianhu/pspider1.8K

Web Crawler Framework

A Python web crawler framework with support for multi-threading and proxy usage.

hu17889/go_spider1.8K

Crawler

A modular, concurrent web crawler framework written in Go.