QueryList
Web Scraper Framework
A PHP framework for building web scrapers and crawlers with a focus on ease of use and extensibility.
The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
3k stars
74 watching
443 forks
Language: PHP
last commit: 11 months ago
Linked from 1 awesome list
crawlerquerylistscraperspider
brucedone/awesome-crawlerRelated projects:
| Repository | Description | Stars |
|---|---|---|
 spatie/crawler | A powerful web crawler written in PHP that can execute JavaScript and crawl multiple URLs concurrently. | 2,552 |
 feng19/spider_man | A high-level web crawling and scraping framework for Elixir. | 23 |
 veliovgroup/spiderable-middleware | intercepts requests from web crawlers and proxies them to a prerendering service for rendering HTML | 39 |
 chenjiandongx/github-spider | A Python-based web crawler for scraping Github user and repository data. | 264 |
 zhuyingda/webster | A framework for automating web scraping and crawling tasks using Node.js | 518 |
 apify/crawlee | A tool for building reliable web scraping and browser automation pipelines in Node.js. | 16,081 |
 benibela/xidel | A tool to extract data from web pages using various query languages and selectors. | 690 |
 kiddyuchina/beanbun | A PHP framework for building distributed web crawlers with modular design and extensibility | 1,249 |
 technosophos/querypath | A PHP library for manipulating XML and HTML documents, supporting various input formats and offering robust functionality through chaining. | 823 |
 joncanning/skyscraper | A framework for building asynchronous web scrapers and crawlers using async/await and Reactive Extensions. | 59 |
 mvdbos/php-spider | A flexible PHP web crawler with configurable traversal algorithms and filters. | 1,336 |
 howie6879/ruia | An async web scraping micro-framework built with asyncio and aiohttp to simplify URL crawling | 1,753 |
 joseconstela/webparsy | A Node.js library and CLI for scraping websites using Puppeteer and YAML definitions | 44 |
 xianhu/pspider | A Python web crawler framework with support for multi-threading and proxy usage. | 1,828 |
 hu17889/go_spider | A modular, concurrent web crawler framework written in Go. | 1,827 |