Beanbun
crawler framework
A PHP framework for building distributed web crawlers with modular design and extensibility
Beanbun 是用 PHP 编写的多进程网络爬虫框架，具有良好的开放性、高可扩展性，基于 Workerman。
1k stars
77 watching
251 forks
Language: PHP
last commit: over 1 year ago
Linked from 1 awesome list
beanbuncrawlerphpspider
jingwentian/awesome-phpRelated projects:
| Repository | Description | Stars |
|---|---|---|
 codesofun/web-bee | A Java framework for building web-based crawlers with features like distributed crawling and proxy support. | 189 |
 zhegexiaohuozi/seimicrawler | An agile and distributed crawler framework designed to simplify and speed up web scraping with Spring Boot support | 1,980 |
 hu17889/go_spider | A modular, concurrent web crawler framework written in Go. | 1,826 |
 xianhu/pspider | A Python web crawler framework with support for multi-threading and proxy usage. | 1,827 |
 qinxuye/cola | A high-level framework for building distributed data extractors from web pages | 1,500 |
 chenjiandongx/github-spider | A Python-based web crawler for scraping Github user and repository data. | 264 |
 feng19/spider_man | A high-level web crawling and scraping framework for Elixir. | 23 |
 crawlzone/crawlzone | A PHP framework for asynchronous internet crawling and web scraping | 77 |
 brendonboshell/supercrawler | A web crawler designed to crawl websites while obeying robots.txt rules, rate limits and concurrency limits, with customizable content handlers for parsing and processing crawled pages. | 378 |
 puerkitobio/fetchbot | A flexible web crawler that follows robots.txt policies and crawl delays. | 786 |
 antchfx/antch | A framework for building fast and efficient web crawlers and scrapers in Go. | 260 |
 wspl/creeper | A framework for building cross-platform web crawlers using Go | 780 |
 joncanning/skyscraper | A framework for building asynchronous web scrapers and crawlers using async/await and Reactive Extensions. | 58 |
 howie6879/ruia | An async web scraping micro-framework built with asyncio and aiohttp to simplify URL crawling | 1,752 |
 bixuehujin/blink | A high-performance web framework and application server built on top of PHP. | 832 |