node-crawler
by bda-research
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
AI summary
Web scraper
A NodeJS-based web crawler and spider that extracts data from websites.
- stars
- 6.7K
- forks
- 876
- watching
- 255
- awesome lists
- 2
Similar projects
Found by comparing what the projects do, not just their names.
apify/crawlee16.1K
Web scraper
A tool for building reliable web scraping and browser automation pipelines in Node.js.
Crawler
A distributed crawling framework that leverages Headless Chrome to scrape dynamic websites
Web scraper
A versatile web scraping module with two scrapers for static and dynamic content extraction.
Web scraper
A fast and flexible web scraping library using native libxml C bindings
npm/cli8.6K
Package manager
A package manager for JavaScript that enables users to manage and install dependencies for web applications.
sindresorhus/got14.4K
HTTP client library
A powerful HTTP client library for Node.js that provides a human-friendly and flexible way to make requests.
axios/axios106K
HTTP client library
An HTTP client library for making requests to web servers using the Promise API.
crawler proxy
intercepts requests from web crawlers and proxies them to a prerendering service for rendering HTML
unclecode/crawl4ai18.5K
Web crawler
A web crawling tool designed to extract structured data from the web for use in AI applications
macbre/phantomas2.3K
Performance monitor
A tool for collecting and monitoring web performance metrics in a headless Chromium browser environment.
multipart parser
A module for parsing multipart form data, especially file uploads in Node.js applications.
spatie/crawler2.6K
Crawler
A powerful web crawler written in PHP that can execute JavaScript and crawl multiple URLs concurrently.
code4craft/webmagic11.5K
crawler
A framework for building scalable web crawlers in Java.
Web scraper
A flexible web scraping framework for extracting data from websites with customizable selectors and pagination support.
sjdirect/abot2.3K
Web crawler
A C# web crawler framework built for speed and flexibility, allowing developers to easily crawl websites with customizable logic.
Featured in 2 awesome lists
Each link jumps to the spot where the list mentions node-crawler.
