colly
Website scraper
A framework for extracting structured data from websites in a fast and elegant way
Elegant Scraper and Crawler Framework for Golang
23k stars
335 watching
2k forks
Language: Go
last commit: 6 months ago
Linked from 3 awesome lists
crawlercrawlingframeworkgogolangscraperscrapingspider
avelino/awesome-go
brucedone/awesome-crawler
xwjdsh/awesome-go-extraRelated projects:
| Repository | Description | Stars |
|---|---|---|
 geziyor/geziyor | A fast and flexible web crawling and scraping framework for extracting structured data from websites. | 2,646 |
 jaeles-project/gospider | A tool for web crawling and exploitation written in Go. | 2,598 |
 unclecode/crawl4ai | A web crawling tool designed to extract structured data from the web for use in AI applications | 18,541 |
 gohugoio/hugo | A fast and flexible tool for generating static websites with built-in support for various content formats. | 76,514 |
 yujiosaka/headless-chrome-crawler | A distributed crawling framework that leverages Headless Chrome to scrape dynamic websites | 5,534 |
 apify/crawlee | A tool for building reliable web scraping and browser automation pipelines in Node.js. | 16,081 |
 hu17889/go_spider | A modular, concurrent web crawler framework written in Go. | 1,827 |
 hakluke/hakrawler | A tool for automatically discovering and crawling web application endpoints and assets | 4,528 |
 ionicabizau/scrape-it | A Node.js library and CLI tool for automating web page scraping and parsing | 4,024 |
 s0md3v/photon | A fast and flexible web crawler designed to gather information from the internet | 11,122 |
 needmorecowbell/giggity | A tool to scrape and store hierarchical data about GitHub organizations, users, or repositories. | 127 |
 golangci/golangci-lint | Automated code quality tool for Go programming language | 15,863 |
 puerkitobio/gocrawl | A concurrent web crawler written in Go that allows flexible and polite crawling of websites. | 2,036 |
 spatie/crawler | A powerful web crawler written in PHP that can execute JavaScript and crawl multiple URLs concurrently. | 2,552 |
 my8100/scrapydweb | A web application for managing Scrapyd clusters, analyzing Scrapy logs, and visualizing results. | 3,180 |