walker
Data Walker
An API-paginated data walker that fetches and processes data concurrently while respecting rate limits.
Seamlessly fetch paginated data from any source. Simple and high performance API scraping included!
11 stars
2 watching
0 forks
Language: Go
last commit: almost 2 years ago
Linked from 1 awesome list
api-scraperindexerpaginationpaginatorscraper
avelino/awesome-goRelated projects:
| Repository | Description | Stars |
|---|---|---|
 rekby/objwalker | A tool for recursively traversing and manipulating data structures using reflection in Go. | 3 |
 waylonwalker/waylonwalker | A personal blog and repository of data engineering projects and knowledge shared by the author. | 209 |
 yurigor/deepdash | A tree traversal library for manipulating nested data structures in JavaScript. | 275 |
 bobg/htree | A package for traversing and processing HTML node trees in Go. | 4 |
 cyruzin/tome | A pagination utility for simple RESTful APIs. | 35 |
 scwilkinson/pd-replicator | A tool that allows users to easily copy data from a pandas DataFrame to the clipboard with one click. | 11 |
 dixonwille/skywalker | A tool for concurrently exploring file systems with filtering capabilities | 101 |
 ulule/deepcopier | A utility for creating deep copies of complex data structures in Go. | 452 |
 lukeed/klona | A utility library for creating deep copies of complex data structures and objects | 1,620 |
| dixonwille/wlog | A simple logging interface with cross-platform color support and concurrency features. | 66 |
 hiveeyes/terkin-datalogger | A flexible data logger platform with sensor and networking capabilities | 60 |
 dependents/node-source-walk | A library that allows traversing and manipulating JavaScript ASTs | 57 |
 bplawler/crawler | A Scala-based DSL for programmatically accessing and interacting with web pages | 148 |
 walkable-server/walkable | A Clojure library for building APIs with a declarative, data-driven approach to query syntax and configuration. | 445 |
 hugocarreira/go-decent-copy | A tool that provides a simple file copying mechanism in Go | 22 |