walker

Data Walker

An API-paginated data walker that fetches and processes data concurrently while respecting rate limits.

Seamlessly fetch paginated data from any source. Simple and high performance API scraping included!

GitHub

11 stars
2 watching
0 forks
Language: Go
last commit: over 3 years ago
Linked from 1 awesome list

api-scraperindexerpaginationpaginatorscraper

Backlinks from these awesome lists:

Related projects:

RepositoryDescriptionStars
rekby/objwalkerA tool for recursively traversing and manipulating data structures using reflection in Go.3
waylonwalker/waylonwalkerA personal blog and repository of data engineering projects and knowledge shared by the author.208
yurigor/deepdashA tree traversal library for manipulating nested data structures in JavaScript.276
bobg/htreeA package for traversing and processing HTML node trees in Go.4
cyruzin/tomeA pagination utility for simple RESTful APIs.35
scwilkinson/pd-replicatorA tool that allows users to easily copy data from a pandas DataFrame to the clipboard with one click.11
dixonwille/skywalkerA tool for concurrently exploring file systems with filtering capabilities102
ulule/deepcopierA utility for creating deep copies of complex data structures in Go.452
lukeed/klonaA utility library for creating deep copies of complex data structures and objects1,635
dixonwille/wlogA simple logging interface with cross-platform color support and concurrency features.66
hiveeyes/terkin-dataloggerA flexible data logger platform with sensor and networking capabilities60
dependents/node-source-walkA library that allows traversing and manipulating JavaScript ASTs57
bplawler/crawlerA Scala-based DSL for programmatically accessing and interacting with web pages149
walkable-server/walkableA Clojure library for building APIs with a declarative, data-driven approach to query syntax and configuration.445
hugocarreira/go-decent-copyA tool that provides a simple file copying mechanism in Go22