micawber

URL extractor

A library for extracting metadata and content from URLs

a small library for extracting rich content from urls

GitHub
636 stars
18 watching
91 forks
Language: Python
last commit: about 1 month ago
Linked from 3 awesome lists

oembedpython

Screenshot of coleifer/micawber website

micawber.readthedocs.org/

Backlinks from these awesome lists:

Related projects:

Repository Description Stars
dwisiswant0/galer A tool to extract URLs from HTML attributes by crawling in and evaluating JavaScript 253
mvdan/xurls A tool to extract URLs from text using regular expressions in the Go programming language. 1,187
unapibageek/ctfr A tool that exploits Certificate Transparency logs to extract subdomains from HTTPS websites. 1,972
s0rg/crawley A utility for systematically extracting URLs from web pages and printing them to the console. 263
eloopwoo/chrome-url-dumper A tool to extract and dump URLs from Chrome's stored databases. 34
recrm/archivetools A collection of tools for extracting and analyzing data from web archives 69
aymericbeaumet/squeeze A tool to extract relevant information from text 17
itsmeichigo/urlpreview An NSURL extension to fetch and display basic webpage metadata 199
obsidianforensics/unfurl Extracts and visualizes data from URLs by breaking them down into components and extracting relevant information. 609
xnl-h4ck3r/xnlinkfinder A Python tool used to automatically discover and extract endpoints, parameters, and wordlists from target websites. 1,204
zachliugis/stock_extractor A package of classes to extract stock data from various online sources. 34
unkl4b/gitminer Automated tool for gathering code information from Github repositories 2,092
eset-la/lord-of-the-strings A tool to extract and classify relevant strings from binary files 9
semanticart/url_unroller A tool to expand shortened URLs into their original forms 5
felipecsl/wombat A Ruby-based web crawler and data extraction tool with an elegant DSL. 1,315