gpt-crawler
Content scraper
Automates the process of generating knowledge files to create custom AI models from website content
Crawl a site to generate knowledge files to create your own custom GPT from a URL
19k stars
124 watching
2k forks
Language: TypeScript
last commit: 7 months ago ai
Related projects:
| Repository | Description | Stars |
|---|---|---|
 gpt-engineer-org/gpt-engineer | An AI-powered platform to experiment with software engineering tasks using natural language input. | 52,634 |
 apify/crawlee | A tool for building reliable web scraping and browser automation pipelines in Node.js. | 16,081 |
 gitbookio/gitbook | A Next.js based web application for managing and hosting documentation sites using Markdown format | 27,297 |
 spatie/crawler | A powerful web crawler written in PHP that can execute JavaScript and crawl multiple URLs concurrently. | 2,552 |
 code4craft/webmagic | A framework for building scalable web crawlers in Java. | 11,456 |
 whoiskatrin/chart-gpt | An AI tool to generate charts from text input | 3,559 |
 ricklamers/gpt-code-ui | An interactive code generation and execution tool using AI models | 3,567 |
 yasserg/crawler4j | A Java-based web crawler for extracting and processing web page content | 4,563 |
 git-bug/git-bug | A distributed, offline-first bug tracker embedded in git that allows collaborative development without vendor lock-in. | 8,165 |
| builderio/figma-html | A tool for converting Figma designs into live webpages and code, supporting various frameworks and languages. | 3,207 |
| builderio/builder | Enables developers to visually create and generate code for various frontend frameworks | 7,645 |
 gitextensions/gitextensions | A standalone UI tool for managing Git repositories, integrating with Windows Explorer and Visual Studio. | 7,823 |
 jitpack/jitpack.io | Provides a package repository and build service for JVM and Android projects | 2,544 |
 yujiosaka/headless-chrome-crawler | A distributed crawling framework that leverages Headless Chrome to scrape dynamic websites | 5,534 |
 unclecode/crawl4ai | A web crawling tool designed to extract structured data from the web for use in AI applications | 18,541 |