Topic · crawling

crawling

Tracked open-source repos tagged crawling, sorted by stars.

39 repos

#31
easy-scraping-tutorial ↗@MorvanZhouJupyter Notebook
Simple but useful Python web scraping tutorial code.
★ 820+0Star change over the last 7 days
#32
seonaut ↗@StJudeWasHereGo
Open source SEO audit tool.
★ 780+7Star change over the last 7 days
#33
linkedin-profile-scraper-api ↗@josephlimtechTypeScript
🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON.
★ 776+0Star change over the last 7 days
#34
deepcrawl ↗@lumpinifTypeScript
100% free and full open-source edge Firecrawl alternative with better links extraction for agents - that you can deploy to cloudflare or vercel by yourself.
★ 668+6Star change over the last 7 days
#35
Craw4LLM ↗@cxcscmuPython
Official repository for "Craw4LLM: Efficient Web Crawling for LLM Pretraining"
★ 664+1Star change over the last 7 days
#36
donsetch ↗@dondai44423Rust
Web fetch, search, and crawl for AI agents. Built from scratch in Rust. No keys, no accounts. AGPL v3.
★ 624—Star change over the last 7 days
#37
LinkedInDumper ↗@l4rm4ndPython
Python 3 script to dump/scrape/extract company employees from LinkedIn API
★ 611+1Star change over the last 7 days
#38
spidermon ↗@scrapinghubPython
Scrapy Extension for monitoring spiders execution.
★ 562+0Star change over the last 7 days
#39
webster ↗@zhuyingdaJavaScript
a reliable high-level web crawling & scraping framework for Node.js.
★ 560+1Star change over the last 7 days