elliotgao2's tracked open-source repos, sorted by stars.
Every web site provides APIs.
Web crawling framework based on asyncio.
Convert HTML to Markdown.