ArchiveTeam/grab-site

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

↗ Open on GitHub

Stars

1,609

Forks

156

Language

Python

License

NOASSERTION

Last push

1 year ago

Python crawler spider archiving crawl warc

No related intel yet

This repo has not appeared in any of the sources the radar tracks. The collector runs on a schedule — check back once it covers this repo.