Recursive clone

[W-Lawless]

I realize this might be dangerous if someone haphazardly attempts WIkipedia or something, but it would be nice to have an -r flag to recursively follow all tags and clone subpages

[mesaglio]

This sounds more like scrapping, no?

[W-Lawless]

This sounds more like scrapping, no?

I apologize, yesterday when I was looking through here and searching github for a tool, I got very limited results for some reason, although now attempting the same search again there are over 17k repos for this. I swapped the scrape keyword for clone trying to find better results.

That being said, is there any repo you recommend for this in particular? I'd like to essentially clone/scrape a subreddit-style website and make a lot of the info there searchable / indexable. A lot of discussion on programming/opsec stuff that could be made much more actionable if I can search it in ways the exceed the standard sidebar search input.

Thanks!