263 Commits

Author SHA1 Message Date
Ivan Kozik
ccaee25497 Link to global ignore set 2015-02-05 19:32:47 +00:00
Ivan Kozik
e2118bbea4 Clarify 2015-02-05 19:31:50 +00:00
Ivan Kozik
4a22b4d593 Tell user to install git as well 2015-02-05 19:27:19 +00:00
Ivan Kozik
65e096a035 Support --ignore-sets= instead of the space-separated version 2015-02-05 06:05:54 +00:00
Ivan Kozik
2d7125951f Link to pythex 2015-02-05 05:39:44 +00:00
Ivan Kozik
f815920a83 Document file formats 2015-02-05 05:37:34 +00:00
Ivan Kozik
d73ee5ba27 Make it real obvious 2015-02-05 05:34:49 +00:00
Ivan Kozik
0699689a14 Add igoff feature 2015-02-05 05:19:34 +00:00
Ivan Kozik
2ccb8b4d6f Add support for --no-offsite-links 2015-02-05 05:15:46 +00:00
Ivan Kozik
979b843458 Load changes from DIR/ignores and DIR/ignore_sets while the crawl is running 2015-02-05 04:59:28 +00:00
Ivan Kozik
429b2032ff Improve README 2015-02-05 04:27:38 +00:00
Ivan Kozik
1705174fb2 CRLF -> LF 2015-02-05 04:25:49 +00:00
Ivan Kozik
91fd89be5d Add a site-grabber based on ArchiveBot's use of wpull 2015-02-05 03:43:50 +00:00