22 Commits

Author SHA1 Message Date
Ivan Kozik
c28c593a83 Explain imdb ignore set 2015-08-21 08:35:57 +00:00
Ivan Kozik
524cdf2cec Fix blogspot search? ignore 2015-08-21 05:35:30 +00:00
Ivan Kozik
f379264ed1 Don't crash on --igsets=blogs even though it's gone 2015-08-21 05:31:51 +00:00
Ivan Kozik
6a6dff0083 Add comment to reddit ignore set 2015-08-21 05:18:35 +00:00
Ivan Kozik
4b50c6de67 Migrate all other ignores from blogs to the global set 2015-08-21 05:16:26 +00:00
Ivan Kozik
954ab31acb Remove ignores that probably only wget needed 2015-08-21 04:48:47 +00:00
Ivan Kozik
b83bce1f2a Migrate some ignores from blogs to global set 2015-08-21 04:48:19 +00:00
Ivan Kozik
1b8b4b0077 Ignore per-post and per-comment Atom feeds on blogspot.com 2015-08-21 04:23:54 +00:00
Ivan Kozik
b8b6248aab Remove no-longer-needed workaround in ignore sets 2015-08-12 07:55:08 +00:00
Ivan Kozik
dfd1e8cd47 singletumblr igset: explain 2015-08-10 11:51:33 +00:00
Ivan Kozik
1cb9331939 nosortedindex igset: add comment 2015-08-10 11:49:45 +00:00
Ivan Kozik
33cc3040ed mediawiki igset: add comments 2015-08-10 11:48:53 +00:00
Ivan Kozik
40cae40dc5 blogs igset: comment more 2015-08-10 11:46:32 +00:00
Ivan Kozik
4e517e2994 blogs igset: remove ignores that are already covered by 'global' 2015-08-10 11:45:28 +00:00
Ivan Kozik
4d570d88bd Add some comments to 'blogs' ignore set 2015-08-10 11:44:20 +00:00
Ivan Kozik
6f03c5137d Move pixel.redditmedia.com from reddit to global ignore set 2015-08-10 11:42:03 +00:00
Ivan Kozik
e304c60586 Describe why various ignores are in the 'global' ignore set; add support for comments in ignore sets 2015-08-10 11:41:16 +00:00
Ivan Kozik
22835a5ddc igsets: global: don't exclude archive.org (that ignore made sense for ArchiveBot, which sent WARCs to IA) 2015-07-29 08:24:42 +00:00
Ivan Kozik
51d3b1f794 igsets: rm internetcentrum - it is long gone 2015-07-29 07:45:58 +00:00
Ivan Kozik
5276fec1a9 Convert JSON ignore sets to plain text to avoid the backslash doubling 2015-07-29 07:44:12 +00:00
Ivan Kozik
68f5fc0dd2 igsets: noonion: fix backslash 2015-07-29 07:40:20 +00:00
Ivan Kozik
e53f4465e5 db/ignore_patterns -> libgrabsite/ignore_sets 2015-07-29 07:37:19 +00:00