Archiveteam/grab-site
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
486 Commits 1 Branch 0 Tags
Commit Graph

194 Commits

Author SHA1 Message Date
Ivan Kozik
4bacfdc601 Ignore /navbar.g because wpull doesn't decode the URL properly 2015-07-29 07:33:48 +00:00
Ivan Kozik
1b44b59a47 Ignore addtoany.com/share_save 2015-07-29 07:33:48 +00:00
Ivan Kozik
f4423bde70 Ignore localhost 2015-07-29 07:33:48 +00:00
Ivan Kozik
23344a0d8c Ignore https as well 2015-07-29 07:33:48 +00:00
Ivan Kozik
175b24d789 Ignore frequently-encountered wikipedia thumbnails 2015-07-29 07:33:48 +00:00
Ivan Kozik
d5ec636cba Ignore pages on draft.blogger.com 2015-07-29 07:33:48 +00:00
Ivan Kozik
0bf569d29a Ignore some reddit wiki pages 2015-07-29 07:33:48 +00:00
Ivan Kozik
37f8aafa59 Ignore more radioscoop 2015-07-29 07:33:48 +00:00
Ivan Kozik
580a204b03 Ignore more js-agent.newrelic.com 2015-07-29 07:33:48 +00:00
Ivan Kozik
68ffb8932f Ignore another Icecast site 2015-07-29 07:33:48 +00:00
Ivan Kozik
85e557f44a Ignore &mobileaction= 2015-07-29 07:33:48 +00:00
Ivan Kozik
d1efcf1fbd Remove moved rule 2015-07-29 07:33:48 +00:00
Ivan Kozik
c0e443476c Copy tumblr rule from blogs set 2015-07-29 07:33:48 +00:00
Ivan Kozik
e1452e6e5b Ignore more & 2015-07-29 07:33:48 +00:00
Ivan Kozik
399ca7eaa7 Ignore Special:ListFiles.*&user= 2015-07-29 07:33:48 +00:00
Ivan Kozik
a34f824965 Ignore some Special:ListFiles 
Note: & args in URL like

https://wiki.unrealengine.com/index.php?title=Special:ListFiles&dir=prev&sort=img_size&limit=50&user=ListFiles/ListFiles/Skins.vector.js

seem to be ignored, not treated as &
 2015-07-29 07:33:48 +00:00
Ivan Kozik
5f46491a17 Ignore stumbleupon without www. as well 2015-07-29 07:33:48 +00:00
Ivan Kozik
2d3d04b790 Ignore per-section edit pages 2015-07-29 07:33:48 +00:00
Ivan Kozik
88c69effc2 Ignore Special:RecentChanges&from= 2015-07-29 07:33:48 +00:00
Ivan Kozik
c9bcacbeb6 Ignore Special:RecentChangesLinked 2015-07-29 07:33:48 +00:00
Ivan Kozik
2d206de1f5 Ignore a SHOUTcast site 2015-07-29 07:33:48 +00:00
Ivan Kozik
60e89778b3 Ignore www as well 2015-07-29 07:33:48 +00:00
Ivan Kozik
14f8f6aab9 Fix literal . 2015-07-29 07:33:48 +00:00
Ivan Kozik
6ca1e37b85 Ignore another Icecast site 2015-07-29 07:33:48 +00:00
Ivan Kozik
0cb5b0c1c6 Ignore another Icecast site 2015-07-29 07:33:48 +00:00
Ivan Kozik
42bc68ce2e Ignore Special:MobileOptions and Special:MobileFeedback 2015-07-29 07:33:48 +00:00
Ivan Kozik
973b2731ee Ignore TMZ videos 
These have popped up as page requisites twice now
 2015-07-29 07:33:48 +00:00
Ivan Kozik
e647876e25 Ignore another Icecast site 2015-07-29 07:33:48 +00:00
Ivan Kozik
af60d35fe9 Ignore delicious.com/save 2015-07-29 07:33:48 +00:00
Ivan Kozik
83c0718adc Ignore add.my.yahoo.com/rss 2015-07-29 07:33:47 +00:00
Ivan Kozik
8acffe9b66 Ignore local/private URLs that have a :port 2015-07-29 07:33:47 +00:00
Ivan Kozik
3f0f6f169d Ignore local and private IP ranges 2015-07-29 07:33:47 +00:00
Ivan Kozik
a424eede41 Ignore dead host 2015-07-29 07:33:47 +00:00
Ivan Kozik
f3a0c21813 Ignore infinite /styles/ and /scripts/ loops 2015-07-29 07:33:47 +00:00
Ivan Kozik
b3ad482143 Ignore Google Analytics tracking image 2015-07-29 07:33:47 +00:00
Ivan Kozik
5871f92710 Ignore /privmsg.php 2015-07-29 07:33:47 +00:00
Ivan Kozik
f885656cd0 Add nogravatar ignore set 2015-07-29 07:33:47 +00:00
Ivan Kozik
5523b6fe53 Ignore dead ad/tracking site 2015-07-29 07:33:47 +00:00
Ivan Kozik
1b29d0dde5 Ignore runaway %25252525 2015-07-29 07:33:47 +00:00
Ivan Kozik
ef60f59a03 Ignore blogger.com/comment-iframe.g 2015-07-29 07:33:47 +00:00
Ivan Kozik
7b2244388e Ignore loop on photobucket.com 2015-07-29 07:33:47 +00:00
Ivan Kozik
dd124c90f5 Ignore blogger.com/email-post.g 2015-07-29 07:33:47 +00:00
Ivan Kozik
e7b67e2b60 Ignore web-beta.archive.org 2015-07-29 07:33:47 +00:00
Ivan Kozik
aefd9ad866 Ignore another facebook infinite loop 
https://www.facebook.com/connect.php/js/FB.SharePro/rsrc.php/v2/yo/r/rsrc.php/v2/yY/r/rsrc.php/v2/yo/r/rsrc.php/v2/yY/r/rsrc.php/v2/yo/r/rsrc.php/v2/yY/r/rsrc.php/v2/yY/r/h_Y6u1wrZPW.png
 2015-07-29 07:33:47 +00:00
Ivan Kozik
1594f5375b Ignore more khaleejtimes.com loops 2015-07-29 07:33:47 +00:00
Ivan Kozik
11fc98a472 Ignore infinite loops on khaleejtimes.com 2015-07-29 07:33:47 +00:00
Ivan Kozik
cbc884233a Ignore &hidepatrolled= 2015-07-29 07:33:47 +00:00
Ivan Kozik
f8419014c0 Ignore some bxSlider loops 2015-07-29 07:33:47 +00:00
Ivan Kozik
00085f6162 Ignore some bad joined URLs 
These don't point to real ad trackers, they are joined URL nonsense
 2015-07-29 07:33:47 +00:00
Ivan Kozik
3da36a6a1f Ignore another default gravatar 2015-07-29 07:33:47 +00:00