Archiveteam
/
grab-site
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Copy in latest dupespotter

master
Ivan Kozik 2015-03-14 20:36:16 +00:00
parent 040afe0d92
commit 8bf1b00c46
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
dupespotter.py
View File

@ -144,7 +144,7 @@ def process_body(body, url):
body = re.sub(br'[&\?]((v|cb)=\d+|pos=[A-Za-z0-9=]+)', b"", body)
# spotted on espn.go.com and others
body = re.sub(br'(splinks-|var hash = .)\d+', b"", body)
body = re.sub(br'(splinks-|var hash = .|":"?)-?\d+', b"", body)
# Kill newrelic inline script
body = re.sub(br'window\.NREUM\|\|\(NREUM=\{\}\);NREUM\.info=\{.{1,3000}?\}', b"", body)