Ivan Kozik
7200878118
extra_docs/custom_hooks_sample.py: add a hook that queues additional URLs
2017-10-25 20:53:51 +00:00
Ivan Kozik
2b56a73aaa
dashboard: adjust code formatting
2017-10-24 18:15:44 +00:00
Ivan Kozik
87e4bd79a6
dashboard: enable context menu for all browsers (Safari 10+ has document.execCommand('copy').)
2017-10-24 18:13:48 +00:00
Ivan Kozik
d9f75f5ae3
README: update "Install on a non-Ubuntu distribution" steps to also use a virtualenv
2017-10-24 17:55:48 +00:00
Ivan Kozik
ad5c4d2449
README: OS X -> macOS and update instructions to use virtualenv
2017-10-24 17:42:33 +00:00
Ivan Kozik
d5698bc08a
README: fix TOC order
2017-10-24 17:25:33 +00:00
Ivan Kozik
112a3175c2
Bump version to 1.3.0
2017-10-24 17:24:57 +00:00
Ivan Kozik
d9b89f551b
README: rework instructions to not require activating the virtualenv
2017-10-24 17:24:27 +00:00
Ivan Kozik
be5db3f397
README: rework the Ubuntu 14.04 install steps to use virtualenv; assume grab-site and related executables are in PATH
2017-10-24 17:14:57 +00:00
Ivan Kozik
0ad6bdf89f
README: ancient non-LTS Ubuntu releases are not supported
2017-10-24 17:02:33 +00:00
Ivan Kozik
a954a0caca
README: "Python 3.5 or newer"
2017-10-24 16:45:40 +00:00
Ivan Kozik
cd3931b5fc
Add install instructions for Windows 10
2017-10-24 16:43:53 +00:00
Ivan Kozik
96e1f229dc
dashboard: adjust the font stacks; add Segoe UI for Windows
2017-10-24 16:31:23 +00:00
Ivan Kozik
6680cf7e50
README: add install steps for Ubuntu 17.10
2017-10-24 01:02:42 +00:00
Ivan Kozik
eefb6a3eba
global igset: ignore another unwanted medium.com URL
2017-09-30 13:36:04 +00:00
Ivan Kozik
0c2f160db6
global igset: ignore unwanted medium.com URLs
2017-09-30 13:29:52 +00:00
Ivan Kozik
a941fdfb9c
global igset: ignore more incorrectly extracted links on YouTube
...
e.g.
404 Not Found https://www.youtube.com/{{data}}
2017-09-22 06:05:02 +00:00
Ivan Kozik
7863f344a7
global igset: ignore incorrectly extracted YouTube links
...
e.g.
404 Not Found https://www.youtube.com/[[data.videoNavigationEndpoint]]
404 Not Found https://www.youtube.com/[[menuRendererData]]
404 Not Found https://www.youtube.com/[[videoReportActionResultRenderer_]]
404 Not Found https://www.youtube.com/[[speedyGData_.videoQualityPromoRenderer]]
2017-09-22 06:02:55 +00:00
Ivan Kozik
a6d5bd0227
global igset: also handle http://finance.google.com/finance
2017-09-18 01:02:41 +00:00
Ivan Kozik
e43bdcbf24
Bump Firefox UA
2017-08-29 19:03:16 +00:00
Ivan Kozik
f1b3501505
global igset: ignore another never-ending video stream
2017-08-29 19:02:09 +00:00
Ivan Kozik
a72272d5c3
Bump Firefox UA
2017-06-26 18:17:01 +00:00
Ivan Kozik
de32a1bfe9
Bump version
2017-06-26 18:14:05 +00:00
Ivan Kozik
0c43dc11f3
dashboard: opt out of DNS prefetching to avoid making DNS lookups on every host
...
Before this fix, if "Use a prediction service to load pages more quickly" was
enabled in Chrome, it would make DNS lookups on the hostname in every URL that
flew by in the dashboard.
https://www.chromium.org/developers/design-documents/dns-prefetching
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-DNS-Prefetch-Control
2017-06-26 18:12:06 +00:00
Ivan Kozik
2d7222cc5f
Remove completely ineffective protection against crawling sites on localhost
...
Any hostname can resolve to 127.0.0.1, 192.168.x.y, etc.
If you care about this protection, run grab-site in a container or use
iptables/ferm to block outbound traffic on loopback for the user that runs
grab-site.
2017-04-27 16:32:34 +00:00
Ivan Kozik
25a19d1dc3
Update install instructions for Ubuntu 17.04 and fold Ubuntu 16.10 instructions into 16.04 instructions
2017-04-09 08:18:59 +00:00
Ivan Kozik
ae400137d3
README: update Help section
2017-03-09 07:58:28 +00:00
Ivan Kozik
69d1dab393
Mention grab-site 'URL' instead of grab-site URL to avoid issues with ? or &
2017-02-26 21:42:53 +00:00
Ivan Kozik
d88dccac27
Fix link to Python installer for OS X (there is no 3.4.5 installer)
2017-02-17 23:33:11 +00:00
Ivan Kozik
bd3c896146
Fix .travis.yml
2017-02-08 20:45:21 +00:00
Ivan Kozik
1f2d915fef
Rename a metavar
2017-02-08 20:25:40 +00:00
Ivan Kozik
57e3455189
Bump Firefox UA
2017-02-08 20:24:07 +00:00
Ivan Kozik
94e486c7cf
Document --permanent-error-status-codes
2017-02-08 20:23:12 +00:00
Ivan Kozik
bf6382d724
Add --permanent-error-status-codes argument
...
https://github.com/ludios/grab-site/issues/97
2017-02-08 20:16:31 +00:00
Ivan Kozik
4fd740e815
Point to Python 3.4.5 instead of 3.4.3
2017-02-04 13:39:46 +00:00
Ivan Kozik
32544d096e
Add install instructions for Ubuntu 16.10
2017-02-04 13:36:47 +00:00
Ivan Kozik
75452363d0
README: Tweak
2016-11-12 22:42:15 +00:00
Ivan Kozik
ec5cc3f287
Opt out of Chrome's misbehaving Scroll Anchoring
2016-10-10 13:10:41 +00:00
Ivan Kozik
c30e92ee02
Bump Firefox UA
2016-10-07 08:57:51 +00:00
Ivan Kozik
76e173f1c9
chmod +x gs-dump-urls
2016-09-04 02:50:08 +00:00
Ivan Kozik
957c8f8aee
chmod +x pause_resume_grab_sites.sh
2016-09-04 02:49:27 +00:00
Ivan Kozik
90d747d9bc
global igset: Add getpocket.com/save
2016-08-17 20:03:34 +00:00
Ivan Kozik
9e360a8a13
global igset: Ignore another loop
2016-08-11 15:59:53 +00:00
Ivan Kozik
2136bae9be
Bump version
2016-08-04 22:30:53 +00:00
Ivan Kozik
8a9a0b0d9f
Revert "dashboard: Tweak font stack and size"
...
This reverts commit 3f31e251784fa4986751ce5613ef4884fe7656ec.
2016-08-04 22:29:33 +00:00
Ivan Kozik
4ef946300f
Revert "dashboard: Improve alignment when using a font with variable-width numbers like San Francisco"
...
This reverts commit 154e99349ca4233d5233582c806719fa0564e1e8.
2016-08-04 22:29:23 +00:00
Ivan Kozik
154e99349c
dashboard: Improve alignment when using a font with variable-width numbers like San Francisco
2016-08-04 19:26:09 +00:00
Ivan Kozik
3f31e25178
dashboard: Tweak font stack and size
2016-08-04 16:37:09 +00:00
Ivan Kozik
19738bf603
global igset: remove some icecast stations that are covered by the icecast skipper in wpull_hooks
2016-08-03 11:06:05 +00:00
Ivan Kozik
ecf9b2e717
.travis.yml: Change URL to try to get an exit code of 0
2016-08-03 10:30:10 +00:00
