Ivan Kozik
|
dbed789ff9
|
README: Tweak
|
2016-02-08 20:15:51 +00:00 |
|
Ivan Kozik
|
50ff7759ba
|
README: Automatically pausing grab-site processes when free disk space runs low
|
2016-02-08 20:09:32 +00:00 |
|
Ivan Kozik
|
7c1afbefe0
|
Use wpull 1.2.3
|
2016-02-05 19:15:25 +00:00 |
|
Ivan Kozik
|
ef5137ae86
|
Update UA
|
2016-02-01 16:19:49 +00:00 |
|
Ivan Kozik
|
1e06fa0a6e
|
.travis.yml: a grab-site of github.com/ludios/grab-site is returning exit code 8 because img.shields.io is down, so crawl google.com instead
|
2016-01-31 13:47:51 +00:00 |
|
Ivan Kozik
|
7ec2f90534
|
global igset: Ignore /CSI/CSI/ loops on blogspot
|
2016-01-12 22:44:57 +00:00 |
|
Ivan Kozik
|
0214558d5e
|
global igset: ignore bogus /search/label/CSI/ links on blogspot
|
2016-01-12 03:20:44 +00:00 |
|
Ivan Kozik
|
192553d6b9
|
Mention that the defaults work for Discourse forums
|
2016-01-11 04:51:23 +00:00 |
|
Ivan Kozik
|
4315cf1831
|
Document how to grab GitHub issues and PRs
|
2016-01-09 22:00:42 +00:00 |
|
Ivan Kozik
|
3b9f8c1a4c
|
global igset: ignore /CaptchaImage.axd
|
2016-01-09 21:46:35 +00:00 |
|
Ivan Kozik
|
dff87eba2f
|
global igset: also ignore www.digg.com/submit
|
2016-01-09 02:19:18 +00:00 |
|
Ivan Kozik
|
3f295d7ecb
|
README: Tweak
|
2016-01-07 07:29:02 +00:00 |
|
Ivan Kozik
|
1dc6cac6d8
|
Document how to use webrecorder.io instead of grab-site for archiving twitter.com/user pages
|
2016-01-07 07:28:23 +00:00 |
|
Ivan Kozik
|
01bf6d527b
|
Bump version
|
2016-01-05 01:08:14 +00:00 |
|
Ivan Kozik
|
64d928b82e
|
Merge pull request #67 from DanielOaks/fix-ws-exception
server: Fix exception when clients visit ws port via a browser
|
2016-01-05 01:06:49 +00:00 |
|
Daniel Oaks
|
04179a2a99
|
server: Fix exception when clients visit ws port via a browser
|
2016-01-05 10:53:37 +10:00 |
|
Ivan Kozik
|
111ffca643
|
global igset: ignore two more loops
|
2016-01-03 02:00:26 +00:00 |
|
Ivan Kozik
|
5f14263070
|
global igset: ignore livejournal.com/identity/login.bml
|
2015-12-30 05:35:57 +00:00 |
|
Ivan Kozik
|
7be4017347
|
README: Upgrade step is not quite right for pyenv users
|
2015-12-20 00:06:21 +00:00 |
|
Ivan Kozik
|
6e9fd55463
|
README: Add ~/.local/bin/ instructions here too
|
2015-12-20 00:04:53 +00:00 |
|
Ivan Kozik
|
c78ec516dc
|
README: Tweak to fix markdown rendering?
|
2015-12-20 00:03:20 +00:00 |
|
Ivan Kozik
|
5edd79c547
|
README: Tweak to fix markdown rendering?
|
2015-12-20 00:02:50 +00:00 |
|
Ivan Kozik
|
7a928ae7fc
|
README: Tweak to fix markdown rendering?
|
2015-12-20 00:02:08 +00:00 |
|
Ivan Kozik
|
ccdad68099
|
README: Update TOC
|
2015-12-20 00:01:07 +00:00 |
|
Ivan Kozik
|
a51ae254df
|
README: Add pyenv instructions
|
2015-12-20 00:00:13 +00:00 |
|
Ivan Kozik
|
2acc826d56
|
lstrip '-' to avoid creating filenames that must be --'ed or quoted
|
2015-12-17 16:54:55 +00:00 |
|
Ivan Kozik
|
495beca32a
|
Revert "README: Tweak"
This reverts commit dd0c005331dc182e6251823f5c90030c120680b3.
```bash coloring is unhelpfully broken
|
2015-12-17 15:19:20 +00:00 |
|
Ivan Kozik
|
dd0c005331
|
README: Tweak
|
2015-12-17 15:18:47 +00:00 |
|
Ivan Kozik
|
cb0fa6591b
|
Update Ubuntu 15.10 instructions now that https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1518023 is fixed
|
2015-12-17 15:17:40 +00:00 |
|
Ivan Kozik
|
4ea80eec80
|
global igset: Ignore a loop on archive.org
|
2015-12-16 13:11:16 +00:00 |
|
Ivan Kozik
|
38f733f9d2
|
global igset: ignore /wp-admin/
|
2015-12-16 11:00:54 +00:00 |
|
Ivan Kozik
|
adb35ee4e3
|
Add --id=, --dir=, and --finished-warc-dir= options
|
2015-12-12 19:52:32 +00:00 |
|
Ivan Kozik
|
17464e7b74
|
README: Tweak
|
2015-12-12 18:35:20 +00:00 |
|
Ivan Kozik
|
f8fece9ebb
|
Set NCR=1 cookie for .blogspot.com to avoid getting redirected
|
2015-12-12 18:29:53 +00:00 |
|
Ivan Kozik
|
7975566add
|
README: Add travis link and shields.io button
|
2015-12-12 17:58:52 +00:00 |
|
Ivan Kozik
|
0e5f8c8b83
|
.travis.yml: Test a few more things
|
2015-12-12 17:50:09 +00:00 |
|
Ivan Kozik
|
9c3c3c3afd
|
.travis.yml: Remove osx until https://github.com/travis-ci/travis-ci/issues/2312 is fixed
|
2015-12-12 17:46:31 +00:00 |
|
Ivan Kozik
|
e023105c5e
|
.travis.yml: pip3 install verbosely
|
2015-12-12 17:41:23 +00:00 |
|
Ivan Kozik
|
bdf1684834
|
.travis.yml: Fix script
|
2015-12-12 17:40:48 +00:00 |
|
Ivan Kozik
|
37ddc8c6b8
|
.travis.yml: remove --user because we're in a virtualenv
|
2015-12-12 17:39:18 +00:00 |
|
Ivan Kozik
|
2f121a864b
|
Add a .travis.yml
|
2015-12-12 17:36:35 +00:00 |
|
Ivan Kozik
|
86e92d684c
|
Send an over18=1 cookie to reddit.com to avoid the age gate on many subreddits
|
2015-12-12 16:49:42 +00:00 |
|
Ivan Kozik
|
6a647f637e
|
If using Python 3.4.0, depend on an older version of aiohttp that works on Python 3.4.0
|
2015-12-12 10:08:02 +00:00 |
|
Ivan Kozik
|
d96809ea78
|
README: Tweak
|
2015-12-11 09:10:07 +00:00 |
|
Ivan Kozik
|
a9a06127ea
|
README: More subreddit crawling
|
2015-12-11 09:03:02 +00:00 |
|
Ivan Kozik
|
01a72b910a
|
README: Tweak
|
2015-12-11 08:56:27 +00:00 |
|
Ivan Kozik
|
a2f1192b92
|
README: Document how to archive websites whose domains have just expired
|
2015-12-11 08:54:09 +00:00 |
|
Ivan Kozik
|
0363f825d5
|
README: Document how to archive subreddits
|
2015-12-11 08:48:18 +00:00 |
|
Ivan Kozik
|
52895892d5
|
README: Tweak
|
2015-12-11 08:44:23 +00:00 |
|
Ivan Kozik
|
e82937a9f6
|
README: Tweak
|
2015-12-11 08:43:30 +00:00 |
|