68 Commits

Author	SHA1	Message	Date
Ivan Kozik	e6f830764e	Allow changing concurrency using DIR/concurrency file	2015-07-28 14:21:28 +00:00
Ivan Kozik	47c9a20ba7	Bump version	2015-07-28 14:01:42 +00:00
Ivan Kozik	3c28b53620	Allow changing delay (in milliseconds) using DIR/delay file	2015-07-28 13:44:51 +00:00
Ivan Kozik	cae516eb5d	Cache these control files for 3 seconds to reduce stat calls: ignores, igsets, igoff, stop	2015-07-28 13:23:00 +00:00
Ivan Kozik	4eb2805df0	Format DUPE/OF messages more nicely in terminal	2015-07-28 12:33:59 +00:00
Ivan Kozik	37d1f2e473	directory name gen: don't try and fail to create directory with > 255 chars when given a long URL	2015-07-28 12:16:32 +00:00
Ivan Kozik	0f1bdfd738	Don't spawn wpull in a subprocess, just import it and call its main()	2015-07-28 11:53:47 +00:00
Ivan Kozik	e506d6a103	Add gs-dump-urls, a utility to dump URLs from a wpull.db file	2015-07-28 11:26:10 +00:00
Ivan Kozik	991718b2e2	hooks: better ws:// connect messages, slow down reconnects exponentially	2015-07-27 14:01:22 +00:00
Ivan Kozik	36f24b03b3	hooks: print which ws:// server it can't connect to	2015-07-27 13:45:04 +00:00
Ivan Kozik	84b183ec84	Write proper --help text and use aliased inputs too	2015-07-27 06:44:55 +00:00
Ivan Kozik	915ed0eeae	Use cchardet for faster encoding detection (imported by wpull/thirdparty/dammit.py)	2015-07-21 03:25:56 +00:00
Ivan Kozik	c35b388677	Allow archiving archive.org content despite it being in the global ignore set	2015-07-20 09:02:19 +00:00
Ivan Kozik	35d6d780bd	Bump version	2015-07-20 08:37:46 +00:00
Ivan Kozik	1fce3af4a0	Add --1 option for turning off recursion; document options	2015-07-20 08:23:35 +00:00
Ivan Kozik	0e38441234	Add OS X support	2015-07-20 06:35:32 +00:00
Ivan Kozik	a5cc1d84c6	Bump version	2015-07-19 20:44:11 +00:00
Ivan Kozik	43d8a9594f	Move everything and make grab-site installable with pip3	2015-07-18 10:39:04 +00:00