mikf/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,874 Commits 1 Branch 0 Tags
Commit Graph

1874 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
ee4d7c3d89
update downloader.find() and related code 
Instead of replacing 'https' with 'http' for every URL in
'get_downloader()', this now only happens once during downloader
initialization. Also unit tests.
 2019-06-20 16:59:44 +02:00
Mike Fährmann
f4ba98771d
use Last-Modified header to set file modification time 
(#236, #277)
 2019-06-19 23:16:32 +02:00
Mike Fährmann
179d112083
[downloader] overhaul http and text modules 
Get rid of the modular structure and simplify/specialize those modules.
 2019-06-19 22:56:11 +02:00
Mike Fährmann
a01f99728c
[postprocessor:zip] delete empty archives when done (#316) 2019-06-19 18:14:33 +02:00
Mike Fährmann
520c8ba106
[hentaicafe] extract 'tags' and 'artist' metadata (closes #238) 
These metadata fields will only be filled in when using a top-level
URL, because that's the only place this information is available. Using
a Foolslide URL (1) will leave these fields empty.

(1) https://hentai.cafe/manga/read/.../en/0/1/"
 2019-06-18 14:30:26 +02:00
Mike Fährmann
b51baa9a4b
[hitomi] fix empty language detection; parse datetime 2019-06-17 20:02:58 +02:00
Mike Fährmann
258e8b2060
[deviantart] small code improvements 2019-06-17 19:49:50 +02:00
Mike Fährmann
a77340c647
[keenspot] fix extraction for "TwoKinds" 2019-06-17 19:49:39 +02:00
Mike Fährmann
03e6876fbe
[instagram] provide 'description' metadata (#310) 2019-06-16 21:54:01 +02:00
Mike Fährmann
b171befa87
implement 'parse_unicode_escapes()' 2019-06-16 21:47:24 +02:00
Mike Fährmann
3a36a0fa1e
release version 1.8.6 2019-06-14 21:11:58 +02:00
Mike Fährmann
ec3e8601f1
[slickpic] add user extractor (#249) 2019-06-14 18:55:56 +02:00
Mike Fährmann
97ef416218
[8muses] support multi-page listings (#305) 2019-06-14 18:48:22 +02:00
Mike Fährmann
f5961ac968
[deviantart] download deviations with no 'content' field 
Some deviations (possibly only from sta.sh sources) are downloadable
(i.e. 'is_downloadable' is true and /deviation/download/ works), but
have no 'content' or similar  in their JSON representation.

(fixes #307)
 2019-06-13 21:14:12 +02:00
Mike Fährmann
4e07f99e3e
[mangoxo] change token message level to debug 
The login page currently doesn't provide and require a login token
(logging in works without a token), so printing a warning during
each login is unnecessary.
 2019-06-13 21:09:11 +02:00
Mike Fährmann
d997c10320
[8muses] add album extractor (#305) 2019-06-10 22:17:46 +02:00
Mike Fährmann
e05a96db5e
[deviantart] rename 'stash' to 'extra' (#302) 
'stash' is already used as a name for the StashExtractor and therefore
expected to be a dictionary.
 2019-06-10 21:05:25 +02:00
Mike Fährmann
2184e3a86b
[slickpic] add album extractor (#249) 2019-06-09 21:59:22 +02:00
Mike Fährmann
c23bf263fe
[deviantart] rename 'external' to 'stash' (#302) 
restrict extracted URLs to ones from https://sta.sh/...
 2019-06-09 11:16:02 +02:00
Mike Fährmann
c73c2cda50
[pornhub] add gallery & user extractor (#282) 2019-06-07 16:31:20 +02:00
Mike Fährmann
7c6cb908f9
[xhamster] update test results 2019-06-07 16:28:49 +02:00
Mike Fährmann
035b850e82
update postprocessor entries in example config 
- use whitelists
- add ugoira example (#299)
 2019-06-07 13:47:02 +02:00
Mike Fährmann
2fb85178da
[deviantart] add 'external' option (#302) 
If a description is available, this will extract URLs from the
description text and try to find Extractors for them.
 2019-06-06 18:53:50 +02:00
Mike Fährmann
f85e42cffc
[deviantart] fix --range for deviation & stash extractor 2019-06-06 18:45:10 +02:00
Mike Fährmann
40c7eb3424
[livedoor] improve extraction (fixes #301) 2019-06-06 15:22:27 +02:00
Mike Fährmann
62335b9015
[paheal] adjust test results 2019-06-05 11:42:01 +02:00
Mike Fährmann
aa1ca4ed35
[shopify] skip deleted products (#175) 
Product pages which return a 4xx status code will now be skipped instead
of raising an exception.
 2019-06-05 11:40:54 +02:00
Mike Fährmann
096009367b
[xhamster] add gallery & user extractor (#281) 2019-06-05 11:11:51 +02:00
Mike Fährmann
208202b962
[tumblr] improve error handling (#297) 
In some cases Tumblr's API responds with an HTML document.
Trying to decode it as JSON would raise an uncaught exception.
 2019-06-04 14:02:17 +02:00
Mike Fährmann
c08c340178
[directlink] make pattern case insensitive (fixes #296) 2019-06-03 10:56:14 +02:00
Mike Fährmann
95b4a53b9c
[keenspot] improve pagination (#223) 
The old code would skip the last comic page for some series.
 2019-06-02 22:12:21 +02:00
Mike Fährmann
12c965d547
release version 1.8.5 2019-06-01 20:57:55 +02:00
Mike Fährmann
731c7cbd5b
[keenspot] support all comics and "random" access (#223) 2019-06-01 20:48:13 +02:00
Mike Fährmann
6a34f4b0c1
skip tests on read timeouts; print list of skipped tests 2019-06-01 20:47:31 +02:00
Mike Fährmann
1c36e65e9b
[exhentai] choose site version depending on input URL (#278) 
Use e-hentai.org as root and cookiedomain if the input URL is from
e-hentai (or g.e-hentai), use exhentai.org otherwise.
 2019-05-31 15:34:39 +02:00
Mike Fährmann
6da3e21237
[downloader:ytdl] provide 'filename' metadata (closes #291) 2019-05-31 14:56:45 +02:00
Mike Fährmann
d33f5a7423
[wallhaven] rewrite 
- use API
- remove login support, add 'api-key' option
- remove support for "alpha" subdomain - alpha.wallhaven.cc used numeric
  IDs that can't be translated to the new ID system
- support direct links to wallpapers
 2019-05-31 14:53:02 +02:00
Mike Fährmann
5499934ae2
[ngomik] fix extraction 2019-05-30 20:18:36 +02:00
Mike Fährmann
f1893b2b5b
[deviantart] add 'folders' option (#276) 2019-05-30 17:28:12 +02:00
Mike Fährmann
c849574def
[keenspot] add comic extractor (#223) 
Doesn't work for
- http://brawlinthefamily.keenspot.com/
- http://flipside.keenspot.com/
- http://lastblood.keenspot.com/
- http://mysticrevolution.keenspot.com/
- http://porcelain.keenspot.com/
- http://twokinds.keenspot.com/
yet, because of custom layouts.
 2019-05-28 21:34:38 +02:00
Mike Fährmann
2b1999476e
implement 'text.rextract()' 2019-05-28 21:03:41 +02:00
Mike Fährmann
8bd5a19515
[hentainexus] add '_extractor' data 2019-05-28 00:20:01 +02:00
Mike Fährmann
2a085a5e96
[sankakucomplex] fix 'date' values (#258) 2019-05-28 00:18:58 +02:00
Mike Fährmann
bcd1801aa8
[sankakucomplex] add 'tag' extractor (#258) 2019-05-27 23:57:44 +02:00
Mike Fährmann
74c2415138
[sankakucomplex] move article extractor to its own module (#258) 2019-05-27 23:49:23 +02:00
Mike Fährmann
4465a3ea68
[kissmanga][readcomiconline] add 'captcha' option (#279) 
to configure how to handle CAPTCHA page redirects:
- either interactively wait for the user to solve the CAPTCHA
- or raise StopExtraction like before
 2019-05-27 22:24:48 +02:00
Mike Fährmann
e30ada162d
fix cookie tests 
update _get_extractor():
- always return an Extractor instance with a _login_impl() method
- use Extractor.from_url()
 2019-05-26 20:22:04 +02:00
Mike Fährmann
1e3e15c4f3
[sankaku] add article extractor (#258) 2019-05-26 17:42:36 +02:00
Mike Fährmann
48233f00c0
[readcomiconline] detect 'AreYouHuman' redirects (#279) 2019-05-26 15:58:37 +02:00
Mike Fährmann
1cde38110d
[livedoor] return 'date' as datetime object 2019-05-25 23:45:56 +02:00