Mike Fährmann
65744a7a31
use alternative for all falsey values in format strings
...
… and not just None (#525 )
It would be better to consistently use None for all non-existent
fields and/or fields without a valid value, but this is a good
enough workaround for now.
2020-09-19 22:02:47 +02:00
Mike Fährmann
c43b3894be
[myhentaigallery] update and fix extraction ( #1001 )
...
- extract more metadata
- match "/show/" URLs
- complete test results
- fix missing images for lines starting with " <img"
- fix missing comma in supportedsites.py
2020-09-17 18:14:23 +02:00
choeronline
05b9ac8d37
[myhentaigallery] add extractor ( #1001 )
...
* adds support for myhentaigallery
* fixes linting issues in myhentaigallery extractor
2020-09-17 17:32:54 +02:00
Mike Fährmann
2626629117
[danbooru] handle posts without 'id' ( fixes #1004 )
2020-09-16 21:35:27 +02:00
Mike Fährmann
cc1fb0b4ea
[500px] update query hash
2020-09-16 01:26:31 +02:00
Mike Fährmann
6a5f6396d4
add clearing cache to dA client-id instructions ( #996 )
2020-09-16 01:24:18 +02:00
Mike Fährmann
da87a5fb7e
[exhentai] fix accessing config before main constructor
...
bug introduced with 055c32e0
Making 'Extractor.config()' quite a bit faster is worth the "cost"
of having to set _cfgpath in exhentai constructors, I think.
2020-09-15 18:09:50 +02:00
Mike Fährmann
f5b7ae01c1
update extractor test results
2020-09-15 18:07:08 +02:00
Mike Fährmann
136df52d1f
[deviantart] support watchers-only/paid deviations ( #995 )
2020-09-15 16:03:46 +02:00
Mike Fährmann
055c32e0f7
precompute extractor config paths
2020-09-14 22:06:54 +02:00
Mike Fährmann
231dd4c800
accumulate postprocessor objects ( #994 )
...
Instead of one 'postprocessors' setting overwriting all others lower
in the hierarchy, all postprocessors along the config path will now
get collected into one big list.
For example '--mtime-from-date' will therefore no longer cause
other postprocessor settings in a config file to get ignored.
2020-09-14 21:51:55 +02:00
Mike Fährmann
392d022b04
implement 'config.accumulate()' ( #994 )
2020-09-14 21:13:08 +02:00
Mike Fährmann
3afd362e2e
add 'sleep-extractor' option ( closes #964 )
...
(would have been nice if this were possible without code duplication)
2020-09-12 21:04:47 +02:00
Mike Fährmann
3108e85b89
[worldthree] remove extractors
...
http://www.slide.world-three.org/ hasn't been accessible for a long time.
2020-09-11 18:12:57 +02:00
Mike Fährmann
8fed3eb8cb
[jaiminisbox] remove extractors
...
https://jaiminisbox.com/post.html
2020-09-11 18:09:35 +02:00
Mike Fährmann
dcf3ad7eef
[furaffinity] update download URL extraction ( fixes #988 )
...
support the new 'd2.facdn.net' subdomain
2020-09-11 13:23:57 +02:00
Mike Fährmann
3918b69677
remove 'extractor.blacklist' context manager
2020-09-11 13:17:35 +02:00
Mike Fährmann
c78aa17506
add general 'blacklist' and 'whitelist' options ( #492 , #844 )
2020-09-11 13:17:12 +02:00
Mike Fährmann
abda352a5b
add '--no-skip' command-line option ( closes #986 )
2020-09-11 01:23:39 +02:00
Mike Fährmann
5912727b88
support format string replacement fields in archive paths
...
(closes #985 )
2020-09-10 22:09:30 +02:00
Mike Fährmann
2b8d57f0ab
[twitter] support '/intent/user?user_id=…' URLs ( #980 )
2020-09-08 23:17:50 +02:00
Mike Fährmann
a3b473bd2f
[twitter] support specifying users by ID ( #980 )
...
by using 'id:…' as their screen name, i.e.
https://www.twitter.com/id:2976459548/media
instead of
https://twitter.com/supernaturepics/media
The user ID can, for example, be obtained from the output of
$ gallery-dl -j --range 1 https://twitter.com/ <screen-name>
2020-09-08 22:56:52 +02:00
Mike Fährmann
a0d916ed41
[exhentai] update wait time before original image download ( #978 )
...
depend on 'wait-max', don't use a hard-coded value
2020-09-07 23:48:28 +02:00
Mike Fährmann
f6fd449b59
reduce wait time growth rate from exponential to linear
...
Waiting for 2**N seconds after each error grows too fast.
Simply waiting N seconds seems far more reasonable.
2020-09-06 22:38:25 +02:00
Mike Fährmann
bc48514d84
[aryion] get post ID via gallery-item ( fixes #981 , closes #982 )
...
this even works when fetching post IDs from '/latest.php?id='
2020-09-06 22:17:23 +02:00
Mike Fährmann
799ca07fc8
[imgur] update
...
- fix image/album detection for galleries
- use new API endpoints for image/album data
2020-09-06 21:11:32 +02:00
Mike Fährmann
b5243297ff
write skipped files to archive ( closes #550 )
2020-09-03 18:37:38 +02:00
Mike Fährmann
ac3036ef56
add 'filesize-min' and 'filesize-max' options ( closes #780 )
2020-09-03 18:21:04 +02:00
Mike Fährmann
7876a03ece
[tumblr] create directories for each post ( fixes #965 )
...
This changes the identifiers for directory format string fields.
Everything blog related is now inside a 'blog' object
and not at the "base level" anymore.
E.g. '{name}' for directories is now '{blog[name]}'
(or '{blog_name}', since that is also available)
2020-08-31 21:58:20 +02:00
Mike Fährmann
fd0685d9b5
[postprocessor:zip] defer zip file creation ( fixes #968 )
...
don't try to create zip files on postprocessor construction,
wait until directory creation during file download,
2020-08-31 21:53:18 +02:00
Mike Fährmann
33fe67b594
release version 1.14.5
2020-08-30 21:20:26 +02:00
Mike Fährmann
d50f3b333a
update extractor test results
2020-08-30 20:55:22 +02:00
Mike Fährmann
0f55b8e80a
[exhentai] fix type check from dbbbb21 ( #940 )
...
'bool' is a subclass of 'int', and therefore
'isinstance(self.limits, int)' also returns True when
'self.limits' has a boolean value
2020-08-30 20:51:22 +02:00
Mike Fährmann
e33293fdd8
[hentaihand] update to new site layout
2020-08-30 00:41:03 +02:00
Mike Fährmann
fda9e296dd
[gelbooru] fix extraction without API
2020-08-28 22:33:37 +02:00
Mike Fährmann
69e4871005
update extractor test results
...
- sensescans: replace 404d chapters
- mangapark: replace 404d chapters
- subscribestar: update test for attached files
2020-08-28 22:32:32 +02:00
Mike Fährmann
ab1af66a97
[imgur] add 'search' extractor ( #934 )
2020-08-27 22:46:17 +02:00
Mike Fährmann
e4bbc1fb5c
[imgur] add 'tag' extractor ( #934 )
2020-08-27 22:46:17 +02:00
Mike Fährmann
deaacc70bb
[hitomi] update URL pattern for tag searches
2020-08-27 22:46:03 +02:00
ArtaxIsSleeping
0e941553ec
[aryion] Add username/password support ( #960 )
...
* Add username/password support to aryion extractor
* Update docs to match
* Fix code style
2020-08-27 22:45:30 +02:00
Mike Fährmann
84e04cc23b
[500px] fix extraction and update URL patterns ( fixes #956 )
...
- rewrite most API calls to GraphQL queries
- match '500px.com/p/<user>' URLs
2020-08-24 18:25:31 +02:00
Mike Fährmann
d4ff767291
[reddit] improve gallery extraction ( fixes #955 )
2020-08-23 22:06:06 +02:00
Mike Fährmann
7140fe7e6d
[hitomi] fix redirect processing
2020-08-23 15:18:44 +02:00
Mike Fährmann
a57b6b3c3a
[reddit] handle deleted galleries ( fixes #953 )
2020-08-20 20:14:07 +02:00
Mike Fährmann
063c71cd84
[furaffinity] add 'search' extractor ( closes #915 )
2020-08-18 21:26:46 +02:00
Mike Fährmann
dbbbb21180
[exhentai] add ability to specify custom image limit ( #940 )
2020-08-17 22:29:20 +02:00
Mike Fährmann
b2009ea39e
[aryion] update folder mime type list ( fixes #945 )
2020-08-16 22:30:15 +02:00
Mike Fährmann
688bd046fc
release version 1.14.4
2020-08-15 21:29:02 +02:00
Mike Fährmann
d06ad148c7
[shopify] use alternate regex for products on collection pages
...
when the first on doesn't yield any results
2020-08-15 18:24:14 +02:00
Mike Fährmann
7619152988
[reactor] sort 'tags'
...
to ensure a consistent order for test results
2020-08-15 18:22:31 +02:00
