mikf/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,900 Commits 1 Branch 0 Tags
Commit Graph

2675 Commits

Author SHA1 Message Date
Mike Fährmann
de0656941b
[twitter] add extractor for followed users (#1337) 
https://twitter.com/USER/following or
https://twitter.com/id:USERID/following
 2021-02-22 18:22:01 +01:00
Mike Fährmann
e39aea42cd
fix supportedsites.py for modules without docstring 
(fixes #1332)
 2021-02-21 22:57:37 +01:00
loragja
7b5ee922b7
cyberdrop extractor (#1328) 
* create cyberdrop extractor

* add cyberdrop to list of extractors

* fix formatting

* change class name from CyberdropExtractor to CyberdropAlbumExtractor

* add cyberdrop to list of supported sites

* attempt to clean up diff of supportedsites.rst

* replace regex with functions from text library
 2021-02-21 20:42:45 +01:00
Mike Fährmann
5ad2b9c82b
[deviantart] extend 'extra' option 
also download from embedded DeviantArt posts
 2021-02-21 19:27:04 +01:00
Mike Fährmann
560277394e
[downloader:http] add 'headers' option (#1322) 2021-02-21 19:13:39 +01:00
Mike Fährmann
6b0ecbf6bc
[hentainexus] add 'orignal' option (#1322) 2021-02-21 19:06:12 +01:00
Mike Fährmann
5542a11c46
[twitter] update GraphQL endpoints 2021-02-20 02:09:17 +01:00
Mike Fährmann
e1a12761d7
strip '/' from instance root URLs 2021-02-17 23:07:17 +01:00
Mike Fährmann
595bdaa4be
add extractors for gelbooru v0.1 sites 
- support https://illusioncards.booru.org/  (closes #426)
- support https://the-collection.booru.org/ (closes #767)
- support https://allgirl.booru.org/
- closes #234, closes #473, closes #1238

To get gallery-dl to recognize other sites running Gelbooru v0.1
(most sites on booru.org), add one or more entries to the
'gelbooru_v01' block in your config file. For example:

{
    "extractor": {
        "gelbooru_v01": {
            "rozenmaidenbooru": {"root": "http://rm.booru.org"},
            "drawfriendsbooru": {"root": "http://drawfriends.booru.org"}
        }
    }
}
 2021-02-17 02:36:27 +01:00
Mike Fährmann
59fd740b47
[tbib] add support for https://tbib.org/ (#473, closes #1082) 2021-02-17 00:28:25 +01:00
Mike Fährmann
08d7934c6e
move extractors from booru.py into their own gelbooru_v02 module 2021-02-17 00:26:24 +01:00
Mike Fährmann
d656892670
remove cloudflare.py 
The old IUAM challenge doesn't get used anymore, i.e. code to bypass it
is pointless, and the 'is_...()' checks are simple enough to directly
include them in 'extractor.request()'.
 2021-02-15 23:17:02 +01:00
Mike Fährmann
65ca923b4e
fix 'whitelist' option for BaseExtractor instances 2021-02-15 21:58:33 +01:00
Mike Fährmann
fbfcbcbf57
Merge branch '1.17.0' 2021-02-15 02:32:33 +01:00
Mike Fährmann
6e40585fb1
release version 1.16.5 2021-02-14 16:46:59 +01:00
Mike Fährmann
ba693d8686
[patreon] skip posts without view permission (#1316) 2021-02-14 16:03:11 +01:00
Mike Fährmann
dcbd995346
[vanillarock] fix metadata extraction 2021-02-14 15:44:22 +01:00
Mike Fährmann
4b1cda4cf7
[paheal] fix metadata extraction 2021-02-14 15:43:39 +01:00
Mike Fährmann
2919d78bfc
update extractor test results 2021-02-14 15:37:39 +01:00
Mike Fährmann
8974f0361c
[pixiv] update (#1304) 
- remove login with username & password
- require a refresh token
- add 'oauth:pixiv' functionality

See also:
- https://github.com/upbit/pixivpy/issues/158
- https://gist.github.com/ZipFile/c9ebedb224406f4f11845ab700124362
 2021-02-12 18:07:16 +01:00
Mike Fährmann
79c0fc249b
[mangadex] add 'api-server' option (#1309) 
and change the API server back to 'https://mangadex.org/api' for now
 2021-02-11 19:52:16 +01:00
Mike Fährmann
96a51ff169
[sankaku] update invalid-token detection (fixes #1309) 2021-02-11 19:49:24 +01:00
Mike Fährmann
b3cd970d87
[postprocessor:metadata] fix crash with 'extension-format' 
Using the 'extension-format' option for events where no filename
extension is available caused a crash.

(fixes #1285)
 2021-02-10 18:45:21 +01:00
Mike Fährmann
23be48427c
[deviantart] fix 'folders' option (closes #1302) 
don't assume parent folders are listed before their children
 2021-02-10 17:45:30 +01:00
Mike Fährmann
ca6b0fc2ac
[imagehosts] cleanup 2021-02-10 17:36:45 +01:00
Mike Fährmann
95a66bdad6
[imgclick] add 'image' extractor (closes #1307) 
basically reverts b0e8daf4154f92e21896728be6fa373a40d5d7be
 2021-02-10 17:22:12 +01:00
Mike Fährmann
fc78210725
[kemonoparty] include 'service' in directories and archive keys 2021-02-09 20:35:36 +01:00
Mike Fährmann
c386a9fabf
[kemonoparty] fix absolute file URLs 2021-02-09 20:29:39 +01:00
Mike Fährmann
7e7158e7c0
[kemonoparty] support URLs with non-numeric user and post IDs 
(fixes #1303)
 2021-02-09 19:37:22 +01:00
Mike Fährmann
e88d5bede8
[500px] update query hash 2021-02-08 22:40:02 +01:00
Mike Fährmann
280b1ac16d
[slideshare] fix extraction 2021-02-08 22:29:30 +01:00
Mike Fährmann
ae530f6365
[erome] add extractors for albums, users, searches (closes #409) 2021-02-07 22:58:19 +01:00
ticklebits
fa6d4d73c7
[hentaifox] support searching by group (#1294) 
Groups on hentaifox lists the items to download the same way as the other pages (artists, search, tag, etc). Added group to the pattern to search, and the test.
 2021-02-06 17:52:00 +01:00
Mike Fährmann
2cc1e04fe5
[kemonoparty] extract inline images (fixes #1286) 2021-02-05 20:10:20 +01:00
Mike Fährmann
56a8968435
remove 'Message.Metadata' (#866) 2021-01-31 02:12:37 +01:00
Mike Fährmann
1d145a6186
[mastodon] use cache for OAuth tokens (#616) 2021-01-31 01:38:23 +01:00
Mike Fährmann
a041a017d1
[pillowfort] ignore files without download URL (#846) 2021-01-30 21:50:56 +01:00
Mike Fährmann
23ff936d46
[nsfwalbum] use fallback for deleted content (fixes #1259) 2021-01-29 23:35:20 +01:00
Mike Fährmann
a228bb3a5f
[downloader:http] support callbacks to validate responses 2021-01-29 22:15:21 +01:00
Mike Fährmann
6b2bce3b7d
[behance] support 'video' modules (closes #1282) 
(requires youtube-dl to download from m3u8 manifests)
 2021-01-29 21:30:14 +01:00
Mike Fährmann
b0cff979b1
[inkbunny] raise NotFoundError for invalid/private submissions 
instead of crashing
 2021-01-28 17:05:52 +01:00
Mike Fährmann
e61b125fcc
[inkbunny] add 'sid' parameter to private file downloads 
(fixes #1281)
 2021-01-28 15:45:48 +01:00
Mike Fährmann
36bf76fa44
update 'oauth:mastodon:<instance>' code 2021-01-28 02:20:12 +01:00
Mike Fährmann
88fae99811
remove 'generate_extractors()' 2021-01-28 01:04:50 +01:00
Mike Fährmann
fa33f13453
[mastodon] update 
- inherit from BaseExtractor
- remove custom generate_extractors() and config()
- improve layout of MastodonAPI internals
 2021-01-27 23:49:01 +01:00
Mike Fährmann
231bcad614
[shopify] use BaseExtractor 2021-01-27 23:45:22 +01:00
Mike Fährmann
2de8ebc098
[moebooru] use BaseExtractor 2021-01-27 01:33:40 +01:00
Mike Fährmann
0978c1e184
[booru] use BaseExtractor 2021-01-27 01:33:01 +01:00
Mike Fährmann
c6cc86d7d0
[deviantart] update parameters for '/browse/popular' 
- limit results to 50 when also querying metadata (fixes #1267)
- remove deprecated 'category_path' parameter
 2021-01-26 23:35:32 +01:00
Mike Fährmann
993856b866
[foolslide] use BaseExtractor 2021-01-26 18:50:32 +01:00