mikf/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,447 Commits 1 Branch 0 Tags
Commit Graph

2253 Commits

Author SHA1 Message Date
Mike Fährmann
80ea9104b8
[8kun] adjust URL pattern 2020-02-11 19:00:13 +01:00
Mike Fährmann
c76c8b765a
[cloudflare] unescape challenge URL 2020-02-09 18:51:29 +01:00
Mike Fährmann
ce26070231
[pixiv] reduce calls to '/user/detail' 2020-02-09 13:54:58 +01:00
Mike Fährmann
da0d5f6092
[oauth] add 'port' option (#604) 2020-02-09 13:45:44 +01:00
Mike Fährmann
719b63d0ca
[bcy] add user and post extractors (#592) 2020-02-09 02:37:14 +01:00
Mike Fährmann
6426e3efc7
[khinsider] fix and improve metadata extraction 2020-02-07 18:20:38 +01:00
Mike Fährmann
4a3d2405de
[postprocessor:ugoira] small optimization 
Use tuples instead of lists when extending the list of
command-line arguments.
 2020-02-07 17:59:28 +01:00
Mike Fährmann
b7eb6cecbb
[pixiv] handle tags at the end of new bookmark URLs 2020-02-06 23:42:13 +01:00
Mike Fährmann
109f6c8685
[patreon] filter duplicate files per post (#590) 2020-02-05 23:38:24 +01:00
Mike Fährmann
b38cf59711
[sexcom] fix image URLs & parse 'date' fields 2020-02-04 22:52:00 +01:00
Mike Fährmann
1f4c9c5f9d
[8kun] add thread and board extractors (closes #582) 2020-02-04 22:50:31 +01:00
Mike Fährmann
facc5daa6d
[twitter] force old login page layout (fixes #584, fixes #598) 2020-02-02 17:24:53 +01:00
Mike Fährmann
d1de7dc296
[hitomi] implement workaround for "broken" redirects 
Some galleries redirect to a new "version" with different gallery id.
This new version might not be available any more, but the /reader/
page for the original gallery id can still work.
 2020-02-02 17:24:23 +01:00
Mike Fährmann
40fe062851
[pixiv] fix user id for bookmarks API calls (closes #596) 2020-02-01 01:48:46 +01:00
Mike Fährmann
91aaaf1a9e
[pixiv] add 'rating' metadata field (#595) 
A human-friendlier representation of 'x_restrict'
 2020-02-01 01:36:06 +01:00
Mike Fährmann
dff33b260c
[reddit] add 'videos' option 2020-01-31 23:45:02 +01:00
Mike Fährmann
2ad43618cc
[piczel] fix extraction 2020-01-31 15:46:21 +01:00
Mike Fährmann
cf7a67d67f
[yaplog] remove module 
Yaplog! ended its service on 2020-01-31
 2020-01-31 12:56:54 +01:00
Mike Fährmann
e0dd073ce0
[twitter] replace embedded tweet test 
the old one was deleted
 2020-01-31 12:51:55 +01:00
Mike Fährmann
ec36df4851
[deviantart] fix video extraction from 'extended_fetch' results 
DeviantArt is now serving videos from wixmp servers (1), instead of
the former film00.deviantart.com (2), even though those URLS are still
functional.

They seem to also have re-encoded those videos. The 10 MB 1080p video
from (2) is now only available in 720p at ~20 MB (with a higher
bitrate, but still …). Other videos are still available in 1080p, but
not this one for some reason.

(Changing the '720p' in (1) to '1080p' doesn't work.)

(1) https://wixmp-ed30a86b8c4ca887773594c2.wixmp.com/v/mp4/9feaa2c9-1baf-4fc2-84f7-f3384b34cefe/d5gxnb5-282a2e9a-b552-40ff-8542-b3c5eed823f5.720p.a837d7cec12c41be8ca2ee53152cea3a.mp4
(2) https://film00.deviantart.net/4c1d/v/mp4/2012/279/d/1/_video____brushes_i_use_in_paint_tool_sai_by_chi_u-d5gxnb5.mp4
 2020-01-30 18:02:21 +01:00
Mike Fährmann
48be2266ed
[deviantart] better error message for 'extended_fetch' (#585) 2020-01-30 15:25:33 +01:00
Mike Fährmann
383795b550
prevent superfluous calls to Logger.makeRecord() 
… by setting an appropriate minimal logging level for the root Logger.
 2020-01-30 15:19:06 +01:00
Mike Fährmann
71851a6241
[pixiv] update URLs of followed users to the new format 2020-01-30 15:17:42 +01:00
Mike Fährmann
d086f30b42
[reddit] restore archive keys for i.redd.it images 2020-01-29 22:12:55 +01:00
Mike Fährmann
56f1c96168
implement 'parent-directory' option (#551) 2020-01-29 18:32:37 +01:00
Mike Fährmann
ae07f92f7e
[reddit] rewrite extractor logic (closes #551) 
Handle images and videos hosted on Reddit "natively",
allowing them to use reddit-specific metadata to build directory
and file names.
 2020-01-29 17:57:25 +01:00
Mike Fährmann
2852691d78
[paheal] replace test URL 
searching for 'k-on' doesn't yield any results anymore
 2020-01-27 22:19:41 +01:00
Mike Fährmann
2a9be48511
improve util.load/save_cookiestxt() and add tests 
- take a file object as argument instead of an filename
- accept whitespace before comments ("   # comment")
- map expiration "0" to None and not the number 0
 2020-01-25 23:02:15 +01:00
Mike Fährmann
e35c2ea1a6
[weibo] use youtube-dl to download from m3u8 manifests 2020-01-24 23:39:34 +01:00
Mike Fährmann
6703b8a86b
[blogger] implement video extraction (closes #587) 2020-01-24 23:37:23 +01:00
Mike Fährmann
c1a6862863
implement functions to load/save cookies.txt files (closes #586) 
The methods of the standard libraries' MozillaCookieJar have
several shortcomings (#HttpOnly_ cookies, 0 expiration timestamps, etc.)
and require construction of an ultimately pointless CookieJar object.
 2020-01-21 21:59:36 +01:00
Mike Fährmann
5d73b7f29c
release version 1.12.3 2020-01-19 23:31:28 +01:00
Mike Fährmann
37247dbaff
miscellaneous fixes 2020-01-19 22:53:06 +01:00
Mike Fährmann
0e9dc5c88e
fix AttributeError when accessing 'temppath' 
[ci skip]
 2020-01-19 00:41:21 +01:00
Mike Fährmann
25d5ec4ff3
[twitter] add option to extract TwitPic embeds (#579) 2020-01-18 21:31:29 +01:00
Mike Fährmann
254f7c3999
implement a post-processor module to compare file versions 
(#530)
 2020-01-18 21:08:25 +01:00
Mike Fährmann
32d7195d08
[pinterest] improve detection of invalid pin.it links 2020-01-18 21:06:44 +01:00
Mike Fährmann
0b84068d84
remove temp files before downloading from fallback URLs 
otherwise the next call to download() with a fallback URL could see
the partially downloaded "remains" from the previous, failed download
attempt and "continue" it, writing the second half of a potentially
different version of that file.
 2020-01-18 00:47:17 +01:00
Mike Fährmann
760b9b4db4
add remove_file() and remove_directory() helpers 
these functions call os.unlink() or os.rmdir()
while catching and suppressing potential OSErrors
 2020-01-18 00:21:26 +01:00
Mike Fährmann
b2d542ad40
improve PathFormat._enum_file() 
open only one try-except block for the whole loop,
instead of one for each iteration in os.path.exists()
 2020-01-18 00:21:25 +01:00
Mike Fährmann
174117f827
allow multiple hashes for content tests 2020-01-18 00:19:35 +01:00
Alice
f498a9057f [twitter] Fix stop before real end (#573) 
* [twitter] Fix stop before real end

Fix for https://github.com/mikf/gallery-dl/issues/544. Makes sure that it really reached the end by checking that both "min_position" is null and "has_more_items" is false before stopping.

* [twitter] Fix stop before real end (update)
 2020-01-14 12:24:30 +01:00
Mike Fährmann
8bb32ee188
[hitomi] fix image URLs 2020-01-14 12:04:48 +01:00
Mike Fährmann
bd5ce9855c
allow GalleryExtractors to set URL-independent extensions 2020-01-14 11:53:32 +01:00
Mike Fährmann
af42c75152
[mangadex] revert domain to 'mangadex.org' 2020-01-14 09:49:57 +01:00
Mike Fährmann
200aea308a
[downloader:common] enable 'job'/'extractor' for logging messages 
(#574)
 2020-01-12 21:41:16 +01:00
Mike Fährmann
e89413da22
update test results 2020-01-12 21:41:06 +01:00
Mike Fährmann
33a6e0ac6e
[hentaifoundry] extract more metadata (closes #565) 2020-01-11 23:22:50 +01:00
Mike Fährmann
5cac79c3d9
[erolord] remove extractor 2020-01-10 14:40:30 +01:00
Mike Fährmann
b9cbf932b4
[pixiv] update URL patterns (fixes #568) 
Pixiv now uses new URLs for
- user profiles and illustration listings:
  - https://www.pixiv.net/en/users/<ID>
  - https://www.pixiv.net/en/users/<ID>/artworks
- bookmarks:
  - https://www.pixiv.net/en/users/<ID>/bookmarks/artworks
 2020-01-10 13:38:09 +01:00