mikf/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,558 Commits 1 Branch 0 Tags
Commit Graph

47 Commits

Author SHA1 Message Date
Mike Fährmann
abbd45d0f4
update handling of extractor URL patterns 
When loading extractor classes during 'extractor.find(…)', their
'pattern' attribute will be replaced with a compiled version of itself.
 2019-02-08 20:08:16 +01:00
Mike Fährmann
53c2fd4664
add mastodon/foolslide/foolfuuka examples to example config 2019-02-05 16:17:25 +01:00
Mike Fährmann
bfbbac4495
[tsumino] add login capabilities (#161) 2019-01-30 17:58:48 +01:00
Mike Fährmann
0156189468
[hentaifox] add chapter extractor (#160) 2019-01-28 18:00:32 +01:00
Mike Fährmann
e4171d6baf
[luscious] add login capabilities (closes #159) 2019-01-28 17:14:15 +01:00
Mike Fährmann
e53cdfd6a8
update build_supportedsites.py 2019-01-09 14:58:35 +01:00
Mike Fährmann
6c71e9cf5d
[deviantart] add separate 'sta.sh' extractor (#113) 
- supports multiple stashed deviations per page
- explicitly mentions sta.sh support on supportedsites.rst
 2018-12-26 18:56:57 +01:00
Mike Fährmann
2cf3f53839
[yuki] add thread extractor (closes #111) 2018-09-28 12:47:32 +02:00
Mike Fährmann
17611bfec0
update build_supportedsites.py script 2018-09-28 12:43:19 +02:00
Mike Fährmann
c402cc4047
[hentaifoundry] add 'popular' and 'recent' extractors 
for "Popular Pictures" and "Recent Pictures" listings
 2018-09-24 13:11:18 +02:00
Mike Fährmann
6ea9a78588
[wallhaven] add login capabilities 
Being logged in is required to access NSFW wallpapers.
 2018-09-19 21:04:01 +02:00
Mike Fährmann
69fd61ea86
[bobx] add gallery and model extractors 2018-09-13 20:13:12 +02:00
Mike Fährmann
3ecea4cf36
[hentaicafe] add chapter and manga extractors (#101) 2018-09-05 21:08:40 +02:00
Mike Fährmann
75862715ac
[behance] add user extractor 2018-08-31 17:42:09 +02:00
Mike Fährmann
e9dd2eff1d
[twitter] add extractor for media-tweet timelines (#96) 
For example "https://twitter.com/PicturesEarth/media".
They are different from normal timelines in that they do not contain
any (re)tweets from other users and feature all media the user ever
posted, including responses to other tweets.
 2018-08-19 20:46:12 +02:00
Mike Fährmann
63fa0b2006
[pinterest] add extractors for related pins 
Related pins can not be accessed by adding a "#related" fragment
to the end of a Pinterest URL, for example:
- https://www.pinterest.com/pin/858146903966145189/#related
- https://www.pinterest.com/g1952849/test-/#related

There are no explicit real URLs for related pins,
using an option to enable them results in "clunky" code,
and a custom "related:<URL>" scheme doesn't feel right either.
 2018-08-15 21:49:45 +02:00
Mike Fährmann
84854fcad7
[myportfolio] add user and gallery extractors (#95) 2018-07-19 18:56:45 +02:00
Mike Fährmann
a6fe2bb594
[whatisthisimnotgoodwithcomputers] remove extractor 2018-07-14 09:53:16 +02:00
Mike Fährmann
55b0913412
[simplyhentai] add gallery extractor (#89) 2018-05-27 15:25:04 +02:00
Mike Fährmann
f3483a2b7c
[smugmug] add OAuth support 2018-05-10 18:58:05 +02:00
Mike Fährmann
82c50fa609
release version 1.3.5 2018-05-04 10:03:20 +02:00
Mike Fährmann
16e014baaa
[smugmug] added image and album extractor 
just some initial code that still requires a lot of work ...

TODO:
- folders
- old-style albums (which are nearly all of them ...)
- images from users
- OAuth

It could also happen that the API credentials used will become invalid
whenever my 14 day trial period ends (7 days remaining), but that
would just require users to supply their own.
 2018-04-29 21:27:25 +02:00
Mike Fährmann
d10579edb5
[pinterest] improve PinterestAPI code; remove OAuth mentions 
on another note: access_tokens have been set to only allow for
10 requests per hour (from 200 yesterday)
 2018-04-17 17:12:42 +02:00
Mike Fährmann
4bd182c107
[pinterest] implement oauth:pinterest (#83) 
Pinterest access tokens are rate limited at 200 requests per
hour (or maybe per 2 or 3 hours?) so having just one access token
for all users isn't going to work in the long run.
 2018-04-16 20:03:28 +02:00
Mike Fährmann
7b562907c3
[nijie] add favorites extractor 
adds support for 'https://nijie.info/user_like_illust_view.php?id=...'
 2018-03-31 18:54:25 +02:00
Mike Fährmann
7121eeae8b
check supportedsites.rst in release script 2018-03-17 15:35:38 +01:00
Mike Fährmann
1400868f53
[mangadex] general improvements 
- support >100 chapter entries per manga
- custom archive ID format
- detect non-existing chapters
 2018-03-06 14:15:15 +01:00
Mike Fährmann
40d35c87bc
[paheal] add tag- and post-extractors (closes #69) 2018-01-15 16:39:05 +01:00
Mike Fährmann
35e09869d1
[mangapark] fix image URLs and use HTTPS 2018-01-12 14:59:49 +01:00
Mike Fährmann
29d75fc3fa
[tumblr] add support for OAuth authentication (#65) 2018-01-11 14:11:37 +01:00
Mike Fährmann
b33efc99a4
[idolcomplex] add support for idol.sankakucomplex.com 2018-01-09 17:54:37 +01:00
Mike Fährmann
5b094328b5
[puremashiro] add chapter- and manga-extractor (closes #66) 
Also adds support for region subtags in language codes (e.g. en-us)
 2018-01-07 21:50:43 +01:00
Mike Fährmann
fc7d165c97
[deviantart] add support for OAuth2 authentication 
Some user galleries [*] require you to be either logged in or
authenticated via OAuth2 to access their deviations.

[*] e.g. https://polinaegorussia.deviantart.com/gallery/

--------------

known issue:
A deviantart 'refresh_token' can only be used once and gets updated
whenever it is used to request a new 'access_token', so storing its
initial value in a config file and reusing it again and again is not
possible.
 2017-12-18 01:16:46 +01:00
Mike Fährmann
0a9a07a6e1
[slideshare] improve metadata; flake8 
- added 'views' and 'published' keywords
- fixed longer titles and descriptions
 2017-12-13 21:16:49 +01:00
Mike Fährmann
2d0cfb33e1
[xvideos] add user profile extractor (#45) 2017-11-02 17:28:35 +01:00
Mike Fährmann
6af921a952
[sankaku] rewrite/improve (fixes #44) 
- add wait-time between HTTP requests similar to exhentai
- add 'wait-min' and 'wait-max' options
- increase retry-count for HTTP requests to 10
- implement user authentication (non-authenticated users can only view
  images up to page 25)
- implement 'skip()' functionality (only works up to page 50)
- implement image-retrieval for pages >= 51
- fix issue with multiple tags
 2017-10-14 23:01:33 +02:00
Mike Fährmann
cebf800a7f
[foolfuuka] add support for more sites (#18) 
- https://arch.b4k.co
- https://archive.whatisthisimnotgoodwithcomputers.com
- https://archive.yeet.net

Notes:
- The name "whatisthisimnotgoodwithcomputers" is way too long ...
- archive.yeet.net is out of date and also blocked by 4chan servers
  - newest threads are 2 weeks old
  - using "https://archive.yeet.net" as Referer header results in
    "403 Forbidden" when accessing 4chan
 2017-09-16 21:36:16 +02:00
Mike Fährmann
18e6ed1c7e
[booru] add extractors for "Popular" images 2017-08-24 21:29:22 +02:00
Mike Fährmann
d7cb3c668a
update supportedsites.rst 2017-07-24 10:50:40 +02:00
Mike Fährmann
74a2b5af06
update supportedsites.rst 2017-07-15 15:01:30 +02:00
Mike Fährmann
1dac76fd1c
update extractor docstrings 2017-06-28 17:39:07 +02:00
Mike Fährmann
fb1904dd59
add 'Authentication' column to supportedsites.rst 2017-06-15 21:06:20 +02:00
Mike Fährmann
c921b4f32a
code cleanup and fixing tests 2017-06-02 09:10:58 +02:00
Mike Fährmann
1e416e5227
update README 2017-05-09 15:43:55 +02:00
Mike Fährmann
4332694492
add simple imagehosts to list of supported sites 2017-04-23 17:08:45 +02:00
Mike Fährmann
9f32cf1f4e
improve supportedsites.rst and build script 2017-04-20 16:56:50 +02:00
Mike Fährmann
659577ae54
add a script to automatically build a list of supported sites 2017-01-15 21:31:21 +01:00