706 Commits

Author SHA1 Message Date
Mike Fährmann
fbe8c519e2
update README
- mention and link to docs/configuration.rst
- add OAuth section
2017-06-23 16:14:51 +02:00
Mike Fährmann
e1d82af5e0
small fixes 2017-06-22 18:46:42 +02:00
Mike Fährmann
ccb7973b57
update configuration.rst 2017-06-20 16:20:28 +02:00
Mike Fährmann
719d45f89e
[flickr] allow the use of Flickr's specifiers for format selection
- renamed the 'width-max' option to 'size-max'
- filter by both width and height
2017-06-20 16:09:25 +02:00
Mike Fährmann
b4c438c9ad
[oauth] add the 'extractor.oauth.browser' option
enables/disables the use of webbrowser.open() during OAuth authorization
2017-06-20 16:06:14 +02:00
Mike Fährmann
1a9b4759a2 Merge pull request #21 from Hrxn/patch-1
Update configuration.rst
2017-06-19 14:27:31 +02:00
H R X N
b206b831ee Update configuration.rst
Fix heading.
2017-06-19 14:12:43 +02:00
Mike Fährmann
2633337833
[kissmanga] update regex (fixes #20) 2017-06-19 09:55:02 +02:00
Mike Fährmann
fac6c02224
[downloader] fix extension from content-type 2017-06-19 09:24:00 +02:00
Mike Fährmann
e68af4febe
[flickr] add 'width-max' option (#16)
This option allows for simple format selection by
specifying a maximum image width.
2017-06-18 22:20:15 +02:00
Mike Fährmann
48b444e276
add a list of all configuration options
(the format might be subject to change; putting everything  in
 one single table is even worse)
2017-06-18 22:16:26 +02:00
Mike Fährmann
2993206c4b
smaller fixes and "security" measures
- move the OAuthSession class into util.py
- block special extractors for reddit and recursive
- ignore 'only matching' tests for testresults script
2017-06-16 21:01:40 +02:00
Mike Fährmann
fb1904dd59
add 'Authentication' column to supportedsites.rst 2017-06-15 21:06:20 +02:00
Mike Fährmann
8d5e92f641
resolve cyclic dependency between oauth and flickr 2017-06-14 16:11:18 +02:00
Mike Fährmann
d60781de7b
[oauth] workaround for ctrl+c on Windows 2017-06-14 15:29:40 +02:00
Mike Fährmann
9759fe8c6b
allow 'only_matching' tests 2017-06-14 08:43:05 +02:00
Mike Fährmann
56bec79e6a
[reddit] add ability to load more comments (#15)
The 'extractor.reddit.morecomments' option enables the use of
the '/api/morechildren' API endpoint (1) to load even more
comments than the usual submission-request provides.
Possible values are the booleans 'true' and 'false' (default).

Note: this feature comes at the cost of 1 extra API call towards
the rate limit for every 100 extra comments.

(1) https://www.reddit.com/dev/api/#GET_api_morechildren
2017-06-13 18:49:07 +02:00
Mike Fährmann
05ed95e5b0
[flickr] add search extractor 2017-06-13 08:01:32 +02:00
Mike Fährmann
5f55c854b9
[flickr] replace getPublic... API call with regular ones 2017-06-12 16:37:06 +02:00
Mike Fährmann
9a620784f9
[flickr] add support for user authentication (#16)
Call '$ gallery-dl oauth:flickr' to get an access_token
and access_token_secret for your account.
2017-06-12 09:36:14 +02:00
Mike Fährmann
d5a70f2580
add simple progress indicator for multiple URLs (#19)
The output can be configured via the 'output.progress'
config value.

Possible values:
    - true:     Show the default progress indicator
                "[{current}/{total}] {url}" (default)
    - false:    Never show the progress indicator
    - <string>: Show the progress indicator using this
                as a custom format string(1).
                Possible replacement keys are:
                - current: current URL index
                - total  : total number of URLs
                - url    : current URL

(1) https://docs.python.org/3/library/string.html#formatstrings
2017-06-09 20:12:15 +02:00
Mike Fährmann
3ee77a0902
[oauth] print URL if webbrowser.open fails 2017-06-08 19:46:24 +02:00
Mike Fährmann
090e11b35d
[reddit] enable user authentication with OAuth2 (#15)
Call '$ gallery-dl oauth:reddit' to get a refresh_token
for your account.
2017-06-08 16:17:13 +02:00
Mike Fährmann
e682e06518
[flickr] add group extractor (#16) 2017-06-06 16:22:30 +02:00
Mike Fährmann
8fd66ef0b3
[flickr] add gallery extractor (#16) 2017-06-06 14:22:36 +02:00
Mike Fährmann
8456b84a12
fix tests and small stuff 2017-06-06 14:22:09 +02:00
Mike Fährmann
fbfc8d0f78
[reddit] ignore Authorization errors for subreddits
- also made the limit for retrieved comments customizable via
  the 'extractor.reddit.comments' config value
- default is 500;  0 ignores comments completely
2017-06-05 18:43:08 +02:00
Mike Fährmann
e365f1d799
[pixiv] rewrite
- same functionality, better(?) code quality, easier to extend

- added test for the user-tag functionality

- removed the 'artist-id', 'artist-name' and 'artist-nick'
  keywords, which can be replaced with 'user[id]', 'user[name]'
  and 'user[account]' respectively
2017-06-04 16:33:36 +02:00
aiasdfd
338f79147f [pixiv] support tag for user downloads (#17)
[pixiv] support tag for user downloads
2017-06-04 10:15:36 +02:00
Mike Fährmann
5f05543f23
[reddit] support filtering by timestamp (#15)
- Added the 'extractor.reddit.date-min' and '….date-max'
  config options. These values should be UTC timestamps.
- All submissions not posted in date-min <= T <= date-max
  will be ignored.

- Fixed the limit parameter for submission comments by setting
  it to its apparent max value (500).
2017-06-03 13:33:48 +02:00
Mike Fährmann
4e80e0c884
[flickr] add user extractor (#16) 2017-06-02 17:15:05 +02:00
Mike Fährmann
b81d068a6d
[flickr] add favorites extractor (#16) 2017-06-02 16:35:04 +02:00
Mike Fährmann
c921b4f32a
code cleanup and fixing tests 2017-06-02 09:10:58 +02:00
Mike Fährmann
dfe82526ee
update README 2017-06-02 09:08:08 +02:00
Mike Fährmann
72f1c6f87a
[flickr] add support for flic.kr/p/... URLs
Example:
    https://flic.kr/p/FPVo9U
2017-06-02 09:01:35 +02:00
Mike Fährmann
93e5d8cba3
[flickr] add album extractor 2017-05-31 17:31:51 +02:00
Mike Fährmann
659c65dbb0
[flickr] add image extractor 2017-05-30 17:43:02 +02:00
Mike Fährmann
b6fffa9e26
[directlink] update filename format and metadata 2017-05-30 17:33:09 +02:00
Mike Fährmann
c184e47ee3
put common directory- and filename formats in base classes 2017-05-30 12:10:16 +02:00
Mike Fährmann
bce51e90a5
[reddit] support sorting options and sub-options (#15)
Example:
    https://www.reddit.com/r/<subreddit>/top/?sort=top&t=month
    (the 'sort=top' parameter is irrelevant and can be omitted)
2017-05-29 12:45:35 +02:00
Mike Fährmann
5f45ce2930
[gfycat] add "format" config key to select a video format
Possible values:
    - one of "mp4" (default), "webm", "gif", "webp", "mjpg"

If the selected format is not available, "mp4", "webm" and "gif"
(in that order) will be tried instead, until an available format
is found.
2017-05-29 12:16:37 +02:00
Mike Fährmann
011659ced5
[imgur] add "mp4" config key to decide between GIF and MP4
possible values:
    - false   : always use GIF
    - true    : use MP4 if "prefer_video" flag is set,
                GIF otherwise (default)
    - "always": always use MP4
2017-05-29 08:48:07 +02:00
Mike Fährmann
48ccee2505
[gfycat] add image extractor 2017-05-28 17:09:54 +02:00
Mike Fährmann
25bcdc8aa9
add --write-unsupported option (#15) 2017-05-27 16:16:57 +02:00
Mike Fährmann
bf452a8516
[imgur] choose .mp4 over .gif if available 2017-05-27 11:49:29 +02:00
Mike Fährmann
f79320e35b
fix tests 2017-05-27 11:47:15 +02:00
Mike Fährmann
67791e1b36
[imgur] improve and add image extractor 2017-05-26 22:30:09 +02:00
Mike Fährmann
99b72130ee
[reddit] enable recursion (#15)
reddit extractors now recursively visit other submissions/posts
linked to in the initial set of submissions.
This behaviour can be configured via the 'extractor.reddit.recursion'
key in the configuration file or by `-o recursion=<value>`.

Example:
{"extractor": {
  "reddit": {
   "recursion": <value>
}}}

Possible values:
* -1 - infinite recursion (don't do this)
*  0 - recursion is disabled (default)
*  1 and higher - maximum recursion level
2017-05-26 17:01:27 +02:00
Mike Fährmann
ae686c4c08
run queue items immediately 2017-05-24 15:15:06 +02:00
Mike Fährmann
691c4dd709
support direct image links 2017-05-24 12:51:18 +02:00