mikf/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,289 Commits 1 Branch 0 Tags
Commit Graph

38 Commits

Author SHA1 Message Date
Mike Fährmann
173add6935
[nijie] fix artist_id extraction 
view_popup.php pages for older images or dojins either have the
artist_id value at a different place or not at all.
 2018-07-10 12:30:53 +02:00
Mike Fährmann
017188d268
improve extractor.request() 
Replace the 'fatal' parameter with 'expect', which is a list/range
of HTTP status codes >= 400 that should also be accepted.
 2018-06-18 16:29:56 +02:00
Mike Fährmann
2d17a9e07f
improve extractor.request() 
- better retry behavior
- exponential back-off
- removed 'allow_empty' argument
 2018-04-23 18:45:59 +02:00
Mike Fährmann
cc36f88586
rename safe_int to parse_int; move parse_* to text module 2018-04-20 14:53:21 +02:00
Mike Fährmann
7b562907c3
[nijie] add favorites extractor 
adds support for 'https://nijie.info/user_like_illust_view.php?id=...'
 2018-03-31 18:54:25 +02:00
Mike Fährmann
445db75955
[nijie] improve extraction and metadata 
- add 'title' and 'description'
- split 'artist_id' into 'user_id' and 'artist_id'
  - 'user_id' is the ID of the user from which the image entry
    originates from
  - 'artist_id' is the ID of the actual image artist
- improve pagination and URL patterns
 2018-03-31 18:48:41 +02:00
Mike Fährmann
a112e3f2a0
[nijie] add doujin extractor 
adds support for "https://nijie.info/members_dojin.php?id=<artist_id>"
 2018-03-31 18:17:41 +02:00
Mike Fährmann
3cec533c28
Merge branch 'archive' 2018-02-12 18:07:58 +01:00
Mike Fährmann
f5f2d29f56
[nijie] fix dojin extraction 
- correctly extract artist_id
- set extension to "jpg" if it was empty and let filetype checks do
  the rest
 2018-02-09 22:06:26 +01:00
Mike Fährmann
34873dbd90
set 'archive_fmt' values 
These are going to be used to create an unique id for each image.
 2018-02-01 15:30:49 +01:00
Mike Fährmann
9c138dfc1f
[common] detect empty HTTP response bodies 2017-09-26 16:49:58 +02:00
Mike Fährmann
6f30cf4c64
change keyword names to valid Python identifiers 
This commit mostly replaces all minus-signs ('-') in keyword names with
underscores ('_') to allow them to be used in filter-expressions. For
example 'gallery-id' got renamed to 'gallery_id'.

(It is theoretically possible to access any variable, regardless of its
name, with 'locals()["NAME"]', but that seems a bit too convoluted if
just 'NAME' could be enough)
 2017-09-10 22:20:47 +02:00
Mike Fährmann
915a0137de
improve 'extractor.request' 
- add 'fatal' argument
- improve internal logic and flow
- raise known exception on error
- update exception hierarchy
 2017-08-05 16:11:46 +02:00
Mike Fährmann
7aa9fa796a
code cleanup and fixes 2017-07-25 14:59:41 +02:00
Mike Fährmann
808f67ba7d
use 'cookiedomain' for cookies set by object-config-values 
otherwise these cookies would not be picked up by the
_check_cookies() method.
 2017-07-22 15:43:35 +02:00
Mike Fährmann
0610ae5000
skip login if cookies are present 2017-07-17 10:33:36 +02:00
Mike Fährmann
d3b04076f7
add .netrc support (#22) 
Use the '--netrc' cmdline option or set the 'netrc' config option
to 'true' to enable the use of .netrc authentication data.

The 'machine' names for the .netrc info are the lowercase extractor
names (or categories): batoto, exhentai, nijie, pixiv, seiga.
 2017-06-24 12:17:26 +02:00
Mike Fährmann
4b967fa189
implement and use extractor.config() method 2017-04-25 17:12:48 +02:00
Mike Fährmann
298d7c45f7
[nijie] support multi-page image listings 2017-04-02 11:43:23 +02:00
Mike Fährmann
1d46be545c
add login notifications 2017-03-17 09:42:59 +01:00
Mike Fährmann
94e10f249a
code adjustments according to pep8 nr2 2017-02-01 00:53:19 +01:00
Mike Fährmann
4a8d74973c
adjust login methods to a specific style 2017-01-08 17:33:25 +01:00
Mike Fährmann
7952b8d18d
add a few tests expecting exceptions 2016-12-30 01:46:42 +01:00
Mike Fährmann
56d810c896
update keyword hashes for tests 2016-09-25 17:28:46 +02:00
Mike Fährmann
19c2d4ff6f
remove explicit (sub)category keywords 2016-09-25 14:22:07 +02:00
Mike Fährmann
fea3be0aed
[nijie] add image-extractor 2016-09-19 08:51:49 +02:00
Mike Fährmann
d7e168799d
consistent extractor naming scheme + docstrings 2016-09-12 10:34:31 +02:00
Mike Fährmann
dcd213f13e
[nijie] raise NotFoundError 2016-09-04 11:01:16 +02:00
Mike Fährmann
d01b21c6c8
[nijie] change 'email'-parameter to 'username' 2016-07-24 22:22:49 +02:00
Mike Fährmann
0ba9601b63
[nijie] raise AuthError 2016-07-20 14:21:13 +02:00
Mike Fährmann
10fcc010bb
[nijie] cache login sessions 2016-03-06 21:08:58 +01:00
Mike Fährmann
2449532132
add more extractor test-cases 2015-12-13 04:36:44 +01:00
Mike Fährmann
75ff08f73c
[nijie] proper login and other issues fixed 2015-12-11 00:07:21 +01:00
Mike Fährmann
4d56b76aa8
update all other extractors 2015-11-21 04:26:30 +01:00
Mike Fährmann
c2f0720184
code cleanup to use nameext_from_url 2015-11-16 17:32:26 +01:00
Mike Fährmann
3c13548f29 rewrite extractors to use config-module 2015-10-05 15:51:08 +02:00
Mike Fährmann
42b8e81a68 rewrite extractors to use text-module 2015-10-03 15:43:02 +02:00
Mike Fährmann
a145a73d05 add extractor 'nijie' 2015-05-07 01:39:30 +02:00