mikf/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,843 Commits 1 Branch 0 Tags
Commit Graph

189 Commits

Author SHA1 Message Date
Mike Fährmann
2153206093
[imgbb] add album extractor (#361) 2019-07-30 23:11:19 +02:00
Mike Fährmann
60cf40380a
[vsco] add user extractor (#331) 2019-07-23 16:23:11 +02:00
Mike Fährmann
3fe5ccdfa6
[adultempire] add gallery extractor (closes #340) 2019-07-21 22:29:57 +02:00
Mike Fährmann
2ff73873f0
[erolord] add gallery extractor (closes #326) 2019-07-04 20:28:04 +02:00
Mike Fährmann
18a1f8c6cd
[vanillarock] add post and tag extractors (closes #254) 2019-06-23 22:45:36 +02:00
Mike Fährmann
f0c5093812
[nsfwalbum] add album extractor (closes #287) 2019-06-23 22:45:07 +02:00
Mike Fährmann
d997c10320
[8muses] add album extractor (#305) 2019-06-10 22:17:46 +02:00
Mike Fährmann
2184e3a86b
[slickpic] add album extractor (#249) 2019-06-09 21:59:22 +02:00
Mike Fährmann
c73c2cda50
[pornhub] add gallery & user extractor (#282) 2019-06-07 16:31:20 +02:00
Mike Fährmann
096009367b
[xhamster] add gallery & user extractor (#281) 2019-06-05 11:11:51 +02:00
Mike Fährmann
c849574def
[keenspot] add comic extractor (#223) 
Doesn't work for
- http://brawlinthefamily.keenspot.com/
- http://flipside.keenspot.com/
- http://lastblood.keenspot.com/
- http://mysticrevolution.keenspot.com/
- http://porcelain.keenspot.com/
- http://twokinds.keenspot.com/
yet, because of custom layouts.
 2019-05-28 21:34:38 +02:00
Mike Fährmann
74c2415138
[sankakucomplex] move article extractor to its own module (#258) 2019-05-27 23:49:23 +02:00
Mike Fährmann
e82cadac61
[patreon] add extractors (#226) 2019-05-17 10:47:22 +02:00
Mike Fährmann
ba8eb1ffec
[hentainexus] add gallery extractor (#256) 2019-05-12 23:59:41 +02:00
Mike Fährmann
a247c94c34
[sexcom] add pin and board extractors (#147) 2019-04-24 22:09:19 +02:00
Mike Fährmann
0b2ff406f6
[plurk] add timeline- and post-extractors (#212) 2019-04-14 21:48:38 +02:00
Mike Fährmann
e7cd5510d5
[pixnet] add extractors (closes #177) 
for:
- users/blogs: http://albertayu773.pixnet.net/
- folders: https://albertayu773.pixnet.net/album/folder/1405768
- sets   : https://albertayu773.pixnet.net/album/set/15078995
- photos : https://albertayu773.pixnet.net/album/photo/159443828
 2019-04-11 19:27:02 +02:00
Mike Fährmann
35919a9bb8
[livedoor] add blog- and post-extractors (#190) 2019-04-06 16:27:48 +02:00
Mike Fährmann
2ff043edfa
[yaplog] add user- and post-extractors (#190) 2019-04-04 17:56:56 +02:00
Mike Fährmann
6da665f32e
[mangoxo] add album- and channel-extractors (closes #184) 2019-04-03 07:55:51 +02:00
Mike Fährmann
c70b21248d
[wikiart] add extractors (#179) 
for
- artists:          https://www.wikiart.org/en/thomas-cole
- artist-listings:  https://www.wikiart.org/en/artists-by-century/12
- artwork-listings: https://www.wikiart.org/en/paintings-by-media/grisaille
 2019-04-02 17:34:57 +02:00
Mike Fährmann
6ba67b0537
[hypnohub] add extractors (closes #196) 2019-03-23 09:50:39 +01:00
Mike Fährmann
8d96a8ce4c
[500px] add user-, gallery-, and image-extractors (#185) 2019-03-20 17:32:36 +01:00
Mike Fährmann
6277a739e4
[35photo] add user-, genre-, and image-extractors (#162) 2019-03-18 01:11:30 +01:00
Mike Fährmann
9c0e2f294b
[shopify] add generic collection and product extractors (#175) 
with fashionnova.com  as a default domain
 2019-03-05 22:33:37 +01:00
Mike Fährmann
280531c8ff
[pururin] add gallery extractor (closes #174) 2019-02-25 14:54:57 +01:00
Mike Fährmann
19860655a3
[weibo] add 'user' and 'status' extractors 2019-02-17 18:18:31 +01:00
Mike Fährmann
abbd45d0f4
update handling of extractor URL patterns 
When loading extractor classes during 'extractor.find(…)', their
'pattern' attribute will be replaced with a compiled version of itself.
 2019-02-08 20:08:16 +01:00
Mike Fährmann
6284731107
simplify extractor constants 
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
 2019-02-08 13:45:40 +01:00
Mike Fährmann
2f3a021d72
[hentaicafe] restore functionality 2019-02-05 10:22:52 +01:00
Mike Fährmann
e1bf3b225e
[foolslide] dynamically generate extractor classes 2019-02-04 12:54:07 +01:00
Mike Fährmann
58a9eede38
[foolfuuka] dynamically generate extractor classes 2019-02-03 17:09:45 +01:00
Mike Fährmann
80a75a1ecf
[tsumino] add gallery extractor (#161) 2019-01-29 17:28:48 +01:00
Mike Fährmann
0156189468
[hentaifox] add chapter extractor (#160) 2019-01-28 18:00:32 +01:00
Mike Fährmann
d19bac71be
[photobucket] add 'album' extractor (#117) 2019-01-20 16:19:13 +01:00
Mike Fährmann
b8fed34548
add generalized extractors for Mastodon instances (#144) 
Extractors for Mastodon instances can now be dynamically generated,
based on the instance names in the 'extractor.mastodon.*' config path.

Example:
{
    "extractor": {
        "mastodon": {
            "pawoo.net": { ... },
            "mastodon.xyz": { ... },
            "tabletop.social": { ... },
            ...
        }
    }
}

Each entry requires an 'access-token' value, which can be generated with
'gallery-dl oauth:mastodon:<instance URL>'.
An 'access-token' (as well as a 'client-id' and 'client-secret') for
pawoo.net is always available, but can be overwritten as necessary.
 2019-01-19 14:28:59 +01:00
Mike Fährmann
8e01cf0ef8
[reactor] generalize extractors (#148) 
- support *.reactor.cc domains
- combine joyreactor and pornreactor modules
 2019-01-07 17:06:47 +01:00
Leonardo Taccari
2655a2ea02 Add support for instagram.com user profiles and pages (#134) 
* [instagram] Add extractor for instagram.com user profiles and pages

The extractor scrapes `instagram.com/<user>' timelines and
`instagram.com/p/<shortcode>' by mimicking the behaviour of a web
browser and extracting the sharedData JSON of the single pages.

Please note that this mean that for user timelines we also do an
extra request to the `instagram.com/p/<shortcode>' page but this
permit to have consistent (and all) information about the media
fetched.

The MD5 logic used for X-Instagram-GIS was documented in

 <https://stackoverflow.com/questions/49786980/>

* [instagram] Test for keywords, not url for GraphImage and GraphSidecar

URLs returned by instagram seems not stable so avoid testing for
them and instead test for keyword returned.

* [instagram] Improve test of InstagramProfilepageExtractor

Also check the count of media returned.

* [instagram] Several cleanup and improvements

- Change description, subcategories to generate a better description in
  docs/supportedsite.rst
- Remove not needed InstagramExtractor.__init__()
- Use text.parse_int() instead of directly using int() (the former is more
  robust)
- Use self.request().json() instead of using json.loads() the
  self.request().text()
- Add `pattern:' to check the URLs where we do not have a stable URLs.
  It seems that only the subdomain is not stable.

Thanks to @mikf!
 2018-12-09 12:52:14 +01:00
Mike Fährmann
ba56827f36
[newgrounds] add user-, video-, image-extractors (#119) 2018-11-27 15:44:53 +01:00
Mike Fährmann
a0ae156edc
[pornreactor] add tag-, user-, post-extractors (#114) 2018-11-23 14:41:26 +01:00
Mike Fährmann
503d42a1c2
[joyreactor] add tag-, user-, post-extractors (#114) 2018-11-23 09:25:02 +01:00
Mike Fährmann
2cf3f53839
[yuki] add thread extractor (closes #111) 2018-09-28 12:47:32 +02:00
Mike Fährmann
c9290d8212
[wallhaven] add wallpaper and search extractors 
todo:
- login support to gain access to NSFW wallpapers
- extractors for tag-, similar-, latest-listings
- skip() support
 2018-09-17 21:26:13 +02:00
Mike Fährmann
69fd61ea86
[bobx] add gallery and model extractors 2018-09-13 20:13:12 +02:00
Mike Fährmann
3ecea4cf36
[hentaicafe] add chapter and manga extractors (#101) 2018-09-05 21:08:40 +02:00
Mike Fährmann
ab2bfaeb46
[ngomik] add replacement for 'subapics' 
http://subapics.com/ got discontinued and replaced by http://ngomik.in/.

ngomik.in is still displaying a link to the "old site" showing a big
"Account Suspended" sign.
 2018-08-23 15:29:53 +02:00
Mike Fährmann
3c90df6635
[piczel] add user, folder and image extractors 2018-08-08 10:53:01 +02:00
Mike Fährmann
df082e923c
[behance] add gallery extractor (#95) 2018-08-01 21:46:55 +02:00
Mike Fährmann
84854fcad7
[myportfolio] add user and gallery extractors (#95) 2018-07-19 18:56:45 +02:00
Mike Fährmann
a6fe2bb594
[whatisthisimnotgoodwithcomputers] remove extractor 2018-07-14 09:53:16 +02:00