Mike Fährmann
36ac2197db
[ytdl] add extractor for sites supported by youtube-dl
...
(#1680 , #878 )
Can be used by prefixing any URL with 'ytdl:',
or by setting 'extractor,ytdl.enabled' to 'true'.
2021-07-10 20:55:47 +02:00
Mike Fährmann
36bf76fa44
update 'oauth:mastodon:<instance>' code
2021-01-28 02:20:12 +01:00
Mike Fährmann
350b1afe1c
speed up _list_classes() after iterating over all modules once
2020-10-26 22:18:15 +01:00
Mike Fährmann
3918b69677
remove 'extractor.blacklist' context manager
2020-09-11 13:17:35 +02:00
Mike Fährmann
5df8f2959b
insert local directory into PYTHONPATH when running tests
2020-05-02 01:15:50 +02:00
Mike Fährmann
406449b0d6
ensure keys for mastodon instances are available during tests
...
Calls to config.clear() from other tests are removing the API
credentials set when importing mastodon.py for the first time.
2020-04-08 21:56:14 +02:00
Mike Fährmann
3b50c4f49d
add tests for "Extractors" in oauth.py ( #670 )
2020-04-07 20:26:12 +02:00
Mike Fährmann
04bd0472de
add tests for Extractor.wait()
2020-04-07 20:24:56 +02:00
Mike Fährmann
2a3bd4e3c7
rename extractor classes starting with a digit
2019-11-02 20:42:09 +01:00
Mike Fährmann
64786363be
[4chan] simplify
...
- remove 'chan.py'
- slight adjustments to directory and filenames
2019-11-02 20:11:21 +01:00
Mike Fährmann
6277a739e4
[35photo] add user-, genre-, and image-extractors ( #162 )
2019-03-18 01:11:30 +01:00
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor
2019-02-11 13:31:10 +01:00
Mike Fährmann
9a9cd32461
implement alternative constructor for extractors
2019-02-09 14:42:25 +01:00
Mike Fährmann
abbd45d0f4
update handling of extractor URL patterns
...
When loading extractor classes during 'extractor.find(…)', their
'pattern' attribute will be replaced with a compiled version of itself.
2019-02-08 20:08:16 +01:00
Mike Fährmann
6284731107
simplify extractor constants
...
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
2019-02-08 13:45:40 +01:00
Mike Fährmann
bc0951d974
allow for simplified test data structures
...
Instead of a strict list of (URL, RESULTS)-tuples, extractor result
tests can now be a single (URL, RESULTS)-tuple, if it's just one test,
and "only matching" tests can now be a simple string.
2019-02-06 17:24:44 +01:00
Mike Fährmann
347398f692
fix various tests
2019-02-04 14:40:21 +01:00
Mike Fährmann
b8fed34548
add generalized extractors for Mastodon instances ( #144 )
...
Extractors for Mastodon instances can now be dynamically generated,
based on the instance names in the 'extractor.mastodon.*' config path.
Example:
{
"extractor": {
"mastodon": {
"pawoo.net": { ... },
"mastodon.xyz": { ... },
"tabletop.social": { ... },
...
}
}
}
Each entry requires an 'access-token' value, which can be generated with
'gallery-dl oauth:mastodon:<instance URL>'.
An 'access-token' (as well as a 'client-id' and 'client-secret') for
pawoo.net is always available, but can be overwritten as necessary.
2019-01-19 14:28:59 +01:00
Mike Fährmann
c9e6ccbd7c
[test:extractor] small fixes and improvements
2018-08-15 21:49:33 +02:00
Mike Fährmann
10cc59f3b5
fix extractor names
2018-04-18 18:12:57 +02:00
Mike Fährmann
b1325d4d2c
fix extractor docstrings
2018-04-18 18:03:43 +02:00
Mike Fährmann
dd314279fb
[test] add unit tests for extractor module functions
2018-03-25 11:49:42 +02:00