7a412f5c32
implement generic manga-chapter extractor
7 years ago
cc0c2cca57
[reddit] add extractor for reddit-hosted images ( closes #68 )
7 years ago
e6814aebe2
add 'extractor.*.user-agent' config option
7 years ago
baf8094868
improve Extractor.request()'s retry behavior
7 years ago
16783e327f
[common] fix UnboundLocalError in Extractor.request()
7 years ago
9aecc67841
[common] explicitly handle HTTP status code 429
7 years ago
b319f4bab3
smaller code and text changes
7 years ago
26a866e7d8
implement (sub)category-transfer between extractors ( #41 )
...
ImageFap- and all Manga-Extractors will transfer their (sub)category
values to other extractors instantiated by them, which will in turn
allow those to use options set for their parents.
Example:
ImagefapGalleryExtractors will use options set under
extractor.imagefap.user, if (and only if) they have been instantiated by
a ImagefapUserExtractor; and options from extractor.imagefap.gallery
otherwise.
7 years ago
9c138dfc1f
[common] detect empty HTTP response bodies
7 years ago
deb2e803ba
simplify MangaExtractor class
7 years ago
0dedbe759c
enable '--chapter-filter'
...
The same filter infrastructure that can be applied to image URLS now
also works for manga chapters and other delegated URLs.
TODO: actually provide any metadata (currently supported is only
deviantart and imagefap).
7 years ago
be30fb2f98
add common config category for boorus and foolslide
7 years ago
915a0137de
improve 'extractor.request'
...
- add 'fatal' argument
- improve internal logic and flow
- raise known exception on error
- update exception hierarchy
7 years ago
7aa9fa796a
code cleanup and fixes
7 years ago
55f048d02b
ignore case of cookiejar magic strings
7 years ago
808f67ba7d
use 'cookiedomain' for cookies set by object-config-values
...
otherwise these cookies would not be picked up by the
_check_cookies() method.
7 years ago
0610ae5000
skip login if cookies are present
7 years ago
726c6f01ae
allow 'cookies' config option to be a dictionary
7 years ago
a804a42e23
add '--cookies' command-line option
7 years ago
d3b04076f7
add .netrc support ( #22 )
...
Use the '--netrc' cmdline option or set the 'netrc' config option
to 'true' to enable the use of .netrc authentication data.
The 'machine' names for the .netrc info are the lowercase extractor
names (or categories): batoto, exhentai, nijie, pixiv, seiga.
7 years ago
c184e47ee3
put common directory- and filename formats in base classes
7 years ago
f226417420
simplify code by using a MangaExtractor base class
7 years ago
4b967fa189
implement and use extractor.config() method
8 years ago
f782282f97
add logger objects to extractors
8 years ago
7a9d66fbce
implement basic way to tell extractors to skip ahead
8 years ago
0b59d9f8c7
disable urllib3s InsecureConnectionWarning
8 years ago
37d4d07d9b
compatibility fixes to make a standalone exe work
8 years ago
cc0b4f2661
[yomanga] add chapter extractor
8 years ago
ad4b02508f
trying to understand travis-ci unit test failures
...
- added some debug output via logging module
- unit tests work on my machine (tm)
8 years ago
e6d26f0476
don't overwrite a response's encoding with None
8 years ago
f0f7306db6
re-raise async exceptions in main thread
8 years ago
000df8d1fa
add 'encoding' argument for Extractor.request
8 years ago
81dcfbec90
initial support for extractor-subcategories
9 years ago
1497da07de
remove unused format-strings
9 years ago
3fb5a8b834
delay 'requests'-import
9 years ago
539faa0322
remove SequentialExtractor class
9 years ago
3c13548f29
rewrite extractors to use config-module
9 years ago
42b8e81a68
rewrite extractors to use text-module
9 years ago
5806e02f97
better support for KeyboardInterrupt exceptions
10 years ago
1cd25b5369
[pixiv] update to new extractor interface
10 years ago
cd4a699dd2
add 'Headers' and 'Cookies' message
10 years ago
513808d156
move code from util.py
10 years ago
41f00809ff
update extractor base classes
10 years ago
deef91eddc
initial commit
10 years ago
Mike Fährmann