Mike Fährmann
5f45ce2930
[gfycat] add "format" config key to select a video format
...
Possible values:
- one of "mp4" (default), "webm", "gif", "webp", "mjpg"
If the selected format is not available, "mp4", "webm" and "gif"
(in that order) will be tried instead, until an available format
is found.
7 years ago
Mike Fährmann
011659ced5
[imgur] add "mp4" config key to decide between GIF and MP4
...
possible values:
- false : always use GIF
- true : use MP4 if "prefer_video" flag is set,
GIF otherwise (default)
- "always": always use MP4
7 years ago
Mike Fährmann
48ccee2505
[gfycat] add image extractor
7 years ago
Mike Fährmann
bf452a8516
[imgur] choose .mp4 over .gif if available
7 years ago
Mike Fährmann
f79320e35b
fix tests
7 years ago
Mike Fährmann
67791e1b36
[imgur] improve and add image extractor
7 years ago
Mike Fährmann
99b72130ee
[reddit] enable recursion ( #15 )
...
reddit extractors now recursively visit other submissions/posts
linked to in the initial set of submissions.
This behaviour can be configured via the 'extractor.reddit.recursion'
key in the configuration file or by `-o recursion=<value>`.
Example:
{"extractor": {
"reddit": {
"recursion": <value>
}}}
Possible values:
* -1 - infinite recursion (don't do this)
* 0 - recursion is disabled (default)
* 1 and higher - maximum recursion level
7 years ago
Mike Fährmann
691c4dd709
support direct image links
7 years ago
Mike Fährmann
d2dceb35b7
implement context-manager to blacklist extractors
7 years ago
Mike Fährmann
e425243b1e
[reddit] some small fixes
...
- filter or complete some URLs
- remove the 'nofollow:' scheme before printing URLs
- (#15 )
7 years ago
Mike Fährmann
a22892f494
[reddit] add subreddit- and submission-extractor
...
- these extractors scan submissions and their comments for
(external) URLs and defer them to other extractors
- (#15 )
7 years ago
Mike Fährmann
832a4a8ee9
[fallenangels] add manga extractor
7 years ago
Mike Fährmann
f226417420
simplify code by using a MangaExtractor base class
7 years ago
Mike Fährmann
2974d782a3
[yomanga] remove module
...
site has been shut down
7 years ago
Mike Fährmann
cbb4323f66
add setup.cfg to configure flake8
7 years ago
Mike Fährmann
232fe2dd08
improve the test extractor
7 years ago
Mike Fährmann
b0131ea402
[fallenangels] support this site's Vietnamese version
...
- https://truyen.fascans.com/
7 years ago
Mike Fährmann
b6b214f7e9
[deviantart] fix headers for custom-style journals
...
example: http://shimoda7.deviantart.com/journal/Temporary-absence-231936282
7 years ago
Mike Fährmann
e9a2738257
[deviantart] support images on top of journal entries
...
example: http://raxnae.deviantart.com/art/Kami-s-Journal-679482236
7 years ago
Mike Fährmann
92597f46d4
[deviantart] add title to journals
7 years ago
Mike Fährmann
107d29ad8a
improve handling of text:... URLs
...
- don't require // after the colon
- open output files in text mode
7 years ago
Mike Fährmann
677c8ced11
[deviantart] add "journal" extractor
...
(#14 )
7 years ago
Mike Fährmann
e5f79ae839
[deviantart] add support for all media types
...
- this includes
- images
- videos
- flash-animations
- journals
- also renamed some of the extractors
- User -> Gallery
- Image -> Deviation
7 years ago
Mike Fährmann
9f1c83297f
[pinterest] allow URLs with any TLD
7 years ago
Mike Fährmann
b3b92ac243
[deviantart] support "All" favorites and add "mature" option
...
- since there is apparently no actual way to get the "All" favorites
listing via API, corresponding URLs (.../favourites/?catpath=/) will
be handled by yielding all deviations from all favorite collections of
that user
- the "mature" config key works on a per extractor basis (like "username"
or "password"). values can be the strings "true" or "false", or the
booleans true or false.
- (#14 )
7 years ago
Mike Fährmann
7376ad7f3d
[deviantart] turn the "Mature Content Filter" off
...
(#14 )
7 years ago
Mike Fährmann
cfbf79d788
[pixiv] fix login
7 years ago
Mike Fährmann
85a46ed700
[booru] fix issue with multiple tags
7 years ago
Mike Fährmann
fc9223c072
add '--abort-on-skip' option and ability to control skip behavior
...
the 'skip' config option controls skipping behavior:
true - skip download if file already exist (default)
false - download and overwrite files even if it exists
"abort" - abort extractor run if a download would be skipped
(same as '--abort-on-skip')
7 years ago
Mike Fährmann
d948ba1322
[readcomics] remove module
...
- site has been unavailable for two weeks
- (#12 )
7 years ago
Mike Fährmann
a610b35a0d
[mangashare] remove module
...
this site has been unavailable for at least two months
7 years ago
Mike Fährmann
4e8587bad4
[pixiv] add support for https://i.pximg.net URLs
7 years ago
Mike Fährmann
e41efbd2d9
[kissmanga] fix edge-case
8 years ago
Mike Fährmann
ffd72424bf
[kissmanga] another attempt at getting the AES key
8 years ago
Mike Fährmann
af56887a47
[exhentai] fall back to e-hentai if no username is given
8 years ago
Mike Fährmann
4b967fa189
implement and use extractor.config() method
8 years ago
Mike Fährmann
82ab1fca07
[seiga] reduce cache maxage to one week
8 years ago
Mike Fährmann
ec48d25afc
[pawoo] fix extraction results
8 years ago
Mike Fährmann
244ab75cad
[kissmanga] update AES key retrieval
8 years ago
Chen John L
a5485a46cb
fixed the module for pixhost
8 years ago
Mike Fährmann
13dc5d72bc
update some extractors to use https
8 years ago
Mike Fährmann
342371086b
[pawoo] add extractors for accounts and statuses
...
https://pawoo.net is a Mastodon[1] instance hosted by Pixiv
[1] https://github.com/tootsuite/mastodon
8 years ago
Mike Fährmann
0770de0ea1
[deviantart:image] add support for sta.sh URLs
8 years ago
Mike Fährmann
f4aa452bd1
update unit test results
8 years ago
Mike Fährmann
71e08dc9c4
[tumblr] keyword consistency
8 years ago
Mike Fährmann
c9a5650cf8
add manga extractors to all foolslide-based modules
8 years ago
Mike Fährmann
bd95fea82c
update unit test results
8 years ago
Mike Fährmann
0456efaa5a
[hentaifoundry] update unit tests
8 years ago
Mike Fährmann
fd3f84d913
[kissmanga] get AES key from site itself
8 years ago
Mike Fährmann
fece09d326
[fallenangels] update to new domain and site-layout
8 years ago
Mike Fährmann
cf79a47b59
update unit tests
8 years ago
Mike Fährmann
e23e4b1100
[seaotterscans] add extractor
8 years ago
Mike Fährmann
616464f8f8
[kireicake] add extractor
8 years ago
Mike Fährmann
f537ad5f2f
[kissmanga] re-enable module
8 years ago
Mike Fährmann
b603b592cf
[exhentai] accept "e-hentai.org" URLs ( #11 )
8 years ago
Mike Fährmann
11ce871718
[deviantart] add favourites-extractor
8 years ago
Mike Fährmann
d430732502
[deviantart] update extractors
...
- use base class for all extractors
- use API for single images
- add support for more API endpoints
- add additional keywords
8 years ago
Mike Fährmann
298d7c45f7
[nijie] support multi-page image listings
8 years ago
Mike Fährmann
91ad8ac2b3
[mangastream] adapt to domain-change to readms.net
8 years ago
Mike Fährmann
841fd50242
move code into util.py
8 years ago
Mike Fährmann
6cbb078392
[batoto] fix cache time for login sessions
...
from 1 year to 1 week ...
8 years ago
Mike Fährmann
6fd9e66223
[whentai] remove module
...
apparently they changed their system and now you can't download the
original images without an account with VIP status.
8 years ago
Mike Fährmann
dce5bce74e
[kissmanga] temporarily disable module
...
... until I've figured out how to decode their image urls
8 years ago
Mike Fährmann
4e7661ab01
[imgtrex] re-add extractor
8 years ago
Mike Fährmann
0257d3e7ac
[mangamint] remove extractors - site is down
8 years ago
Mike Fährmann
1d46be545c
add login notifications
8 years ago
Mike Fährmann
ed94d9b92d
fix/improve various things
8 years ago
Mike Fährmann
619c74159a
[seiga] fix file extension and xml parsing
...
- The file extension of the first image had been used for all further
images
- API responses can contain invalid characters, which cause the XML
parser to fail (http://seiga.nicovideo.jp/user/illust/26377934
contains several \x08 characters)
8 years ago
Mike Fährmann
66eb3d3488
fix tests
8 years ago
Mike Fährmann
f622411be5
[deviantart] implement 'skip' method
8 years ago
Mike Fährmann
dfe2c2dced
[batoto] update test results
8 years ago
Mike Fährmann
43e3bb24ae
[imagefap] don't rely on image-count
...
(fixes #9 )
8 years ago
Mike Fährmann
379125746a
[deviantart] improve API error handling
...
(#7 )
8 years ago
Mike Fährmann
f782282f97
add logger objects to extractors
8 years ago
Mike Fährmann
2c2de778f2
[deviantart] adjust error message
8 years ago
Mike Fährmann
c24aeb5cbc
[whentai] update test results
8 years ago
Mike Fährmann
f36d685ce1
[deviantart] always request access token before api calls
...
(#7 )
8 years ago
Mike Fährmann
ff92674379
[booru] simple skip functionality
...
(#6 )
8 years ago
Mike Fährmann
7a9d66fbce
implement basic way to tell extractors to skip ahead
8 years ago
Mike Fährmann
872b8aed97
[4chan] update test
8 years ago
Mike Fährmann
f361cb13e0
[hentai2read] fix extraction
8 years ago
Mike Fährmann
38aa496ecb
[deviantart] restore tests + set api version
8 years ago
Mike Fährmann
39b1a18f12
[deviantart] fix issue with missing 'content'-key
8 years ago
Mike Fährmann
e87e6fbc67
change some config keys
...
directory_fmt -> directory
filename_fmt -> filename
download-original -> original
8 years ago
Mike Fährmann
96baea0959
[hentaibox] remove extractor
8 years ago
Mike Fährmann
9d36acbbc4
[yonkouprod] update test results
8 years ago
Mike Fährmann
9a08f8a097
improved foolslide-based extractors
...
- this includes dokireader, fallenangels, jaiminisbox, powermanga,
sensescans, worldthree, yonkouprod, gomanga, yomanga
- added 'chapter_string', 'chapter_id', 'chapter_minor' and 'count'
keywords
- changed the 'chapter' keyword to always be just a number
- changed the default directory format
8 years ago
Mike Fährmann
0a6487afe8
[exhentai] fix detection of invalid gallery keys
8 years ago
Mike Fährmann
0b59d9f8c7
disable urllib3s InsecureConnectionWarning
8 years ago
Mike Fährmann
bad623751f
[hentaihere] transition to https
8 years ago
Mike Fährmann
331b413c60
update extractor test script
8 years ago
Mike Fährmann
c84e975dcb
[twitter] fix image extraction
8 years ago
Mike Fährmann
0af02007a9
[imagetwist] fix site access
8 years ago
Mike Fährmann
2f6e3874a7
[sensescans support http://sensescans.com/reader/ ... URLs
8 years ago
Mike Fährmann
52104b2bb6
[fallenangels] add chapter extractor
8 years ago
Mike Fährmann
7880cc1ad7
[imgtrex] remove extractor - domain no longer exists
8 years ago
Mike Fährmann
21e0dfbe20
[chronos] raise NotFoundError instead of crashing
8 years ago
Mike Fährmann
2b38398940
[imgyt] raise NotFoundError instead of crashing
8 years ago
Mike Fährmann
94e10f249a
code adjustments according to pep8 nr2
8 years ago
Mike Fährmann
f1b7d41608
[mangamint] fix keyword extraction
8 years ago