Mike Fährmann
4357966a70
[kissmanga] make URL pattern case-insensitive (fixes 28)
7 years ago
Mike Fährmann
7aa9fa796a
code cleanup and fixes
7 years ago
Mike Fährmann
f08af03845
Merge branch 'cookies'
7 years ago
Mike Fährmann
55f048d02b
ignore case of cookiejar magic strings
7 years ago
Mike Fährmann
f53bf1a323
[thebarchive] add thread extractor
7 years ago
Mike Fährmann
b8cf434bb0
[rebeccablacktech] add thread extractor
7 years ago
Mike Fährmann
808f67ba7d
use 'cookiedomain' for cookies set by object-config-values
...
otherwise these cookies would not be picked up by the
_check_cookies() method.
7 years ago
Mike Fährmann
390eeded4c
[mangazuki] support 'raws.…' subdomain
7 years ago
Mike Fährmann
4a60f6068a
[mangazuki] add manga extractor
7 years ago
Mike Fährmann
394241cd6f
[2chan] fix extraction
7 years ago
Mike Fährmann
a13eb6010f
[fallenangels] fix extraction of chapter URLs
7 years ago
Mike Fährmann
1cb1d2e0a3
[mangazuki] add chapter extractor
7 years ago
Mike Fährmann
2f2e363c97
[imgur] use /a/<key>/all as album-url
7 years ago
Mike Fährmann
1cec03c9c6
[imgur] fix extraction of large albums
7 years ago
Mike Fährmann
0610ae5000
skip login if cookies are present
7 years ago
Mike Fährmann
f105782435
[fireden] add thread extractor
7 years ago
Mike Fährmann
c93f7d7496
[archiveofsins] add thread extractor
7 years ago
Mike Fährmann
96e13604da
[archivedmoe] add thread extractor
7 years ago
Mike Fährmann
30d3a5f9b2
support redirects on 4chan archives
7 years ago
Mike Fährmann
98464d1f1b
[loveisover] add thread extractor
7 years ago
Mike Fährmann
47692f28da
[2chan] add thread extractor
7 years ago
Mike Fährmann
3460dc8950
update gallery-dl.conf
7 years ago
Mike Fährmann
9be8f7e106
[deviantart] add "extractor.deviantart.flat" option
...
Setting this to 'false' downloads images into individual subdirectories
for each gallery-folder or favourite-collection, otherwise it is just
creating a flat list of images.
7 years ago
Mike Fährmann
d075627fd9
[deviantart] support group galleries ( #26 )
...
For groups the 'GalleryExtractor' collects all gallery-folder URLs
and defers its work to the 'FolderExtractor'.
7 years ago
Mike Fährmann
b37a62501b
[pixiv] unquote tags
7 years ago
Mike Fährmann
fbd7dcdfdb
[desuarchive] add thread extractor
7 years ago
Mike Fährmann
af9bd17b19
[deviantart] adjust default paths
...
- user.deviantart.com/(gallery|favourites|journal)/ images go into
* <user>/
* <user>/Favourites/
* <user>/Journal/
(having an extra "Gallery" folder for a user's gallery-images seems
a bit too much if these are all you want to download, which is
probably the default use-case)
- single "deviations" (user.deviantart.com/(art|journal)/name-123) go
into their owner's directory:
* <user>/
(putting them into their own directory seems weird in practice)
7 years ago
Mike Fährmann
eb64fb267c
[nyafuu] add thread extractor ( #18 )
7 years ago
Mike Fährmann
726c6f01ae
allow 'cookies' config option to be a dictionary
7 years ago
Mike Fährmann
4877ef6314
[deviantart] support '?catpath=/' URLs ( #26 )
...
They previously weren't supported for galleries and journals.
This also increases the 'limit' parameter for API calls to its
respective maximum.
7 years ago
Mike Fährmann
8c16cbe7ea
fix tests
7 years ago
Mike Fährmann
a6f689e01a
[deviantart] add gallery-folder extractor ( #26 )
...
The code for this and the available metadata is probably going
to change again. This extractor is very similar to the favorite-
extractor, so they might be "combined" or something like that.
7 years ago
Mike Fährmann
474e9c1aec
[4plebs] add thread extractor ( #18 )
7 years ago
Mike Fährmann
a804a42e23
add '--cookies' command-line option
7 years ago
Mike Fährmann
dcc1d3b2ea
[hentaifoundry] fix infinite loop for multiple of 25 images
7 years ago
Mike Fährmann
34e6e1099e
[batoto] adapt to https chapter URLs
7 years ago
Mike Fährmann
85696d0b3b
[reddit] fix issue with datetime errors
7 years ago
Mike Fährmann
80c2e03aaa
[reddit] allow 'date-min/max' to be human readable dates
...
If the date-min/max config value is a string, try parsing it using
datetime.strptime [1] with 'date-format' as format string [2]
(default: "%Y-%m-%dT%H:%M:%S")
Example: get all submissions posted in 2016
$ gallery-dl reddit.com/r/... \
-o date-format=%Y \
-o date-min=\"2016\" \
-o date-max=\"2017\"
[1] https://docs.python.org/3/library/datetime.html#datetime.datetime.strptime
[2] https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior
7 years ago
Mike Fährmann
58e95a7487
share extractor and downloader sessions
...
There was never any "good" reason for the strict separation
between extractors and downloaders. This change allows for
reduced resource usage (probably unnoticeable) and less lines
of code at the "cost" of tighter coupling.
7 years ago
Mike Fährmann
4414aefe97
small fix for aes_cbc_decrypt_text
7 years ago
Mike Fährmann
21064146c1
fix test
7 years ago
Mike Fährmann
f3d0373120
[reddit] add ability to filter by submission id
...
'extractor.reddit.id-min' and '….id-max' specify the lowest and
highest submission-/post-id to consider, similar to 'date-min' and
'date-max'
7 years ago
Mike Fährmann
1dac76fd1c
update extractor docstrings
7 years ago
Mike Fährmann
92a11528d1
smaller changes
7 years ago
Mike Fährmann
44d98e562b
[pixiv] support pixiv.me URLs ( #23 )
7 years ago
Mike Fährmann
b373fe0eea
[pixiv] support shortened URLs and other variants ( #23 )
7 years ago
Mike Fährmann
c951d6276c
[imagetwist] use https
7 years ago
Mike Fährmann
d3b04076f7
add .netrc support ( #22 )
...
Use the '--netrc' cmdline option or set the 'netrc' config option
to 'true' to enable the use of .netrc authentication data.
The 'machine' names for the .netrc info are the lowercase extractor
names (or categories): batoto, exhentai, nijie, pixiv, seiga.
7 years ago
Mike Fährmann
e1d82af5e0
small fixes
7 years ago
Mike Fährmann
719d45f89e
[flickr] allow the use of Flickr's specifiers for format selection
...
- renamed the 'width-max' option to 'size-max'
- filter by both width and height
7 years ago
Mike Fährmann
b4c438c9ad
[oauth] add the 'extractor.oauth.browser' option
...
enables/disables the use of webbrowser.open() during OAuth authorization
7 years ago
Mike Fährmann
2633337833
[kissmanga] update regex ( fixes #20 )
7 years ago
Mike Fährmann
e68af4febe
[flickr] add 'width-max' option ( #16 )
...
This option allows for simple format selection by
specifying a maximum image width.
7 years ago
Mike Fährmann
2993206c4b
smaller fixes and "security" measures
...
- move the OAuthSession class into util.py
- block special extractors for reddit and recursive
- ignore 'only matching' tests for testresults script
7 years ago
Mike Fährmann
8d5e92f641
resolve cyclic dependency between oauth and flickr
7 years ago
Mike Fährmann
d60781de7b
[oauth] workaround for ctrl+c on Windows
7 years ago
Mike Fährmann
9759fe8c6b
allow 'only_matching' tests
7 years ago
Mike Fährmann
56bec79e6a
[reddit] add ability to load more comments ( #15 )
...
The 'extractor.reddit.morecomments' option enables the use of
the '/api/morechildren' API endpoint (1) to load even more
comments than the usual submission-request provides.
Possible values are the booleans 'true' and 'false' (default).
Note: this feature comes at the cost of 1 extra API call towards
the rate limit for every 100 extra comments.
(1) https://www.reddit.com/dev/api/#GET_api_morechildren
7 years ago
Mike Fährmann
05ed95e5b0
[flickr] add search extractor
7 years ago
Mike Fährmann
5f55c854b9
[flickr] replace getPublic... API call with regular ones
7 years ago
Mike Fährmann
9a620784f9
[flickr] add support for user authentication ( #16 )
...
Call '$ gallery-dl oauth:flickr' to get an access_token
and access_token_secret for your account.
7 years ago
Mike Fährmann
d5a70f2580
add simple progress indicator for multiple URLs ( #19 )
...
The output can be configured via the 'output.progress'
config value.
Possible values:
- true: Show the default progress indicator
"[{current}/{total}] {url}" (default)
- false: Never show the progress indicator
- <string>: Show the progress indicator using this
as a custom format string(1).
Possible replacement keys are:
- current: current URL index
- total : total number of URLs
- url : current URL
(1) https://docs.python.org/3/library/string.html#formatstrings
7 years ago
Mike Fährmann
3ee77a0902
[oauth] print URL if webbrowser.open fails
7 years ago
Mike Fährmann
090e11b35d
[reddit] enable user authentication with OAuth2 ( #15 )
...
Call '$ gallery-dl oauth:reddit' to get a refresh_token
for your account.
7 years ago
Mike Fährmann
e682e06518
[flickr] add group extractor ( #16 )
7 years ago
Mike Fährmann
8fd66ef0b3
[flickr] add gallery extractor ( #16 )
7 years ago
Mike Fährmann
8456b84a12
fix tests and small stuff
7 years ago
Mike Fährmann
fbfc8d0f78
[reddit] ignore Authorization errors for subreddits
...
- also made the limit for retrieved comments customizable via
the 'extractor.reddit.comments' config value
- default is 500; 0 ignores comments completely
7 years ago
Mike Fährmann
e365f1d799
[pixiv] rewrite
...
- same functionality, better(?) code quality, easier to extend
- added test for the user-tag functionality
- removed the 'artist-id', 'artist-name' and 'artist-nick'
keywords, which can be replaced with 'user[id]', 'user[name]'
and 'user[account]' respectively
7 years ago
aiasdfd
338f79147f
[pixiv] support tag for user downloads ( #17 )
...
[pixiv] support tag for user downloads
7 years ago
Mike Fährmann
5f05543f23
[reddit] support filtering by timestamp ( #15 )
...
- Added the 'extractor.reddit.date-min' and '….date-max'
config options. These values should be UTC timestamps.
- All submissions not posted in date-min <= T <= date-max
will be ignored.
- Fixed the limit parameter for submission comments by setting
it to its apparent max value (500).
7 years ago
Mike Fährmann
4e80e0c884
[flickr] add user extractor ( #16 )
7 years ago
Mike Fährmann
b81d068a6d
[flickr] add favorites extractor ( #16 )
7 years ago
Mike Fährmann
c921b4f32a
code cleanup and fixing tests
7 years ago
Mike Fährmann
72f1c6f87a
[flickr] add support for flic.kr/p/... URLs
...
Example:
https://flic.kr/p/FPVo9U
7 years ago
Mike Fährmann
93e5d8cba3
[flickr] add album extractor
7 years ago
Mike Fährmann
659c65dbb0
[flickr] add image extractor
7 years ago
Mike Fährmann
b6fffa9e26
[directlink] update filename format and metadata
7 years ago
Mike Fährmann
c184e47ee3
put common directory- and filename formats in base classes
7 years ago
Mike Fährmann
bce51e90a5
[reddit] support sorting options and sub-options ( #15 )
...
Example:
https://www.reddit.com/r/ <subreddit>/top/?sort=top&t=month
(the 'sort=top' parameter is irrelevant and can be omitted)
7 years ago
Mike Fährmann
5f45ce2930
[gfycat] add "format" config key to select a video format
...
Possible values:
- one of "mp4" (default), "webm", "gif", "webp", "mjpg"
If the selected format is not available, "mp4", "webm" and "gif"
(in that order) will be tried instead, until an available format
is found.
7 years ago
Mike Fährmann
011659ced5
[imgur] add "mp4" config key to decide between GIF and MP4
...
possible values:
- false : always use GIF
- true : use MP4 if "prefer_video" flag is set,
GIF otherwise (default)
- "always": always use MP4
7 years ago
Mike Fährmann
48ccee2505
[gfycat] add image extractor
7 years ago
Mike Fährmann
bf452a8516
[imgur] choose .mp4 over .gif if available
7 years ago
Mike Fährmann
f79320e35b
fix tests
7 years ago
Mike Fährmann
67791e1b36
[imgur] improve and add image extractor
7 years ago
Mike Fährmann
99b72130ee
[reddit] enable recursion ( #15 )
...
reddit extractors now recursively visit other submissions/posts
linked to in the initial set of submissions.
This behaviour can be configured via the 'extractor.reddit.recursion'
key in the configuration file or by `-o recursion=<value>`.
Example:
{"extractor": {
"reddit": {
"recursion": <value>
}}}
Possible values:
* -1 - infinite recursion (don't do this)
* 0 - recursion is disabled (default)
* 1 and higher - maximum recursion level
7 years ago
Mike Fährmann
691c4dd709
support direct image links
7 years ago
Mike Fährmann
d2dceb35b7
implement context-manager to blacklist extractors
7 years ago
Mike Fährmann
e425243b1e
[reddit] some small fixes
...
- filter or complete some URLs
- remove the 'nofollow:' scheme before printing URLs
- (#15 )
7 years ago
Mike Fährmann
a22892f494
[reddit] add subreddit- and submission-extractor
...
- these extractors scan submissions and their comments for
(external) URLs and defer them to other extractors
- (#15 )
7 years ago
Mike Fährmann
832a4a8ee9
[fallenangels] add manga extractor
7 years ago
Mike Fährmann
f226417420
simplify code by using a MangaExtractor base class
7 years ago
Mike Fährmann
2974d782a3
[yomanga] remove module
...
site has been shut down
7 years ago
Mike Fährmann
cbb4323f66
add setup.cfg to configure flake8
7 years ago
Mike Fährmann
232fe2dd08
improve the test extractor
7 years ago
Mike Fährmann
b0131ea402
[fallenangels] support this site's Vietnamese version
...
- https://truyen.fascans.com/
7 years ago
Mike Fährmann
b6b214f7e9
[deviantart] fix headers for custom-style journals
...
example: http://shimoda7.deviantart.com/journal/Temporary-absence-231936282
7 years ago
Mike Fährmann
e9a2738257
[deviantart] support images on top of journal entries
...
example: http://raxnae.deviantart.com/art/Kami-s-Journal-679482236
7 years ago
Mike Fährmann
92597f46d4
[deviantart] add title to journals
7 years ago
Mike Fährmann
107d29ad8a
improve handling of text:... URLs
...
- don't require // after the colon
- open output files in text mode
7 years ago
Mike Fährmann
677c8ced11
[deviantart] add "journal" extractor
...
(#14 )
7 years ago
Mike Fährmann
e5f79ae839
[deviantart] add support for all media types
...
- this includes
- images
- videos
- flash-animations
- journals
- also renamed some of the extractors
- User -> Gallery
- Image -> Deviation
7 years ago
Mike Fährmann
9f1c83297f
[pinterest] allow URLs with any TLD
7 years ago
Mike Fährmann
b3b92ac243
[deviantart] support "All" favorites and add "mature" option
...
- since there is apparently no actual way to get the "All" favorites
listing via API, corresponding URLs (.../favourites/?catpath=/) will
be handled by yielding all deviations from all favorite collections of
that user
- the "mature" config key works on a per extractor basis (like "username"
or "password"). values can be the strings "true" or "false", or the
booleans true or false.
- (#14 )
7 years ago
Mike Fährmann
7376ad7f3d
[deviantart] turn the "Mature Content Filter" off
...
(#14 )
7 years ago
Mike Fährmann
cfbf79d788
[pixiv] fix login
7 years ago
Mike Fährmann
85a46ed700
[booru] fix issue with multiple tags
7 years ago
Mike Fährmann
fc9223c072
add '--abort-on-skip' option and ability to control skip behavior
...
the 'skip' config option controls skipping behavior:
true - skip download if file already exist (default)
false - download and overwrite files even if it exists
"abort" - abort extractor run if a download would be skipped
(same as '--abort-on-skip')
7 years ago
Mike Fährmann
d948ba1322
[readcomics] remove module
...
- site has been unavailable for two weeks
- (#12 )
7 years ago
Mike Fährmann
a610b35a0d
[mangashare] remove module
...
this site has been unavailable for at least two months
7 years ago
Mike Fährmann
4e8587bad4
[pixiv] add support for https://i.pximg.net URLs
7 years ago
Mike Fährmann
e41efbd2d9
[kissmanga] fix edge-case
7 years ago
Mike Fährmann
ffd72424bf
[kissmanga] another attempt at getting the AES key
8 years ago
Mike Fährmann
af56887a47
[exhentai] fall back to e-hentai if no username is given
8 years ago
Mike Fährmann
4b967fa189
implement and use extractor.config() method
8 years ago
Mike Fährmann
82ab1fca07
[seiga] reduce cache maxage to one week
8 years ago
Mike Fährmann
ec48d25afc
[pawoo] fix extraction results
8 years ago
Mike Fährmann
244ab75cad
[kissmanga] update AES key retrieval
8 years ago
Chen John L
a5485a46cb
fixed the module for pixhost
8 years ago
Mike Fährmann
13dc5d72bc
update some extractors to use https
8 years ago
Mike Fährmann
342371086b
[pawoo] add extractors for accounts and statuses
...
https://pawoo.net is a Mastodon[1] instance hosted by Pixiv
[1] https://github.com/tootsuite/mastodon
8 years ago
Mike Fährmann
0770de0ea1
[deviantart:image] add support for sta.sh URLs
8 years ago
Mike Fährmann
f4aa452bd1
update unit test results
8 years ago
Mike Fährmann
71e08dc9c4
[tumblr] keyword consistency
8 years ago
Mike Fährmann
c9a5650cf8
add manga extractors to all foolslide-based modules
8 years ago
Mike Fährmann
bd95fea82c
update unit test results
8 years ago
Mike Fährmann
0456efaa5a
[hentaifoundry] update unit tests
8 years ago
Mike Fährmann
fd3f84d913
[kissmanga] get AES key from site itself
8 years ago
Mike Fährmann
fece09d326
[fallenangels] update to new domain and site-layout
8 years ago
Mike Fährmann
cf79a47b59
update unit tests
8 years ago
Mike Fährmann
e23e4b1100
[seaotterscans] add extractor
8 years ago
Mike Fährmann
616464f8f8
[kireicake] add extractor
8 years ago
Mike Fährmann
f537ad5f2f
[kissmanga] re-enable module
8 years ago
Mike Fährmann
b603b592cf
[exhentai] accept "e-hentai.org" URLs ( #11 )
8 years ago
Mike Fährmann
11ce871718
[deviantart] add favourites-extractor
8 years ago
Mike Fährmann
d430732502
[deviantart] update extractors
...
- use base class for all extractors
- use API for single images
- add support for more API endpoints
- add additional keywords
8 years ago
Mike Fährmann
298d7c45f7
[nijie] support multi-page image listings
8 years ago
Mike Fährmann
91ad8ac2b3
[mangastream] adapt to domain-change to readms.net
8 years ago
Mike Fährmann
841fd50242
move code into util.py
8 years ago
Mike Fährmann
6cbb078392
[batoto] fix cache time for login sessions
...
from 1 year to 1 week ...
8 years ago
Mike Fährmann
6fd9e66223
[whentai] remove module
...
apparently they changed their system and now you can't download the
original images without an account with VIP status.
8 years ago
Mike Fährmann
dce5bce74e
[kissmanga] temporarily disable module
...
... until I've figured out how to decode their image urls
8 years ago
Mike Fährmann
4e7661ab01
[imgtrex] re-add extractor
8 years ago
Mike Fährmann
0257d3e7ac
[mangamint] remove extractors - site is down
8 years ago
Mike Fährmann
1d46be545c
add login notifications
8 years ago
Mike Fährmann
ed94d9b92d
fix/improve various things
8 years ago
Mike Fährmann
619c74159a
[seiga] fix file extension and xml parsing
...
- The file extension of the first image had been used for all further
images
- API responses can contain invalid characters, which cause the XML
parser to fail (http://seiga.nicovideo.jp/user/illust/26377934
contains several \x08 characters)
8 years ago
Mike Fährmann
66eb3d3488
fix tests
8 years ago
Mike Fährmann
f622411be5
[deviantart] implement 'skip' method
8 years ago
Mike Fährmann
dfe2c2dced
[batoto] update test results
8 years ago
Mike Fährmann
43e3bb24ae
[imagefap] don't rely on image-count
...
(fixes #9 )
8 years ago
Mike Fährmann
379125746a
[deviantart] improve API error handling
...
(#7 )
8 years ago
Mike Fährmann
f782282f97
add logger objects to extractors
8 years ago
Mike Fährmann
2c2de778f2
[deviantart] adjust error message
8 years ago
Mike Fährmann
c24aeb5cbc
[whentai] update test results
8 years ago
Mike Fährmann
f36d685ce1
[deviantart] always request access token before api calls
...
(#7 )
8 years ago
Mike Fährmann
ff92674379
[booru] simple skip functionality
...
(#6 )
8 years ago
Mike Fährmann
7a9d66fbce
implement basic way to tell extractors to skip ahead
8 years ago
Mike Fährmann
872b8aed97
[4chan] update test
8 years ago
Mike Fährmann
f361cb13e0
[hentai2read] fix extraction
8 years ago
Mike Fährmann
38aa496ecb
[deviantart] restore tests + set api version
8 years ago
Mike Fährmann
39b1a18f12
[deviantart] fix issue with missing 'content'-key
8 years ago
Mike Fährmann
e87e6fbc67
change some config keys
...
directory_fmt -> directory
filename_fmt -> filename
download-original -> original
8 years ago
Mike Fährmann
96baea0959
[hentaibox] remove extractor
8 years ago
Mike Fährmann
9d36acbbc4
[yonkouprod] update test results
8 years ago
Mike Fährmann
9a08f8a097
improved foolslide-based extractors
...
- this includes dokireader, fallenangels, jaiminisbox, powermanga,
sensescans, worldthree, yonkouprod, gomanga, yomanga
- added 'chapter_string', 'chapter_id', 'chapter_minor' and 'count'
keywords
- changed the 'chapter' keyword to always be just a number
- changed the default directory format
8 years ago
Mike Fährmann
0a6487afe8
[exhentai] fix detection of invalid gallery keys
8 years ago
Mike Fährmann
0b59d9f8c7
disable urllib3s InsecureConnectionWarning
8 years ago
Mike Fährmann
bad623751f
[hentaihere] transition to https
8 years ago
Mike Fährmann
331b413c60
update extractor test script
8 years ago
Mike Fährmann
c84e975dcb
[twitter] fix image extraction
8 years ago
Mike Fährmann
0af02007a9
[imagetwist] fix site access
8 years ago
Mike Fährmann
2f6e3874a7
[sensescans support http://sensescans.com/reader/ ... URLs
8 years ago
Mike Fährmann
52104b2bb6
[fallenangels] add chapter extractor
8 years ago
Mike Fährmann
7880cc1ad7
[imgtrex] remove extractor - domain no longer exists
8 years ago
Mike Fährmann
21e0dfbe20
[chronos] raise NotFoundError instead of crashing
8 years ago
Mike Fährmann
2b38398940
[imgyt] raise NotFoundError instead of crashing
8 years ago
Mike Fährmann
94e10f249a
code adjustments according to pep8 nr2
8 years ago
Mike Fährmann
f1b7d41608
[mangamint] fix keyword extraction
8 years ago
Mike Fährmann
8e93633319
[nhentai] fix image urls
8 years ago
Mike Fährmann
d82508f245
fix tests for turboimagehost and pinterest
8 years ago
Mike Fährmann
3434c5a026
[pinterest] add support for pin.it URLs
8 years ago
Mike Fährmann
37d4d07d9b
compatibility fixes to make a standalone exe work
8 years ago
Mike Fährmann
4553a6392f
[whentai] add unittests
8 years ago
Mike Fährmann
434ef4d5a2
[mangafox] raise proper exception if chapter is not available
8 years ago
Mike Fährmann
98fa5cf2fa
[mangafox] add chapter extractor
8 years ago
Mike Fährmann
cc0b4f2661
[yomanga] add chapter extractor
8 years ago
Mike Fährmann
ad4b02508f
trying to understand travis-ci unit test failures
...
- added some debug output via logging module
- unit tests work on my machine (tm)
8 years ago
Mike Fährmann
72d3ca0bf9
[deviantart] use API for user-extractor
8 years ago
Mike Fährmann
00074a71d7
several changes to make travis build work
...
- fixed html.unescape not being available on Python3.3
- removed inconsistent test result
- added username/password pairs for authenticating extractors
8 years ago
Mike Fährmann
cfe5bf732a
[gomanga] add chapter extractor
8 years ago
Mike Fährmann
4a8d74973c
adjust login methods to a specific style
8 years ago
Mike Fährmann
98d6d131a3
update unittests for batoto and yonkouprod
8 years ago
Mike Fährmann
468c9aeeec
make tests more reliable
8 years ago
Mike Fährmann
33c799b2ae
[yonkouprod] rename 'yonkou' to 'yonkouprod'
8 years ago
Mike Fährmann
95986ed566
[seiga] add user extractor
8 years ago
Mike Fährmann
efaba5f398
[luscious] support new album URLs + small fixes
8 years ago
Mike Fährmann
79ea457e24
[luscious] use new 'html' attribute in API responses
8 years ago
Mike Fährmann
a8d716c043
[pinterest] tests + fix error message
8 years ago
Mike Fährmann
a849d8f2f7
add a few more tests
8 years ago
Mike Fährmann
7952b8d18d
add a few tests expecting exceptions
8 years ago
Mike Fährmann
efdc299547
[hentaifoundry] get artist name from webpage
8 years ago
Mike Fährmann
1aeb60d997
[jaiminisbox] add extractor
8 years ago
Mike Fährmann
8b2024a1a5
[hentaifoundry] support direct links to images
8 years ago
Mike Fährmann
c604a65b88
[imgyt] use token as filename if none is given
8 years ago
Mike Fährmann
3a7421a6ce
[twitter] get 'original' instead of 'large' image
8 years ago
Mike Fährmann
cb180507ab
[batoto] add manga extractor
8 years ago
Mike Fährmann
ff2a65d5c1
[exhentai] raise proper exception for 'unavailable' galleries
8 years ago
Mike Fährmann
f69209871d
[kisscomic] change some names from 'manga' to 'comic'
8 years ago
Mike Fährmann
d623b9bfdc
[readcomics] code cleanup + tests
8 years ago
Mike Fährmann
c19241e299
[readcomics] add comic extractor
8 years ago
Mike Fährmann
179d62023c
[readcomics] add comic-issue extractor
8 years ago
Mike Fährmann
d12a04af3c
[test] filter out classes without a 'test' attribute
8 years ago
Mike Fährmann
40dbea7ed2
rewrite parts of the cloudflare bypass system
8 years ago
Mike Fährmann
3c4155c9c9
[kisscomic] add extractors
8 years ago
Mike Fährmann
06c798d18e
[yonkou] add chapter extracter
8 years ago
Mike Fährmann
effdf9c3be
[luscious] adapt to API changes
8 years ago
Mike Fährmann
4c55275305
update tests
8 years ago
Mike Fährmann
69cde97c93
add utility extractor that runs test-URLs
8 years ago
Mike Fährmann
b0e8daf415
[imgclick] remove extractor - uses captcha
8 years ago
Mike Fährmann
0211ec4114
update some tests
8 years ago
Mike Fährmann
dfd1992a2c
[hentaifoundry] small updates
...
- throw an exception if an user or image does not exist
- update tests, since the user of the old ones left
8 years ago
Mike Fährmann
2fae0b1803
[fapat] add extractor
8 years ago
Mike Fährmann
583f1b8bbb
[postimg] add extractor
8 years ago
Mike Fährmann
d1cd9acf54
[pixhost] adjust to new site layout
8 years ago
Mike Fährmann
0989cd2430
add basic support for file:// URLs
...
this allows you to feed local files into the recursive extractor
8 years ago
Mike Fährmann
91dff9827f
[luscious] always provide URLs to the original images
8 years ago
Mike Fährmann
d402e644bf
update tests
8 years ago
Mike Fährmann
46440fda2d
[imagevenue] add extractor
8 years ago
Mike Fährmann
2449825d53
[kissmanga] solve cloudflare challenge on demand
8 years ago
Mike Fährmann
9e3788175e
implement decorator for cloudflare bypass
...
this method for enabling and caching a cloudflare bypass for a
requests.session object allows for different cache-timeouts for
different domains
8 years ago
Mike Fährmann
6e98538d36
[mangamint] small fixes and additions
...
- force https
- update patterns to also catch non-standard URLs to manga and chapters
- handle URLs to non-existing resources gracefully
- fix terminating condition for chapter listings
- add a base class for common code
- add and update tests
8 years ago
Mike Fährmann
c333bc33e3
[mangapark] small fixes and additions
...
- add a 'title' keyword for chapter-titles and update the directory
format accordingly
- add a 'type' keyword to distinguish between manga and manhwa
- fix an issue where an exception would be thrown if a chapter number
did not have any special additions (2.5, 55a, v2, etc.)
- add a test-case without a special chapter number
- unescape manga title
8 years ago
Mike Fährmann
b634ace39e
[readcomiconline] add comic-issue and comic extractor
8 years ago
Mike Fährmann
99440ca51a
[imgtrial] add extractor
8 years ago
Mike Fährmann
5f2824dfe6
[imgspot] add extractor
8 years ago
Mike Fährmann
c85adbdc9d
[mangamint] code simplifications
8 years ago
Mike Fährmann
88193718e8
[pixhost] add extractor
8 years ago
Mike Fährmann
07e9e2c4f1
[imgmaid] add extractor
8 years ago
Mike Fährmann
885aa7f7cf
[img4ever] add extractor
8 years ago
Mike Fährmann
5c69a380f0
[kissmanga] code simplifications
8 years ago
Mike Fährmann
4f51d72d22
[imgbox] handle non-existing images/galleries
8 years ago
Mike Fährmann
68fe58ee72
[imgchili] set encoding
8 years ago
Mike Fährmann
58ce0b55e3
[deviantart] make sure to pick the highest quality image
8 years ago
Mike Fährmann
37ea66098a
add tests
8 years ago
Mike Fährmann
653885abf2
[doujinmode] remove extractor - site is down
8 years ago
Mike Fährmann
78b168820d
move another 4 extractors
8 years ago
Mike Fährmann
5e82e8dfbd
move chronos & co. to imagehosts file
8 years ago
Mike Fährmann
8b7f719f36
combine imagehost extractors into a single file
...
added extractors for
- hosturimage.com
- imageontime.org
- imgupload.yt
- imgspice.com
- imgclick.net
All of these would have shared a lot of common code, so i created a base
class for imagehost extractors and put them all in the same file to
avoid clutter.
8 years ago