e1d82af5e0
small fixes
7 years ago
719d45f89e
[flickr] allow the use of Flickr's specifiers for format selection
...
- renamed the 'width-max' option to 'size-max'
- filter by both width and height
7 years ago
b4c438c9ad
[oauth] add the 'extractor.oauth.browser' option
...
enables/disables the use of webbrowser.open() during OAuth authorization
7 years ago
2633337833
[kissmanga] update regex ( fixes #20 )
7 years ago
fac6c02224
[downloader] fix extension from content-type
7 years ago
e68af4febe
[flickr] add 'width-max' option ( #16 )
...
This option allows for simple format selection by
specifying a maximum image width.
7 years ago
2993206c4b
smaller fixes and "security" measures
...
- move the OAuthSession class into util.py
- block special extractors for reddit and recursive
- ignore 'only matching' tests for testresults script
7 years ago
8d5e92f641
resolve cyclic dependency between oauth and flickr
7 years ago
d60781de7b
[oauth] workaround for ctrl+c on Windows
7 years ago
9759fe8c6b
allow 'only_matching' tests
7 years ago
56bec79e6a
[reddit] add ability to load more comments ( #15 )
...
The 'extractor.reddit.morecomments' option enables the use of
the '/api/morechildren' API endpoint (1) to load even more
comments than the usual submission-request provides.
Possible values are the booleans 'true' and 'false' (default).
Note: this feature comes at the cost of 1 extra API call towards
the rate limit for every 100 extra comments.
(1) https://www.reddit.com/dev/api/#GET_api_morechildren
7 years ago
05ed95e5b0
[flickr] add search extractor
7 years ago
5f55c854b9
[flickr] replace getPublic... API call with regular ones
7 years ago
9a620784f9
[flickr] add support for user authentication ( #16 )
...
Call '$ gallery-dl oauth:flickr' to get an access_token
and access_token_secret for your account.
7 years ago
d5a70f2580
add simple progress indicator for multiple URLs ( #19 )
...
The output can be configured via the 'output.progress'
config value.
Possible values:
- true: Show the default progress indicator
"[{current}/{total}] {url}" (default)
- false: Never show the progress indicator
- <string>: Show the progress indicator using this
as a custom format string(1).
Possible replacement keys are:
- current: current URL index
- total : total number of URLs
- url : current URL
(1) https://docs.python.org/3/library/string.html#formatstrings
7 years ago
3ee77a0902
[oauth] print URL if webbrowser.open fails
7 years ago
090e11b35d
[reddit] enable user authentication with OAuth2 ( #15 )
...
Call '$ gallery-dl oauth:reddit' to get a refresh_token
for your account.
7 years ago
e682e06518
[flickr] add group extractor ( #16 )
7 years ago
8fd66ef0b3
[flickr] add gallery extractor ( #16 )
7 years ago
8456b84a12
fix tests and small stuff
7 years ago
fbfc8d0f78
[reddit] ignore Authorization errors for subreddits
...
- also made the limit for retrieved comments customizable via
the 'extractor.reddit.comments' config value
- default is 500; 0 ignores comments completely
7 years ago
e365f1d799
[pixiv] rewrite
...
- same functionality, better(?) code quality, easier to extend
- added test for the user-tag functionality
- removed the 'artist-id', 'artist-name' and 'artist-nick'
keywords, which can be replaced with 'user[id]', 'user[name]'
and 'user[account]' respectively
7 years ago
338f79147f
[pixiv] support tag for user downloads ( #17 )
...
[pixiv] support tag for user downloads
7 years ago
5f05543f23
[reddit] support filtering by timestamp ( #15 )
...
- Added the 'extractor.reddit.date-min' and '….date-max'
config options. These values should be UTC timestamps.
- All submissions not posted in date-min <= T <= date-max
will be ignored.
- Fixed the limit parameter for submission comments by setting
it to its apparent max value (500).
7 years ago
4e80e0c884
[flickr] add user extractor ( #16 )
7 years ago
b81d068a6d
[flickr] add favorites extractor ( #16 )
7 years ago
c921b4f32a
code cleanup and fixing tests
7 years ago
72f1c6f87a
[flickr] add support for flic.kr/p/... URLs
...
Example:
https://flic.kr/p/FPVo9U
7 years ago
93e5d8cba3
[flickr] add album extractor
7 years ago
659c65dbb0
[flickr] add image extractor
7 years ago
b6fffa9e26
[directlink] update filename format and metadata
7 years ago
c184e47ee3
put common directory- and filename formats in base classes
7 years ago
bce51e90a5
[reddit] support sorting options and sub-options ( #15 )
...
Example:
https://www.reddit.com/r/ <subreddit>/top/?sort=top&t=month
(the 'sort=top' parameter is irrelevant and can be omitted)
7 years ago
5f45ce2930
[gfycat] add "format" config key to select a video format
...
Possible values:
- one of "mp4" (default), "webm", "gif", "webp", "mjpg"
If the selected format is not available, "mp4", "webm" and "gif"
(in that order) will be tried instead, until an available format
is found.
7 years ago
011659ced5
[imgur] add "mp4" config key to decide between GIF and MP4
...
possible values:
- false : always use GIF
- true : use MP4 if "prefer_video" flag is set,
GIF otherwise (default)
- "always": always use MP4
7 years ago
48ccee2505
[gfycat] add image extractor
7 years ago
25bcdc8aa9
add `--write-unsupported` option ( #15 )
7 years ago
bf452a8516
[imgur] choose .mp4 over .gif if available
7 years ago
f79320e35b
fix tests
7 years ago
67791e1b36
[imgur] improve and add image extractor
7 years ago
99b72130ee
[reddit] enable recursion ( #15 )
...
reddit extractors now recursively visit other submissions/posts
linked to in the initial set of submissions.
This behaviour can be configured via the 'extractor.reddit.recursion'
key in the configuration file or by `-o recursion=<value>`.
Example:
{"extractor": {
"reddit": {
"recursion": <value>
}}}
Possible values:
* -1 - infinite recursion (don't do this)
* 0 - recursion is disabled (default)
* 1 and higher - maximum recursion level
7 years ago
ae686c4c08
run queue items immediately
7 years ago
691c4dd709
support direct image links
7 years ago
d2dceb35b7
implement context-manager to blacklist extractors
7 years ago
30eef527d8
update output logic on error
...
[ci skip]
7 years ago
e425243b1e
[reddit] some small fixes
...
- filter or complete some URLs
- remove the 'nofollow:' scheme before printing URLs
- (#15 )
7 years ago
a22892f494
[reddit] add subreddit- and submission-extractor
...
- these extractors scan submissions and their comments for
(external) URLs and defer them to other extractors
- (#15 )
7 years ago
398506da45
update release script
7 years ago
8db3a2fea8
release version 0.8.4
7 years ago
832a4a8ee9
[fallenangels] add manga extractor
7 years ago
f226417420
simplify code by using a MangaExtractor base class
7 years ago
2974d782a3
[yomanga] remove module
...
site has been shut down
7 years ago
cbb4323f66
add setup.cfg to configure flake8
7 years ago
232fe2dd08
improve the test extractor
7 years ago
b0131ea402
[fallenangels] support this site's Vietnamese version
...
- https://truyen.fascans.com/
7 years ago
a90c6acc9c
code cleanup + fixes
7 years ago
4c88c0d496
rework the output format for --list-keywords
7 years ago
b6b214f7e9
[deviantart] fix headers for custom-style journals
...
example: http://shimoda7.deviantart.com/journal/Temporary-absence-231936282
7 years ago
e9a2738257
[deviantart] support images on top of journal entries
...
example: http://raxnae.deviantart.com/art/Kami-s-Journal-679482236
7 years ago
92597f46d4
[deviantart] add title to journals
7 years ago
107d29ad8a
improve handling of text:... URLs
...
- don't require // after the colon
- open output files in text mode
7 years ago
677c8ced11
[deviantart] add "journal" extractor
...
(#14 )
7 years ago
e5f79ae839
[deviantart] add support for all media types
...
- this includes
- images
- videos
- flash-animations
- journals
- also renamed some of the extractors
- User -> Gallery
- Image -> Deviation
7 years ago
9f1c83297f
[pinterest] allow URLs with any TLD
7 years ago
b3b92ac243
[deviantart] support "All" favorites and add "mature" option
...
- since there is apparently no actual way to get the "All" favorites
listing via API, corresponding URLs (.../favourites/?catpath=/) will
be handled by yielding all deviations from all favorite collections of
that user
- the "mature" config key works on a per extractor basis (like "username"
or "password"). values can be the strings "true" or "false", or the
booleans true or false.
- (#14 )
7 years ago
7376ad7f3d
[deviantart] turn the "Mature Content Filter" off
...
(#14 )
7 years ago
ef90a2de2f
implement the "exit" option for the "skip" config-key
7 years ago
cfbf79d788
[pixiv] fix login
7 years ago
85a46ed700
[booru] fix issue with multiple tags
7 years ago
fc9223c072
add '--abort-on-skip' option and ability to control skip behavior
...
the 'skip' config option controls skipping behavior:
true - skip download if file already exist (default)
false - download and overwrite files even if it exists
"abort" - abort extractor run if a download would be skipped
(same as '--abort-on-skip')
7 years ago
7c8f61a116
release version 0.8.3
7 years ago
d948ba1322
[readcomics] remove module
...
- site has been unavailable for two weeks
- (#12 )
7 years ago
a610b35a0d
[mangashare] remove module
...
this site has been unavailable for at least two months
7 years ago
4e8587bad4
[pixiv] add support for https://i.pximg.net URLs
7 years ago
e41efbd2d9
[kissmanga] fix edge-case
8 years ago
ffd72424bf
[kissmanga] another attempt at getting the AES key
8 years ago
af56887a47
[exhentai] fall back to e-hentai if no username is given
8 years ago
48a5b11204
fix error if no file extension is found
8 years ago
701c016b97
add '-q/--quiet' option
8 years ago
4b967fa189
implement and use extractor.config() method
8 years ago
f0aa35ac84
add '--ignore-config' option
8 years ago
82ab1fca07
[seiga] reduce cache maxage to one week
8 years ago
ec48d25afc
[pawoo] fix extraction results
8 years ago
244ab75cad
[kissmanga] update AES key retrieval
8 years ago
a5485a46cb
fixed the module for pixhost
8 years ago
13dc5d72bc
update some extractors to use https
8 years ago
342371086b
[pawoo] add extractors for accounts and statuses
...
https://pawoo.net is a Mastodon[1] instance hosted by Pixiv
[1] https://github.com/tootsuite/mastodon
8 years ago
5af35ea150
add -v/--verbose option and reduce error verbosity
...
(#12 )
8 years ago
0770de0ea1
[deviantart:image] add support for sta.sh URLs
8 years ago
f4aa452bd1
update unit test results
8 years ago
71e08dc9c4
[tumblr] keyword consistency
8 years ago
b43cd88101
add '-j/--dump-json' option
...
this outputs the extractor-results in JSON format rather then
downloading files
8 years ago
c9a5650cf8
add manga extractors to all foolslide-based modules
8 years ago
bd95fea82c
update unit test results
8 years ago
0456efaa5a
[hentaifoundry] update unit tests
8 years ago
af82467627
release version 0.8.2
8 years ago
fd3f84d913
[kissmanga] get AES key from site itself
8 years ago
fa1cb6f74c
release version 0.8.1
8 years ago
fece09d326
[fallenangels] update to new domain and site-layout
8 years ago
cf79a47b59
update unit tests
8 years ago
e23e4b1100
[seaotterscans] add extractor
8 years ago
616464f8f8
[kireicake] add extractor
8 years ago
8d5f26d530
small fixes to the output module
8 years ago
f537ad5f2f
[kissmanga] re-enable module
8 years ago
c9738ea80b
add an aes implementation
8 years ago
b603b592cf
[exhentai] accept "e-hentai.org" URLs ( #11 )
8 years ago
11ce871718
[deviantart] add favourites-extractor
8 years ago
d430732502
[deviantart] update extractors
...
- use base class for all extractors
- use API for single images
- add support for more API endpoints
- add additional keywords
8 years ago
298d7c45f7
[nijie] support multi-page image listings
8 years ago
91ad8ac2b3
[mangastream] adapt to domain-change to readms.net
8 years ago
e9b445a0f8
update release script
8 years ago
43d8b66e70
release version 0.8.0
8 years ago
841fd50242
move code into util.py
8 years ago
e3212dd98f
fix some smaller stuff
...
- remove support for old windows config paths
- catch exception if cache-database can't be opened
- fix username/password settings for unit tests
- rename variable 'max_tries' to 'retries'
8 years ago
e4b3077168
improve config module
...
- speed improvements, especially in the 'interpolate' function
- 'interpolate' now prioritizes base-level values if they exist
- "username" is chosen before "extractor.<category>.username"
- -u/--username & co can now override config-file values
8 years ago
6cbb078392
[batoto] fix cache time for login sessions
...
from 1 year to 1 week ...
8 years ago
e2b5cd9918
change config-path for 'retries' and 'timeout'
8 years ago
6fd9e66223
[whentai] remove module
...
apparently they changed their system and now you can't download the
original images without an account with VIP status.
8 years ago
cdecd0b37b
add notification when solving cloudflare challenge
8 years ago
dce5bce74e
[kissmanga] temporarily disable module
...
... until I've figured out how to decode their image urls
8 years ago
f2ef49563b
fix argument order for python33 and 34
8 years ago
11d5c6f717
move option parsing to seperate module
8 years ago
0b5076815d
always delete incompletely downloaded files
8 years ago
4e7661ab01
[imgtrex] re-add extractor
8 years ago
0257d3e7ac
[mangamint] remove extractors - site is down
8 years ago
1d46be545c
add login notifications
8 years ago
ed94d9b92d
fix/improve various things
8 years ago
abfe7456d6
add '-R/--retries' and '--http-timeout' options
...
(#10 )
8 years ago
22910f9562
improve error handling of http file downloads
...
(#10 )
8 years ago
80df2b3527
add custom argparse action
8 years ago
619c74159a
[seiga] fix file extension and xml parsing
...
- The file extension of the first image had been used for all further
images
- API responses can contain invalid characters, which cause the XML
parser to fail (http://seiga.nicovideo.jp/user/illust/26377934
contains several \x08 characters)
8 years ago
8816dab549
add 'dev' suffix for non-release versions
...
(#9 )
8 years ago
66eb3d3488
fix tests
8 years ago
f622411be5
[deviantart] implement 'skip' method
8 years ago
27ae152f57
use logging to report errors
8 years ago
dfe2c2dced
[batoto] update test results
8 years ago
43e3bb24ae
[imagefap] don't rely on image-count
...
(fixes #9 )
8 years ago
0cfe51dc78
add '--config-yaml' option
...
(#8 )
8 years ago
379125746a
[deviantart] improve API error handling
...
(#7 )
8 years ago
f782282f97
add logger objects to extractors
8 years ago
0b214eeb9d
bump version to 0.7.0
8 years ago
2c2de778f2
[deviantart] adjust error message
8 years ago
c24aeb5cbc
[whentai] update test results
8 years ago
f36d685ce1
[deviantart] always request access token before api calls
...
(#7 )
8 years ago
ff92674379
[booru] simple skip functionality
...
(#6 )
8 years ago
7a9d66fbce
implement basic way to tell extractors to skip ahead
8 years ago
872b8aed97
[4chan] update test
8 years ago
2fa575b273
restore exception-testing to its old form
8 years ago
f361cb13e0
[hentai2read] fix extraction
8 years ago
40be4933b8
fix exception based tests
8 years ago
24f41e13b3
move some exception handling code
8 years ago
6208d9dd79
implement '--images' and '--chapters' options
...
- the former '--items' has been renamed to '--chapters'
- #6
8 years ago
38aa496ecb
[deviantart] restore tests + set api version
8 years ago
39b1a18f12
[deviantart] fix issue with missing 'content'-key
8 years ago
e87e6fbc67
change some config keys
...
directory_fmt -> directory
filename_fmt -> filename
download-original -> original
8 years ago
2a32b12043
add '--items' option
...
this allows to specify which manga-chapters/comic-issues to download
when using gallery-dl on a manga/comic URL
8 years ago
96baea0959
[hentaibox] remove extractor
8 years ago
3bca866185
rework the '-g' cmdline option
...
the amount of how often the -g option is given now determines up until
what level URLs are resolved.
example:
$ gallery-dl -g http://kissmanga.com/Manga/Dropout
http://kissmanga.com/Manga/Dropout/Ch-000---Oneshot-?id=145847
- when applied to a manga-extractor, specifying the -g option once will
now print a list of all chapter URls
$ gallery-dl -gg http://kissmanga.com/Manga/Dropout
http://2.bp.blogspot.com/.../000.png
http://2.bp.blogspot.com/.../001.png
...
- specifying it twice (or even more often) will go a level deeper and
print the image URLs found in those chapters
8 years ago
9d36acbbc4
[yonkouprod] update test results
8 years ago
9a08f8a097
improved foolslide-based extractors
...
- this includes dokireader, fallenangels, jaiminisbox, powermanga,
sensescans, worldthree, yonkouprod, gomanga, yomanga
- added 'chapter_string', 'chapter_id', 'chapter_minor' and 'count'
keywords
- changed the 'chapter' keyword to always be just a number
- changed the default directory format
8 years ago
0a6487afe8
[exhentai] fix detection of invalid gallery keys
8 years ago
4cb49cfd76
bump version to 0.6.4
8 years ago
0b59d9f8c7
disable urllib3s InsecureConnectionWarning
8 years ago
bad623751f
[hentaihere] transition to https
8 years ago
331b413c60
update extractor test script
8 years ago
c84e975dcb
[twitter] fix image extraction
8 years ago
0af02007a9
[imagetwist] fix site access
8 years ago
2f6e3874a7
[sensescans support http://sensescans.com/reader/ ... URLs
8 years ago
52104b2bb6
[fallenangels] add chapter extractor
8 years ago
7880cc1ad7
[imgtrex] remove extractor - domain no longer exists
8 years ago
21e0dfbe20
[chronos] raise NotFoundError instead of crashing
8 years ago
2b38398940
[imgyt] raise NotFoundError instead of crashing
8 years ago
94e10f249a
code adjustments according to pep8 nr2
8 years ago
f1b7d41608
[mangamint] fix keyword extraction
8 years ago
4f123b8513
code adjustments according to pep8
8 years ago
8e93633319
[nhentai] fix image urls
8 years ago
d82508f245
fix tests for turboimagehost and pinterest
8 years ago
3434c5a026
[pinterest] add support for pin.it URLs
8 years ago
138e39686b
bump version to 0.6.3
8 years ago
37d4d07d9b
compatibility fixes to make a standalone exe work
8 years ago
4553a6392f
[whentai] add unittests
8 years ago
501b64fa9d
add a __main__.py file
8 years ago
434ef4d5a2
[mangafox] raise proper exception if chapter is not available
8 years ago
98fa5cf2fa
[mangafox] add chapter extractor
8 years ago
cc0b4f2661
[yomanga] add chapter extractor
8 years ago
ad4b02508f
trying to understand travis-ci unit test failures
...
- added some debug output via logging module
- unit tests work on my machine (tm)
8 years ago
72d3ca0bf9
[deviantart] use API for user-extractor
8 years ago
989820d719
workaround for python 3.6s behaviour with BEGIN EXCLUSIVE in sqlite
8 years ago
8780abcc77
fix a small spelling error
8 years ago
00074a71d7
several changes to make travis build work
...
- fixed html.unescape not being available on Python3.3
- removed inconsistent test result
- added username/password pairs for authenticating extractors
8 years ago
cfe5bf732a
[gomanga] add chapter extractor
8 years ago
4a8d74973c
adjust login methods to a specific style
8 years ago
98d6d131a3
update unittests for batoto and yonkouprod
8 years ago
041c70a3e1
bump version to 0.6.2
8 years ago
468c9aeeec
make tests more reliable
8 years ago
33c799b2ae
[yonkouprod] rename 'yonkou' to 'yonkouprod'
8 years ago
95986ed566
[seiga] add user extractor
8 years ago
efaba5f398
[luscious] support new album URLs + small fixes
8 years ago
79ea457e24
[luscious] use new 'html' attribute in API responses
8 years ago
a8d716c043
[pinterest] tests + fix error message
8 years ago
a849d8f2f7
add a few more tests
8 years ago
7952b8d18d
add a few tests expecting exceptions
8 years ago
efdc299547
[hentaifoundry] get artist name from webpage
8 years ago
1aeb60d997
[jaiminisbox] add extractor
8 years ago
8b2024a1a5
[hentaifoundry] support direct links to images
8 years ago
c604a65b88
[imgyt] use token as filename if none is given
8 years ago
3a7421a6ce
[twitter] get 'original' instead of 'large' image
8 years ago
cb180507ab
[batoto] add manga extractor
8 years ago
ff2a65d5c1
[exhentai] raise proper exception for 'unavailable' galleries
8 years ago
f69209871d
[kisscomic] change some names from 'manga' to 'comic'
8 years ago
d623b9bfdc
[readcomics] code cleanup + tests
8 years ago
c19241e299
[readcomics] add comic extractor
8 years ago
179d62023c
[readcomics] add comic-issue extractor
8 years ago
d12a04af3c
[test] filter out classes without a 'test' attribute
8 years ago
40dbea7ed2
rewrite parts of the cloudflare bypass system
8 years ago
3c4155c9c9
[kisscomic] add extractors
8 years ago
06c798d18e
[yonkou] add chapter extracter
8 years ago
effdf9c3be
[luscious] adapt to API changes
8 years ago
4c55275305
update tests
8 years ago
69cde97c93
add utility extractor that runs test-URLs
8 years ago
b0e8daf415
[imgclick] remove extractor - uses captcha
8 years ago
0211ec4114
update some tests
8 years ago
dfd1992a2c
[hentaifoundry] small updates
...
- throw an exception if an user or image does not exist
- update tests, since the user of the old ones left
8 years ago
2fae0b1803
[fapat] add extractor
8 years ago
583f1b8bbb
[postimg] add extractor
8 years ago
d1cd9acf54
[pixhost] adjust to new site layout
8 years ago
0989cd2430
add basic support for file:// URLs
...
this allows you to feed local files into the recursive extractor
8 years ago
07ffab04c3
add -i/--input-file option
8 years ago
02b6f8b165
bump version to 0.6.1
8 years ago
91dff9827f
[luscious] always provide URLs to the original images
8 years ago
d402e644bf
update tests
8 years ago
46440fda2d
[imagevenue] add extractor
8 years ago
3c1daef839
don't delete downloaded files in certain edge cases
8 years ago
2b2bdce366
don't raise an exception if a download fails ( #5 )
8 years ago
2449825d53
[kissmanga] solve cloudflare challenge on demand
8 years ago
1e0cf8ddcc
implement a method to manually invalidate cached values
8 years ago
9e3788175e
implement decorator for cloudflare bypass
...
this method for enabling and caching a cloudflare bypass for a
requests.session object allows for different cache-timeouts for
different domains
8 years ago
6e98538d36
[mangamint] small fixes and additions
...
- force https
- update patterns to also catch non-standard URLs to manga and chapters
- handle URLs to non-existing resources gracefully
- fix terminating condition for chapter listings
- add a base class for common code
- add and update tests
8 years ago
c333bc33e3
[mangapark] small fixes and additions
...
- add a 'title' keyword for chapter-titles and update the directory
format accordingly
- add a 'type' keyword to distinguish between manga and manhwa
- fix an issue where an exception would be thrown if a chapter number
did not have any special additions (2.5, 55a, v2, etc.)
- add a test-case without a special chapter number
- unescape manga title
8 years ago
b634ace39e
[readcomiconline] add comic-issue and comic extractor
8 years ago
99440ca51a
[imgtrial] add extractor
8 years ago
5f2824dfe6
[imgspot] add extractor
8 years ago
c85adbdc9d
[mangamint] code simplifications
8 years ago
7c8fae62f6
update default config file paths
8 years ago
88193718e8
[pixhost] add extractor
8 years ago
07e9e2c4f1
[imgmaid] add extractor
8 years ago
885aa7f7cf
[img4ever] add extractor
8 years ago
5c69a380f0
[kissmanga] code simplifications
8 years ago
4f51d72d22
[imgbox] handle non-existing images/galleries
8 years ago
68fe58ee72
[imgchili] set encoding
8 years ago
58ce0b55e3
[deviantart] make sure to pick the highest quality image
8 years ago
37ea66098a
add tests
8 years ago
653885abf2
[doujinmode] remove extractor - site is down
8 years ago
78b168820d
move another 4 extractors
8 years ago
5e82e8dfbd
move chronos & co. to imagehosts file
8 years ago
8b7f719f36
combine imagehost extractors into a single file
...
added extractors for
- hosturimage.com
- imageontime.org
- imgupload.yt
- imgspice.com
- imgclick.net
All of these would have shared a lot of common code, so i created a base
class for imagehost extractors and put them all in the same file to
avoid clutter.
8 years ago
e6d26f0476
don't overwrite a response's encoding with None
8 years ago
e0b3339107
[imgur] match mobile urls (m.imgur.com)
8 years ago
04b1519779
[hitomi] fix domain of image-urls
8 years ago
2004f6bd67
add a few tests
8 years ago
005c539829
improve foolslide volume- and chapter-keywords
8 years ago
2a632c9bfd
[worldthree] add chapter extractor
8 years ago
91c446805b
replace platform.system() with os.name
8 years ago
828aedd571
[khinsider] unescape soundtrack title
8 years ago
3078e4c9f5
[sensescans] add chapter extractor
8 years ago
5374f12bd0
[dokireader] use foolslide base class
8 years ago
f4c2461ba7
[powermanga] use foolslide base class
8 years ago
e1ce27f41b
add base class for foolslide based sites
8 years ago
46a3f666df
[whentai] add single image extractor
8 years ago
a0f56b2c54
update tests
8 years ago
f415d4816b
[whentai] add user extractor
8 years ago
d793e16463
[nhentai] fix subcategory value
8 years ago
02a441503a
[pinterest] adjust regex pattern
8 years ago
a33b13c631
[8chan] fix file urls
8 years ago
ffc3c2d32c
[imgbox] transition to https
8 years ago
492cb38391
[exhentai] use image-count as stop signal
8 years ago
607f50effb
[exhentai] retry failed api calls
8 years ago
691f3d1a02
bump version
8 years ago
f434a0711b
put centralized version string in 'version.py'
8 years ago
f7c993d952
set 'extension' keyword whenever possible
8 years ago
be73af551e
[rapidimg] unset test from parent class
8 years ago
8c8ad08e28
[imgur] fix image url retrieval
8 years ago
bf8d88499a
[twitter] add extractor
8 years ago
e3926bf351
[hentaihere] add manga- and chapter-extractors
8 years ago
c155c7b94b
[hentai2read] put some common code in a base class
8 years ago
0f96eb180e
add Python2 compatible version check
8 years ago
277c505b8d
[picmaniac] add extractor
8 years ago
a0c044f0c9
[rapidimg] add extractor
8 years ago
883e702fd6
[mangapark] remove 'url' keyword + fix tests
8 years ago
c20a3b3a01
[*chan] code cleanup
8 years ago
14237142d8
use 'format_map()'
8 years ago
d31ccb16f2
rename 'generic' to 'recursive'
8 years ago
12c99293b6
allow extension by Content-Type for exhentai, seiga, senmanga
8 years ago
dd8236e733
enable non-standard MIME types
8 years ago
29692c5784
get extension from Content-Type header if not provided
8 years ago
8d106a447c
[tumblr] delete more useless keywords
8 years ago
051c181138
[dokireader] add chapter extractor
8 years ago
1134339c1f
Merge branch 'category'
8 years ago
56d810c896
update keyword hashes for tests
8 years ago
19c2d4ff6f
remove explicit (sub)category keywords
8 years ago
f32cf28758
enable long pathnames on windows ( #4 )
8 years ago
0a3fb198f3
[batoto] raise exception if chapter is unavailable ( #4 )
8 years ago
2418bfe91b
replace JSONDecodeError with ValueError
8 years ago
581daebc4b
remove trailing spaces from path segments ( #4 )
8 years ago
a347d50ef5
add (sub)category keyword automatically
8 years ago
813317045e
bump version
8 years ago
e12862520d
expand path for cache database
8 years ago
3f985d5342
[hitomi] fix some keywords
8 years ago
9d107b8e1b
[dynastyscans] add chapter extractor
8 years ago
975a7cb6b9
[batoto] add (optional) login capabilities
8 years ago
e3d156078c
[exhentai] rewrite
8 years ago
49a05c32ed
add missing tests
8 years ago
a017cd9e63
[exhentai] metadata consistency
8 years ago
fea3be0aed
[nijie] add image-extractor
8 years ago
d324c58038
[rule34] add tag- and image-extractor
8 years ago
85ff3d160e
[tumblr] fix json parsing + metadata consistency
8 years ago
ba86bbfbdb
add '--list-extractors' argument
8 years ago
80d98f97fa
[hbrowse] add manga extractor
8 years ago
4cb886be78
[turboimagehost] add tests
8 years ago
26a9b7beee
[imagetwist] small fixes + tests
8 years ago
a9be82ecc2
[imgbox] small fixes + tests
8 years ago
d7e168799d
consistent extractor naming scheme + docstrings
8 years ago
888a988725
[imgchili] workaround for long filenames
8 years ago
155af213a9
[imgtrex] add extractor
8 years ago
dbdd43cff5
[imgcandy] add extractor
8 years ago
44a5052302
[coreimg] add extractor
8 years ago
dcd213f13e
[nijie] raise NotFoundError
8 years ago
8f6d055deb
[pinterest] raise NotFoundError + tests
8 years ago
517e88680f
[pinterest] add board extractor
8 years ago
79721a65ff
[pinterest] add pin extractor
8 years ago
2afa65cfc7
[imagebam] add single-image extractor
8 years ago
808cf69556
update a few tests
8 years ago
49359d375c
[exhentai] unescape title
8 years ago
95097fac00
[exhentai] transition to https
8 years ago
98877a45fb
[seiga] raise NotFoundError
8 years ago
30d10cf595
[pixiv] fix KeyError if no status is present
8 years ago
effa1084f2
[pixiv] raise NotFoundError instead of failing
8 years ago
687478f6f6
new exception + docstrings
8 years ago
813320d7db
[seiga] match direct-links to images
8 years ago
85cd7c34b1
[pixiv] match direct-links to images
8 years ago
143bd9de11
add '--version'
8 years ago
6d401b1118
precompile regular expressions
8 years ago
57a616a36f
update README and bump version
8 years ago
fa14ef17ea
[imagefap] deal with long filenames
8 years ago
406add217c
print urls recursively
8 years ago
5a5b47e77a
[imagefap] add user extractor
8 years ago
d9c5b7a102
[imagefap] add single-image extractor
8 years ago
6792c68254
[seiga] add extractor
8 years ago
dac796879a
[imagefap] add extractor
8 years ago
f17e49dcf2
write error messages to stderr
8 years ago
3e8f70188f
[hentaibox] add extractor
8 years ago
b0ea9021dc
handle broken pipes
8 years ago
5a13ac8e3d
[doujinmode] small fixes + test
8 years ago
25f56c9061
[doujinmode] add extractor
8 years ago
a2c5b1e07d
[imgchili] rewrite + single-image extractor
8 years ago
43210391ea
[imgyt] add extractor
8 years ago
b9be7f5907
[chronos] fix + test
8 years ago
b6a68775d4
[senmanga] add chapter extractor
8 years ago
2faa7393b1
[mangahere] adjust for image domain
8 years ago
c2c2e85da3
[imgur] unescape album title + other fixes
8 years ago
db1738d451
[luscious] add extractor
8 years ago
6366ddf154
bump version number
8 years ago
3bcc697853
change default config paths
8 years ago
d01b21c6c8
[nijie] change 'email'-parameter to 'username'
8 years ago
fe85195023
add --username and --password cmd-line options
8 years ago
f0f7306db6
re-raise async exceptions in main thread
8 years ago
0e20be5a0a
[exhentai] provide username/password auth
8 years ago
de9aa8f310
better key=value option parsing
8 years ago
0ba9601b63
[nijie] raise AuthError
8 years ago
3136278f79
[pixiv] add option to skip ugoiras
8 years ago
59445c821b
[gelbooru] remove login/auth requirement
8 years ago
221b3869cf
[mangastream] adapt to new page structure
8 years ago
d6c06f9efd
add authentication-exception
8 years ago
6f7f29d684
rename a few files
8 years ago
6f7d42b974
update tests
8 years ago
93c8f6e777
[nhentai] fix json detection
8 years ago
000df8d1fa
add 'encoding' argument for Extractor.request
8 years ago
3c94d85409
fix cloudflare bypass
8 years ago
b5e62ec308
[mangastream] adjust regex agein ( fixes #2 )
9 years ago
b0c333b799
rewrite cache module
9 years ago
2b15b81673
[khinsider] add extractor
9 years ago
0736fe29e2
[mangahere] fix parsing
9 years ago
7af8ae509c
[mangastream] adjust regex
9 years ago
1416e7f6f7
[hentai2read] fix parsing for new page layout
9 years ago
c9b7db5af5
[pixiv] code cleanup
9 years ago
8accf4b790
use cache-decorator for cloudflare bypass
9 years ago
10fcc010bb
[nijie] cache login sessions
9 years ago
c8e83f6560
[pixiv] cache login sessions
9 years ago
738c65d54f
add 'keyarg' argument to cache-decorator
9 years ago
81096f7790
[tumblr] fix json parsing
9 years ago
b8c6937d39
first draft for a cache-module
9 years ago
f974ea73db
[tumblr] add tag-extractor
9 years ago
58a0029bb2
[tumblr] add post-extractor
9 years ago
8eb7232169
[tumblr] add extractor
9 years ago
595e5872d3
[hentai2read] add multi-chapter extractor
9 years ago
f3dc8851c8
[hentai2read] add a couple more keywords
9 years ago
20228a003f
[hentai2read] add extractor
9 years ago
23a572a20d
[pixiv] update testdata
9 years ago
7f9aeb774e
[powermanga] fix image-urls
9 years ago
5a98be1c45
[nhentai] remove 'favorites' keyword
9 years ago
8a49a28d13
replace deprecated 'unescape' method
9 years ago
9b46322e70
fix indentation
9 years ago
616e0aedd6
update booru testdata
9 years ago
565ea042c9
update HashJob to generate hashes for downloaded content
9 years ago
ecc6542fc8
change required parameter type to file-like objects
9 years ago
5a8541afa5
remove 'unstable' tests
9 years ago
ba99506c72
more extractor test-cases
9 years ago
a99fdb0d1e
[mangapark] fix regexes
9 years ago
2449532132
add more extractor test-cases
9 years ago
f0e9e8e0dc
ensure extractors() returns each extractor only once
9 years ago
9c9b1e112a
[imgur] fix 'date' and 'count' keywords
9 years ago
5589bc2ed7
[deviantart] fix non-terminating loop
9 years ago
c8fdfbec42
[danbooru] add test cases
9 years ago
5304e5beef
testing environment for extractor results
9 years ago
9ca4426b72
add HashJob for automated testing
9 years ago
059b1ee5e8
add base class for job types
9 years ago
75ff08f73c
[nijie] proper login and other issues fixed
9 years ago
2dfed4d40a
implement -g,--get-urls option
9 years ago
50ec170b00
[mangapark] add manga extractor
9 years ago
0ff437ca88
[mangapark] add chapter extractor
9 years ago
995d8578bf
[deviantart] fix dates in image-extractor
9 years ago
7d965a2897
[deviantart] add single image extractor
9 years ago
e4a661fd6b
[deviantart] fix issue with small images
9 years ago
3ebd126b35
[imgbox] add single image extractor
9 years ago
a71805bacf
[chronos] add extractor
9 years ago
30e47848cc
update setup.py and bump version
9 years ago
cb53b29391
[imagetwist] adjust pattern and other fixes
9 years ago
65c8566326
add provisional generic extractor
9 years ago
e7512f626d
[turboimagehost] add extractor
9 years ago
56876b76f1
[imagetwist] add extractor
9 years ago
810aa4e146
add a space after status indicators
9 years ago
b2401542eb
implement output.mode and .shorten options
9 years ago
8d257510d0
rework output module
9 years ago
bd87b5186d
[spectrumnexus] recognize volumes
9 years ago
2ced575c4a
[spectrumnexus] add manga-extractor
9 years ago
a8c0b4531d
fix issue with Ctrl+c on windows
9 years ago
140f0475a6
allow lists to specify base-dir segments
9 years ago
d26eec8470
workarounds for windows console
9 years ago
4b377ccc09
use output-module during downloads
9 years ago
c786843a6f
add output-module
9 years ago
f7c47a6018
add subcategories to extractors
9 years ago
81dcfbec90
initial support for extractor-subcategories
9 years ago
1146ed95dc
improved get_base_directory
9 years ago
1497da07de
remove unused format-strings
9 years ago
c23eaccf46
[hentaifoundry] add single image extractor
9 years ago
34210edf9a
[mangashare] add manga-extractor
9 years ago
f48712c9c9
docstrings
9 years ago
9cb184d409
[powermanga] support another url-pattern
9 years ago
1eaf0a55a6
[mangamint] add manga-extractor
9 years ago
914062d172
use text.extract_iter where applicable
9 years ago
Mike Fährmann
