e23e4b1100
[seaotterscans] add extractor
8 years ago
616464f8f8
[kireicake] add extractor
8 years ago
8d5f26d530
small fixes to the output module
8 years ago
f537ad5f2f
[kissmanga] re-enable module
8 years ago
c9738ea80b
add an aes implementation
8 years ago
b603b592cf
[exhentai] accept "e-hentai.org" URLs ( #11 )
8 years ago
11ce871718
[deviantart] add favourites-extractor
8 years ago
d430732502
[deviantart] update extractors
...
- use base class for all extractors
- use API for single images
- add support for more API endpoints
- add additional keywords
8 years ago
298d7c45f7
[nijie] support multi-page image listings
8 years ago
91ad8ac2b3
[mangastream] adapt to domain-change to readms.net
8 years ago
e9b445a0f8
update release script
8 years ago
43d8b66e70
release version 0.8.0
8 years ago
841fd50242
move code into util.py
8 years ago
e3212dd98f
fix some smaller stuff
...
- remove support for old windows config paths
- catch exception if cache-database can't be opened
- fix username/password settings for unit tests
- rename variable 'max_tries' to 'retries'
8 years ago
e4b3077168
improve config module
...
- speed improvements, especially in the 'interpolate' function
- 'interpolate' now prioritizes base-level values if they exist
- "username" is chosen before "extractor.<category>.username"
- -u/--username & co can now override config-file values
8 years ago
6cbb078392
[batoto] fix cache time for login sessions
...
from 1 year to 1 week ...
8 years ago
e2b5cd9918
change config-path for 'retries' and 'timeout'
8 years ago
6fd9e66223
[whentai] remove module
...
apparently they changed their system and now you can't download the
original images without an account with VIP status.
8 years ago
cdecd0b37b
add notification when solving cloudflare challenge
8 years ago
dce5bce74e
[kissmanga] temporarily disable module
...
... until I've figured out how to decode their image urls
8 years ago
f2ef49563b
fix argument order for python33 and 34
8 years ago
11d5c6f717
move option parsing to seperate module
8 years ago
0b5076815d
always delete incompletely downloaded files
8 years ago
4e7661ab01
[imgtrex] re-add extractor
8 years ago
0257d3e7ac
[mangamint] remove extractors - site is down
8 years ago
1d46be545c
add login notifications
8 years ago
ed94d9b92d
fix/improve various things
8 years ago
abfe7456d6
add '-R/--retries' and '--http-timeout' options
...
(#10 )
8 years ago
22910f9562
improve error handling of http file downloads
...
(#10 )
8 years ago
80df2b3527
add custom argparse action
8 years ago
619c74159a
[seiga] fix file extension and xml parsing
...
- The file extension of the first image had been used for all further
images
- API responses can contain invalid characters, which cause the XML
parser to fail (http://seiga.nicovideo.jp/user/illust/26377934
contains several \x08 characters)
8 years ago
8816dab549
add 'dev' suffix for non-release versions
...
(#9 )
8 years ago
66eb3d3488
fix tests
8 years ago
f622411be5
[deviantart] implement 'skip' method
8 years ago
27ae152f57
use logging to report errors
8 years ago
dfe2c2dced
[batoto] update test results
8 years ago
43e3bb24ae
[imagefap] don't rely on image-count
...
(fixes #9 )
8 years ago
0cfe51dc78
add '--config-yaml' option
...
(#8 )
8 years ago
379125746a
[deviantart] improve API error handling
...
(#7 )
8 years ago
f782282f97
add logger objects to extractors
8 years ago
0b214eeb9d
bump version to 0.7.0
8 years ago
2c2de778f2
[deviantart] adjust error message
8 years ago
c24aeb5cbc
[whentai] update test results
8 years ago
f36d685ce1
[deviantart] always request access token before api calls
...
(#7 )
8 years ago
ff92674379
[booru] simple skip functionality
...
(#6 )
8 years ago
7a9d66fbce
implement basic way to tell extractors to skip ahead
8 years ago
872b8aed97
[4chan] update test
8 years ago
2fa575b273
restore exception-testing to its old form
8 years ago
f361cb13e0
[hentai2read] fix extraction
8 years ago
40be4933b8
fix exception based tests
8 years ago
24f41e13b3
move some exception handling code
8 years ago
6208d9dd79
implement '--images' and '--chapters' options
...
- the former '--items' has been renamed to '--chapters'
- #6
8 years ago
38aa496ecb
[deviantart] restore tests + set api version
8 years ago
39b1a18f12
[deviantart] fix issue with missing 'content'-key
8 years ago
e87e6fbc67
change some config keys
...
directory_fmt -> directory
filename_fmt -> filename
download-original -> original
8 years ago
2a32b12043
add '--items' option
...
this allows to specify which manga-chapters/comic-issues to download
when using gallery-dl on a manga/comic URL
8 years ago
96baea0959
[hentaibox] remove extractor
8 years ago
3bca866185
rework the '-g' cmdline option
...
the amount of how often the -g option is given now determines up until
what level URLs are resolved.
example:
$ gallery-dl -g http://kissmanga.com/Manga/Dropout
http://kissmanga.com/Manga/Dropout/Ch-000---Oneshot-?id=145847
- when applied to a manga-extractor, specifying the -g option once will
now print a list of all chapter URls
$ gallery-dl -gg http://kissmanga.com/Manga/Dropout
http://2.bp.blogspot.com/.../000.png
http://2.bp.blogspot.com/.../001.png
...
- specifying it twice (or even more often) will go a level deeper and
print the image URLs found in those chapters
8 years ago
9d36acbbc4
[yonkouprod] update test results
8 years ago
9a08f8a097
improved foolslide-based extractors
...
- this includes dokireader, fallenangels, jaiminisbox, powermanga,
sensescans, worldthree, yonkouprod, gomanga, yomanga
- added 'chapter_string', 'chapter_id', 'chapter_minor' and 'count'
keywords
- changed the 'chapter' keyword to always be just a number
- changed the default directory format
8 years ago
0a6487afe8
[exhentai] fix detection of invalid gallery keys
8 years ago
4cb49cfd76
bump version to 0.6.4
8 years ago
0b59d9f8c7
disable urllib3s InsecureConnectionWarning
8 years ago
bad623751f
[hentaihere] transition to https
8 years ago
331b413c60
update extractor test script
8 years ago
c84e975dcb
[twitter] fix image extraction
8 years ago
0af02007a9
[imagetwist] fix site access
8 years ago
2f6e3874a7
[sensescans support http://sensescans.com/reader/ ... URLs
8 years ago
52104b2bb6
[fallenangels] add chapter extractor
8 years ago
7880cc1ad7
[imgtrex] remove extractor - domain no longer exists
8 years ago
21e0dfbe20
[chronos] raise NotFoundError instead of crashing
8 years ago
2b38398940
[imgyt] raise NotFoundError instead of crashing
8 years ago
94e10f249a
code adjustments according to pep8 nr2
8 years ago
f1b7d41608
[mangamint] fix keyword extraction
8 years ago
4f123b8513
code adjustments according to pep8
8 years ago
8e93633319
[nhentai] fix image urls
8 years ago
d82508f245
fix tests for turboimagehost and pinterest
8 years ago
3434c5a026
[pinterest] add support for pin.it URLs
8 years ago
138e39686b
bump version to 0.6.3
8 years ago
37d4d07d9b
compatibility fixes to make a standalone exe work
8 years ago
4553a6392f
[whentai] add unittests
8 years ago
501b64fa9d
add a __main__.py file
8 years ago
434ef4d5a2
[mangafox] raise proper exception if chapter is not available
8 years ago
98fa5cf2fa
[mangafox] add chapter extractor
8 years ago
cc0b4f2661
[yomanga] add chapter extractor
8 years ago
ad4b02508f
trying to understand travis-ci unit test failures
...
- added some debug output via logging module
- unit tests work on my machine (tm)
8 years ago
72d3ca0bf9
[deviantart] use API for user-extractor
8 years ago
989820d719
workaround for python 3.6s behaviour with BEGIN EXCLUSIVE in sqlite
8 years ago
8780abcc77
fix a small spelling error
8 years ago
00074a71d7
several changes to make travis build work
...
- fixed html.unescape not being available on Python3.3
- removed inconsistent test result
- added username/password pairs for authenticating extractors
8 years ago
cfe5bf732a
[gomanga] add chapter extractor
8 years ago
4a8d74973c
adjust login methods to a specific style
8 years ago
98d6d131a3
update unittests for batoto and yonkouprod
8 years ago
041c70a3e1
bump version to 0.6.2
8 years ago
468c9aeeec
make tests more reliable
8 years ago
33c799b2ae
[yonkouprod] rename 'yonkou' to 'yonkouprod'
8 years ago
95986ed566
[seiga] add user extractor
8 years ago
efaba5f398
[luscious] support new album URLs + small fixes
8 years ago
79ea457e24
[luscious] use new 'html' attribute in API responses
8 years ago
a8d716c043
[pinterest] tests + fix error message
8 years ago
a849d8f2f7
add a few more tests
8 years ago
7952b8d18d
add a few tests expecting exceptions
8 years ago
efdc299547
[hentaifoundry] get artist name from webpage
8 years ago
1aeb60d997
[jaiminisbox] add extractor
8 years ago
8b2024a1a5
[hentaifoundry] support direct links to images
8 years ago
c604a65b88
[imgyt] use token as filename if none is given
8 years ago
3a7421a6ce
[twitter] get 'original' instead of 'large' image
8 years ago
cb180507ab
[batoto] add manga extractor
8 years ago
ff2a65d5c1
[exhentai] raise proper exception for 'unavailable' galleries
8 years ago
f69209871d
[kisscomic] change some names from 'manga' to 'comic'
8 years ago
d623b9bfdc
[readcomics] code cleanup + tests
8 years ago
c19241e299
[readcomics] add comic extractor
8 years ago
179d62023c
[readcomics] add comic-issue extractor
8 years ago
d12a04af3c
[test] filter out classes without a 'test' attribute
8 years ago
40dbea7ed2
rewrite parts of the cloudflare bypass system
8 years ago
3c4155c9c9
[kisscomic] add extractors
8 years ago
06c798d18e
[yonkou] add chapter extracter
8 years ago
effdf9c3be
[luscious] adapt to API changes
8 years ago
4c55275305
update tests
8 years ago
69cde97c93
add utility extractor that runs test-URLs
8 years ago
b0e8daf415
[imgclick] remove extractor - uses captcha
8 years ago
0211ec4114
update some tests
8 years ago
dfd1992a2c
[hentaifoundry] small updates
...
- throw an exception if an user or image does not exist
- update tests, since the user of the old ones left
8 years ago
2fae0b1803
[fapat] add extractor
8 years ago
583f1b8bbb
[postimg] add extractor
8 years ago
d1cd9acf54
[pixhost] adjust to new site layout
8 years ago
0989cd2430
add basic support for file:// URLs
...
this allows you to feed local files into the recursive extractor
8 years ago
07ffab04c3
add -i/--input-file option
8 years ago
02b6f8b165
bump version to 0.6.1
8 years ago
91dff9827f
[luscious] always provide URLs to the original images
8 years ago
d402e644bf
update tests
8 years ago
46440fda2d
[imagevenue] add extractor
8 years ago
3c1daef839
don't delete downloaded files in certain edge cases
8 years ago
2b2bdce366
don't raise an exception if a download fails ( #5 )
8 years ago
2449825d53
[kissmanga] solve cloudflare challenge on demand
8 years ago
1e0cf8ddcc
implement a method to manually invalidate cached values
8 years ago
9e3788175e
implement decorator for cloudflare bypass
...
this method for enabling and caching a cloudflare bypass for a
requests.session object allows for different cache-timeouts for
different domains
8 years ago
6e98538d36
[mangamint] small fixes and additions
...
- force https
- update patterns to also catch non-standard URLs to manga and chapters
- handle URLs to non-existing resources gracefully
- fix terminating condition for chapter listings
- add a base class for common code
- add and update tests
8 years ago
c333bc33e3
[mangapark] small fixes and additions
...
- add a 'title' keyword for chapter-titles and update the directory
format accordingly
- add a 'type' keyword to distinguish between manga and manhwa
- fix an issue where an exception would be thrown if a chapter number
did not have any special additions (2.5, 55a, v2, etc.)
- add a test-case without a special chapter number
- unescape manga title
8 years ago
b634ace39e
[readcomiconline] add comic-issue and comic extractor
8 years ago
99440ca51a
[imgtrial] add extractor
8 years ago
5f2824dfe6
[imgspot] add extractor
8 years ago
c85adbdc9d
[mangamint] code simplifications
8 years ago
7c8fae62f6
update default config file paths
8 years ago
88193718e8
[pixhost] add extractor
8 years ago
07e9e2c4f1
[imgmaid] add extractor
8 years ago
885aa7f7cf
[img4ever] add extractor
8 years ago
5c69a380f0
[kissmanga] code simplifications
8 years ago
4f51d72d22
[imgbox] handle non-existing images/galleries
8 years ago
68fe58ee72
[imgchili] set encoding
8 years ago
58ce0b55e3
[deviantart] make sure to pick the highest quality image
8 years ago
37ea66098a
add tests
8 years ago
653885abf2
[doujinmode] remove extractor - site is down
8 years ago
78b168820d
move another 4 extractors
8 years ago
5e82e8dfbd
move chronos & co. to imagehosts file
8 years ago
8b7f719f36
combine imagehost extractors into a single file
...
added extractors for
- hosturimage.com
- imageontime.org
- imgupload.yt
- imgspice.com
- imgclick.net
All of these would have shared a lot of common code, so i created a base
class for imagehost extractors and put them all in the same file to
avoid clutter.
8 years ago
e6d26f0476
don't overwrite a response's encoding with None
8 years ago
e0b3339107
[imgur] match mobile urls (m.imgur.com)
8 years ago
04b1519779
[hitomi] fix domain of image-urls
8 years ago
2004f6bd67
add a few tests
8 years ago
005c539829
improve foolslide volume- and chapter-keywords
8 years ago
2a632c9bfd
[worldthree] add chapter extractor
8 years ago
91c446805b
replace platform.system() with os.name
8 years ago
828aedd571
[khinsider] unescape soundtrack title
8 years ago
3078e4c9f5
[sensescans] add chapter extractor
8 years ago
5374f12bd0
[dokireader] use foolslide base class
8 years ago
f4c2461ba7
[powermanga] use foolslide base class
8 years ago
e1ce27f41b
add base class for foolslide based sites
8 years ago
46a3f666df
[whentai] add single image extractor
8 years ago
a0f56b2c54
update tests
8 years ago
f415d4816b
[whentai] add user extractor
8 years ago
d793e16463
[nhentai] fix subcategory value
8 years ago
02a441503a
[pinterest] adjust regex pattern
8 years ago
a33b13c631
[8chan] fix file urls
8 years ago
ffc3c2d32c
[imgbox] transition to https
8 years ago
492cb38391
[exhentai] use image-count as stop signal
8 years ago
607f50effb
[exhentai] retry failed api calls
8 years ago
691f3d1a02
bump version
8 years ago
f434a0711b
put centralized version string in 'version.py'
8 years ago
f7c993d952
set 'extension' keyword whenever possible
8 years ago
be73af551e
[rapidimg] unset test from parent class
8 years ago
8c8ad08e28
[imgur] fix image url retrieval
8 years ago
bf8d88499a
[twitter] add extractor
8 years ago
e3926bf351
[hentaihere] add manga- and chapter-extractors
8 years ago
c155c7b94b
[hentai2read] put some common code in a base class
8 years ago
0f96eb180e
add Python2 compatible version check
8 years ago
277c505b8d
[picmaniac] add extractor
8 years ago
a0c044f0c9
[rapidimg] add extractor
8 years ago
883e702fd6
[mangapark] remove 'url' keyword + fix tests
8 years ago
c20a3b3a01
[*chan] code cleanup
8 years ago
14237142d8
use 'format_map()'
8 years ago
d31ccb16f2
rename 'generic' to 'recursive'
8 years ago
12c99293b6
allow extension by Content-Type for exhentai, seiga, senmanga
8 years ago
dd8236e733
enable non-standard MIME types
8 years ago
29692c5784
get extension from Content-Type header if not provided
8 years ago
8d106a447c
[tumblr] delete more useless keywords
8 years ago
051c181138
[dokireader] add chapter extractor
8 years ago
1134339c1f
Merge branch 'category'
8 years ago
56d810c896
update keyword hashes for tests
8 years ago
19c2d4ff6f
remove explicit (sub)category keywords
8 years ago
f32cf28758
enable long pathnames on windows ( #4 )
8 years ago
0a3fb198f3
[batoto] raise exception if chapter is unavailable ( #4 )
8 years ago
2418bfe91b
replace JSONDecodeError with ValueError
8 years ago
581daebc4b
remove trailing spaces from path segments ( #4 )
8 years ago
a347d50ef5
add (sub)category keyword automatically
8 years ago
813317045e
bump version
8 years ago
e12862520d
expand path for cache database
8 years ago
3f985d5342
[hitomi] fix some keywords
8 years ago
9d107b8e1b
[dynastyscans] add chapter extractor
8 years ago
975a7cb6b9
[batoto] add (optional) login capabilities
8 years ago
e3d156078c
[exhentai] rewrite
8 years ago
49a05c32ed
add missing tests
8 years ago
a017cd9e63
[exhentai] metadata consistency
8 years ago
fea3be0aed
[nijie] add image-extractor
8 years ago
d324c58038
[rule34] add tag- and image-extractor
8 years ago
85ff3d160e
[tumblr] fix json parsing + metadata consistency
8 years ago
ba86bbfbdb
add '--list-extractors' argument
8 years ago
80d98f97fa
[hbrowse] add manga extractor
8 years ago
4cb886be78
[turboimagehost] add tests
8 years ago
26a9b7beee
[imagetwist] small fixes + tests
8 years ago
a9be82ecc2
[imgbox] small fixes + tests
8 years ago
d7e168799d
consistent extractor naming scheme + docstrings
8 years ago
888a988725
[imgchili] workaround for long filenames
8 years ago
155af213a9
[imgtrex] add extractor
8 years ago
dbdd43cff5
[imgcandy] add extractor
8 years ago
44a5052302
[coreimg] add extractor
8 years ago
dcd213f13e
[nijie] raise NotFoundError
8 years ago
8f6d055deb
[pinterest] raise NotFoundError + tests
8 years ago
517e88680f
[pinterest] add board extractor
8 years ago
79721a65ff
[pinterest] add pin extractor
8 years ago
2afa65cfc7
[imagebam] add single-image extractor
8 years ago
808cf69556
update a few tests
8 years ago
49359d375c
[exhentai] unescape title
8 years ago
95097fac00
[exhentai] transition to https
8 years ago
98877a45fb
[seiga] raise NotFoundError
8 years ago
30d10cf595
[pixiv] fix KeyError if no status is present
8 years ago
effa1084f2
[pixiv] raise NotFoundError instead of failing
8 years ago
687478f6f6
new exception + docstrings
8 years ago
813320d7db
[seiga] match direct-links to images
8 years ago
85cd7c34b1
[pixiv] match direct-links to images
8 years ago
143bd9de11
add '--version'
8 years ago
6d401b1118
precompile regular expressions
8 years ago
57a616a36f
update README and bump version
8 years ago
fa14ef17ea
[imagefap] deal with long filenames
8 years ago
406add217c
print urls recursively
8 years ago
5a5b47e77a
[imagefap] add user extractor
8 years ago
d9c5b7a102
[imagefap] add single-image extractor
8 years ago
6792c68254
[seiga] add extractor
8 years ago
dac796879a
[imagefap] add extractor
8 years ago
f17e49dcf2
write error messages to stderr
8 years ago
Mike Fährmann
