Mike Fährmann
347baf7ac5
improve util.parse_range() performance
...
It is never going to actually matter, but using partition() instead
of split() is twice as fast.
7 years ago
Mike Fährmann
7b5ba69951
[hentaihere] ensure consistent extraction results
...
sometimes there is a random space before the next <a>
7 years ago
Mike Fährmann
377b78b3c9
[hentai2read] fix manga name extraction
7 years ago
Mike Fährmann
54c36a8a34
[subapics] add chapter- and manga-extractor ( #70 )
7 years ago
Mike Fährmann
2dd3aeeeae
[komikcast] add chapter- and manga-extractor ( #70 )
7 years ago
Mike Fährmann
7a412f5c32
implement generic manga-chapter extractor
7 years ago
Mike Fährmann
aa38eab2be
allow not-defined fields in format strings
...
... and replace them with "None", for now
7 years ago
Mike Fährmann
6a07e38366
implement extractor.add() and .add_module()
...
... as a public and non-hacky way to add (external) extractors to
gallery-dl's pool and make them available for extractor.find()
7 years ago
Mike Fährmann
a34cebc253
[luscious] jump to first image if cover does not link to it
7 years ago
Mike Fährmann
915807dd77
log HTTP errors as warnings
7 years ago
Mike Fährmann
db7f04dd97
emit log messages on download failure
...
and when retrying with fallback URLs
7 years ago
Mike Fährmann
d951f13e37
add config option for unsupported-URL file
...
for consistency's sake
7 years ago
Mike Fährmann
619387cbb1
update extractor unittest results
7 years ago
Mike Fährmann
364e335440
smaller adjustments and improvements
...
- requests and urllib3 version on 1 line
- close input file after reading from it
- use expand_path for unsupported-urls file
- remove unnecessary logging from options.py
7 years ago
Mike Fährmann
c9a9664a65
change --write-log behaviour
...
- log files now get truncated when opening them
(mode "w" instead of "a")
- log verbosity to file depends on -q/-v
(same as logging to stderr)
7 years ago
Mike Fährmann
97f4f15ec0
add option to write logging output to a file
...
- '--write-log FILE' as cmdline argument
- 'output.logfile' as config file option
7 years ago
Mike Fährmann
f94e3706a8
use logging module for error messages during downloads
7 years ago
Mike Fährmann
db91cf871c
document message identifiers
7 years ago
Mike Fährmann
0dd48d644f
update test results
...
nothing broke, but things got updated or changed
7 years ago
Mike Fährmann
1e93955170
[batoto] remove module
...
Site officially shut down on 2018.01.18
7 years ago
Mike Fährmann
27fce6f600
fix UrlJob behavior
7 years ago
Mike Fährmann
76509a6d3c
[imgur] update test results
7 years ago
Mike Fährmann
9fccd7b783
[tumblr] provide fallback URLs ( #64 )
...
Each image now produces 3 URLs:
- amazonaws.com _raw (or _1280 for older images)
- amazonaws.com _500
- media.tumblr.com (URL returned by API)
7 years ago
Mike Fährmann
b837420291
fix minor urllist issues
7 years ago
Mike Fährmann
9d69401391
initial support for multiple URLs per image
7 years ago
Mike Fährmann
6174a5c4ef
[download] adjust filename extension on filetype mismatch
...
(closes #63 )
7 years ago
Mike Fährmann
91ed147cef
[oauth] use custom key/secret values during oauth:…
7 years ago
Mike Fährmann
421a9740a3
[tumblr] add 'tumblr:' to force Tumblr extractor ( #71 )
7 years ago
Mike Fährmann
40d35c87bc
[paheal] add tag- and post-extractors ( closes #69 )
7 years ago
Mike Fährmann
cc0c2cca57
[reddit] add extractor for reddit-hosted images ( closes #68 )
7 years ago
Mike Fährmann
f10ffc0839
update extractor blacklist to also allow classes
7 years ago
Mike Fährmann
b6797032e3
release version 1.1.2
7 years ago
Mike Fährmann
35e09869d1
[mangapark] fix image URLs and use HTTPS
7 years ago
Mike Fährmann
9a049bdf51
[tumblr] add 'likes' extractor ( #65 )
7 years ago
Mike Fährmann
67d4462d26
[batoto] rudimentary Cloudflare bypass
7 years ago
Mike Fährmann
29d75fc3fa
[tumblr] add support for OAuth authentication ( #65 )
7 years ago
Mike Fährmann
4edb25346e
[slideshare] support mobile URLs ( closes #67 )
7 years ago
Mike Fährmann
e420a28bbc
fix cookie tests
7 years ago
Mike Fährmann
b33efc99a4
[idolcomplex] add support for idol.sankakucomplex.com
7 years ago
Mike Fährmann
75b2e84b6d
[tumblr] use s3.amazonaws.com for image URLs ( #64 )
7 years ago
Mike Fährmann
9a8e98f699
add gitter badge to README
7 years ago
Mike Fährmann
5b094328b5
[puremashiro] add chapter- and manga-extractor ( closes #66 )
...
Also adds support for region subtags in language codes (e.g. en-us)
7 years ago
Mike Fährmann
974e73bdbb
[booru] smaller code adjustments
7 years ago
Mike Fährmann
03b8a548cb
[tumblr] change `reblogs` default value to `true` ( #61 )
7 years ago
Mike Fährmann
d235f68f59
[tumblr] add option to filter reblogged posts ( #61 )
...
Reblogs are ignored by default, but can be included by setting
'extractor.tumblr.reblogs' to 'true'.
7 years ago
Mike Fährmann
a794fffc6d
[batoto] extend chapter-string regex ( closes #60 )
...
Non-numeric chapter indices exist after all ...
7 years ago
Mike Fährmann
1219ebb7f5
[danbooru] use alternate subdomains; support safebooru
7 years ago
Mike Fährmann
9e8a84ab6c
[booru] rewrite using Mixin classes ( #59 )
...
- improved code structure
- improved URL patterns
- better pagination to work around page limits on
- Danbooru
- e621
- 3dbooru
7 years ago
Mike Fährmann
0876541e43
[seiga] update tests
7 years ago
Mike Fährmann
1a70857a12
update extractor-unittest capabilities
...
- "count" can now be a string defining a comparison in the form of
'<operator> <value>', for example: '> 12' or '!= 1'. If its value
is not a string, it is assumed to be a concrete integer as before.
- "keyword" can now be a dictionary defining tests for individual keys.
These tests can either be a type, a concrete value or a regex
starting with "re:". Dictionaries can be stacked inside each other.
Optional keys can be indicated with a "?" before its name.
For example:
"keyword:" {
"image_id": int,
"gallery_id", 123,
"name": "re:pattern",
"user": {
"id": 321,
},
"?optional": None,
}
7 years ago