3c9f190757
extend output of --list-keywords
7 years ago
cfa479fab5
update error message for unspecified exceptions
...
- ask user to report unexpected errors, which usually indicate
extractor failure
- handle OSErrors separately (permissions, disk full, etc)
- revert 30eef52
7 years ago
915a0137de
improve 'extractor.request'
...
- add 'fatal' argument
- improve internal logic and flow
- raise known exception on error
- update exception hierarchy
7 years ago
58e95a7487
share extractor and downloader sessions
...
There was never any "good" reason for the strict separation
between extractors and downloaders. This change allows for
reduced resource usage (probably unnoticeable) and less lines
of code at the "cost" of tighter coupling.
7 years ago
c921b4f32a
code cleanup and fixing tests
7 years ago
25bcdc8aa9
add `--write-unsupported` option ( #15 )
7 years ago
99b72130ee
[reddit] enable recursion ( #15 )
...
reddit extractors now recursively visit other submissions/posts
linked to in the initial set of submissions.
This behaviour can be configured via the 'extractor.reddit.recursion'
key in the configuration file or by `-o recursion=<value>`.
Example:
{"extractor": {
"reddit": {
"recursion": <value>
}}}
Possible values:
* -1 - infinite recursion (don't do this)
* 0 - recursion is disabled (default)
* 1 and higher - maximum recursion level
7 years ago
ae686c4c08
run queue items immediately
7 years ago
30eef527d8
update output logic on error
...
[ci skip]
7 years ago
e425243b1e
[reddit] some small fixes
...
- filter or complete some URLs
- remove the 'nofollow:' scheme before printing URLs
- (#15 )
7 years ago
a90c6acc9c
code cleanup + fixes
7 years ago
4c88c0d496
rework the output format for --list-keywords
7 years ago
13dc5d72bc
update some extractors to use https
8 years ago
5af35ea150
add -v/--verbose option and reduce error verbosity
...
(#12 )
8 years ago
b43cd88101
add '-j/--dump-json' option
...
this outputs the extractor-results in JSON format rather then
downloading files
8 years ago
841fd50242
move code into util.py
8 years ago
ed94d9b92d
fix/improve various things
8 years ago
27ae152f57
use logging to report errors
8 years ago
7a9d66fbce
implement basic way to tell extractors to skip ahead
8 years ago
2fa575b273
restore exception-testing to its old form
8 years ago
40be4933b8
fix exception based tests
8 years ago
24f41e13b3
move some exception handling code
8 years ago
6208d9dd79
implement '--images' and '--chapters' options
...
- the former '--items' has been renamed to '--chapters'
- #6
8 years ago
2a32b12043
add '--items' option
...
this allows to specify which manga-chapters/comic-issues to download
when using gallery-dl on a manga/comic URL
8 years ago
3bca866185
rework the '-g' cmdline option
...
the amount of how often the -g option is given now determines up until
what level URLs are resolved.
example:
$ gallery-dl -g http://kissmanga.com/Manga/Dropout
http://kissmanga.com/Manga/Dropout/Ch-000---Oneshot-?id=145847
- when applied to a manga-extractor, specifying the -g option once will
now print a list of all chapter URls
$ gallery-dl -gg http://kissmanga.com/Manga/Dropout
http://2.bp.blogspot.com/.../000.png
http://2.bp.blogspot.com/.../001.png
...
- specifying it twice (or even more often) will go a level deeper and
print the image URLs found in those chapters
8 years ago
4f123b8513
code adjustments according to pep8
8 years ago
29692c5784
get extension from Content-Type header if not provided
8 years ago
1134339c1f
Merge branch 'category'
8 years ago
f32cf28758
enable long pathnames on windows ( #4 )
8 years ago
581daebc4b
remove trailing spaces from path segments ( #4 )
8 years ago
a347d50ef5
add (sub)category keyword automatically
8 years ago
406add217c
print urls recursively
8 years ago
6f7f29d684
rename a few files
8 years ago
Mike Fährmann
