eb7a1f3b98
[pixiv] rework ugoira handling
...
Frame information now gets attached to the ZIP file's keyword dict
instead of being written to a separate text file.
6 years ago
017188d268
improve extractor.request()
...
Replace the 'fatal' parameter with 'expect', which is a list/range
of HTTP status codes >= 400 that should also be accepted.
6 years ago
b84e71da91
add postprocessor documentation to configuration.rst
6 years ago
613b692275
[postprocessor:ugoira] add a few options
...
- ffmpeg-location: path to the ffmpeg (or avconv) executable
- ffmpeg-args: additional command line args for ffmpeg
- extension: filename extension of the resulting video file
6 years ago
a444755979
[postprocessor] add 'ugoira' to convert pixiv animations to webm
6 years ago
f10bd5cdbe
[4chan] unescape filenames
6 years ago
eec081dd3e
[postprocessor:zip] delete directory ( #85 )
6 years ago
2d1a104739
[mangadex] unescape manga names and chapter titles
...
pretty sure I previously tested if unescaping strings from the
embedded JSON object was necessary ... maybe they changed it
6 years ago
3bcce77f6d
release version 1.4.0
6 years ago
6ac403c5d3
add postprocessor config example
6 years ago
2403c405e3
Merge branch 'postprocessor'
6 years ago
baccf8a958
improve postprocessor handling
...
- add pathfmt argument for __init__()
- add finalization step
- add option to keep or delete zipped files
6 years ago
2628911ba0
[pp:exec] add 'async' option
6 years ago
7646bdbcfd
improve postprocessor initialization code
6 years ago
b344f2290f
fix downloader tests
6 years ago
37d97ff02c
[pp:classify] use temppath
6 years ago
97189e50cd
[pp:zip] use temppath; add options
6 years ago
821535b458
adjust PathFormat class
6 years ago
a47c6136cd
[simplyhentai] avoid redirects for all-pages.json ( #89 )
6 years ago
ad14de19c6
[imgur] support "unmuted" URLs
6 years ago
72e66f0aac
[simplyhentai] improve URL pattern
...
[ci skip]
6 years ago
cdcc3427a0
[simplyhentai] add video extractor ( #89 )
...
All videos hosted on their own servers seem be to dead,
but myhentai.tv embeds, which are most of the videos, work fine.
6 years ago
f9a6a19658
[simplyhentai] add image extractor ( #89 )
6 years ago
ebf596b399
[pawoo] restore metadata fields + smaller improvements
6 years ago
f7e7306e5a
[komikcast] update URL pattern and unescape image URLs
6 years ago
70f3617d88
[mangafox] fix URL extraction
6 years ago
a62bd81e9b
[pixiv] fix filter for 'type=all'
6 years ago
12797e3b1f
update configuration.rst
...
... again
- some more 'Path' references
- fixed some inconsistencies and errors
- added note about logging config for files
6 years ago
c43f02245f
update configuration.rst
...
- fix default values for 'log' and 'unsupportedfile'
[ci skip]
6 years ago
dacda69c9e
update configuration.rst
...
- document logging options
- add a section for "custom types"
[ci skip]
6 years ago
55b0913412
[simplyhentai] add gallery extractor ( #89 )
6 years ago
ae9a37a528
implement text.split_html()
6 years ago
53f36176fd
update configuration.rst
...
- update the API Tokens & IDs section
- mention redirect URIs for deviantart
- include api-secret for tumblr
- add instructions for smugmug
- [ci skip]
6 years ago
b08d95ebe4
add an 'encoding' option for logging files (default 'utf-8')
6 years ago
513d807632
explicitly open config files as utf-8
6 years ago
2df1a15fb8
add '-s/--simulate' to run data extraction without download
...
Useful for quick testing (even though -g and -j kind of do the same)
and to fill a download archive without actually downloading the files.
-s does the same as the default behaviour, except downloading stuff.
Maybe it should get a more fitting name, as it does actually write to
disk (cache, archive)?
6 years ago
15cce22d82
[mangadex] fix parsing of unusual chapter strings
6 years ago
ecdc3475b8
[pixhost] support .to TLDs
6 years ago
aab2391c7b
Fix UnicodeDecodeError during installation ( #86 )
...
* fix UnicodeDecodeError during installation
* simplify opening with utf-8 encoding
6 years ago
f3d770d4e2
Merge branch '1.4-dev'
6 years ago
d0ae3ed52c
[postprocessor] add 'zip' to write files to a ZIP archive
...
(#85 )
6 years ago
ca4008e1c1
[postprocessor] add 'classify' to sort downloads by fileext
6 years ago
d378c0a323
[postprocessor] add 'exec' to execute user-defined processes
6 years ago
76c32d58e5
[postprocessor] initial code
6 years ago
1ff626db97
[pixiv] improve bookmark extraction
...
- combine 'favorite' and 'bookmark' extractors
- it is now one extractor class, but its subcategory still
distinguishes between your own bookmarks ('bookmark') and other
user's bookmarks ('favorite') like before
- allow filtering by bookmark tags and public/private bookmarks
- fix pagination for bookmark results
6 years ago
0a1863fce3
[pixiv] respect more query parameters for user URLs
...
The API endpoint responsible for user illustrations does not
provide sufficient filter capabilities* to match the actual
website, so we are spinning our own filters.
Respected parameters are
'type': illust, manga, ugoira
'tag' : any image tag (this was already supported)
'p' : the page to start on
*
- API can filter for illustrations and manga, but not for ugoira.
- 'offset' is applied before filtering
- no 'tag' filter
6 years ago
f43d446692
[mangahere] extract chapter titles
6 years ago
b8e53b8c6b
[pixiv] move query parsing out of constructor
...
better exception handling, among other things
6 years ago
909d105ae6
[pixiv] add extractor for illusts from followed users
6 years ago
7f899bd5d8
Merge branch 'master' into 1.4-dev
6 years ago
Mike Fährmann