Mike Fährmann
ff8d0d7280
[postprocessor:metadata] ensure reliable tags order ( fixes #1598 )
3 years ago
Mike Fährmann
4a747a31a3
[postprocessor:metadata] handle dicts in mode;tags ( fixes #1598 )
3 years ago
Mike Fährmann
3cbbefd4ed
support 'filter' option for post processors ( #1460 )
3 years ago
Mike Fährmann
bb50e85795
[postprocessor:ugoira] optimize writing ffconcat files
...
collect all content in-memory first and
write everything with a single 'write()'
3 years ago
Mike Fährmann
5c18db5fab
[postprocessor:ugoira] use 'concat' demuxer by default ( #1550 )
...
'image2' with nanasecond mtime timestamps doesn't work on Windows
3 years ago
Mike Fährmann
19a11fa11d
[postprocessor:ugoira] use frame extension from filenames ( #1550 )
3 years ago
Mike Fährmann
74d97e6c6b
[postprocessor:ugoira] add 'repeat-last-frame' option ( #1550 )
3 years ago
Mike Fährmann
be9547a5f3
[postprocessor:ugoira] use 'image2' demuxer ( #1550 )
...
Ensures exact frame timecodes with no duplicate frames.
Possible issues are the duration the last frame in an Ugoira with variable
frame durations is shown and insufficient timestamp precision of the
underlying file system (e.g. FAT32, ext3; works on ext4, tmpfs, NTFS).
3 years ago
Mike Fährmann
e5123f56c9
fix crash when using --no-download with --ugoira-conv ( #1507 )
3 years ago
Mike Fährmann
8821dceb79
use __import__() to dynamically load modules
4 years ago
Mike Fährmann
1bd3d7cfb0
[postprocessor:metadata] call expand_path() on custom paths
...
(#1299 )
4 years ago
Mike Fährmann
fbfcbcbf57
Merge branch '1.17.0'
4 years ago
Mike Fährmann
b3cd970d87
[postprocessor:metadata] fix crash with 'extension-format'
...
Using the 'extension-format' option for events where no filename
extension is available caused a crash.
(fixes #1285 )
4 years ago
Mike Fährmann
ebf417f31f
remove support for deprecated options
...
- instagram.highlights
- metadata.bypost
- exec.final
4 years ago
Mike Fährmann
0fdaea00a3
[postprocessor:metadata] sanitize filenames
4 years ago
Mike Fährmann
e8c64dd961
[postprocessor:exec] do not auto-add '{}' to command ( #1185 )
...
This was initially done to mimic youtube-dl's behavior and
implementation of --exec, and it seemed reasonable at the time.
4 years ago
Mike Fährmann
0a3bbc9c63
[postprocessor:exec] update output
4 years ago
Mike Fährmann
b6f1fe59cb
add deprecation warnings for exec.final and metadata.bypost
4 years ago
Mike Fährmann
ca59bd691c
[postprocessor:metadata] add 'event' and 'filename' options
4 years ago
Mike Fährmann
9c3568c397
[postprocessor:exec] add 'event' option
...
and remove 'final' option -- use '"event": "finalize"' instead.
4 years ago
Mike Fährmann
9fffa9c343
rework post processor callbacks
4 years ago
Mike Fährmann
d83b95fd28
[postprocessor:metadata] accept a string-list for 'content-format'
...
(closes #1080 )
4 years ago
Mike Fährmann
fd0685d9b5
[postprocessor:zip] defer zip file creation ( fixes #968 )
...
don't try to create zip files on postprocessor construction,
wait until directory creation during file download,
4 years ago
Mike Fährmann
c8787647ed
add global WINDOWS bool
4 years ago
Mike Fährmann
ece73b5b2a
make 'path' and 'keywords' available in logging messages
...
Wrap all loggers used by job, extractor, downloader, and postprocessor
objects into a (custom) LoggerAdapter that provides access to the
underlying job, extractor, pathfmt, and kwdict objects and their
properties.
__init__() signatures for all downloader and postprocessor classes have
been changed to take the current Job object as their first argument,
instead of the current extractor or pathfmt.
(#574 , #575 )
4 years ago
Mike Fährmann
4a3d2405de
[postprocessor:ugoira] small optimization
...
Use tuples instead of lists when extending the list of
command-line arguments.
5 years ago
Mike Fährmann
254f7c3999
implement a post-processor module to compare file versions
...
(#530 )
5 years ago
Mike Fährmann
760b9b4db4
add remove_file() and remove_directory() helpers
...
these functions call os.unlink() or os.rmdir()
while catching and suppressing potential OSErrors
5 years ago
Mike Fährmann
87c8b89ddd
[postprocessor:metadata] add 'directory' option ( #520 )
5 years ago
Mike Fährmann
63e6993716
merge 'bypost' functionality into metadata postprocessor
5 years ago
Gio
c0b9ad678d
Separate metadata from handle_url into handle_metadata, commenting
5 years ago
Gio
c20bb5c338
Naming convention, as per travis.
5 years ago
Gio
cfc70a97ab
Added an additional channel for downloading the metadata of an entire post or gallery.
5 years ago
Mike Fährmann
26d2334550
[postprocessor:metadata] rename 'format' to 'content-format'
...
Just to be consistent with the other 'extension-format' option name,
and only 'format' is also still accepted.
5 years ago
Mike Fährmann
a412531451
[postprocessor:metadata] implement 'extension-format' option
...
closes #477
5 years ago
Mike Fährmann
3fc1e12949
[postprocessor:metadata] filter private entries
...
i.e. keys starting with an underscore
5 years ago
Mike Fährmann
9e88e7a344
[postprocessor:exec] improve ( #421 , #413 )
...
- add 'final' option
- include job status in pp finalization
- improve and extend documentation
5 years ago
Mike Fährmann
ad81c07204
[postprocessor] match logger names of downloader modules
...
The logger name for a postprocessor object got changed to
"postprocessor.<module-name>" instead of just
"postprocessor"
5 years ago
Mike Fährmann
03bc8adfc7
[postprocessor:exec] run after file moved to target location
...
(#421 )
5 years ago
Mike Fährmann
35958bebd4
[postprocessor:exec] fix filename quoting on Windows ( #421 )
5 years ago
Mike Fährmann
b06c372e4d
[postprocessor:exec] improve; add command-line option ( #421 )
5 years ago
Mike Fährmann
5fcebb69c2
[postprocessor:ugoira] improve error messages ( #406 )
5 years ago
Mike Fährmann
60c8e090da
[postprocessor:zip] fix archive names ( closes #397 )
...
Remove the trailing path separator introduced in 3284c62
before
adding the archive's filename extension.
[ci skip]
5 years ago
Mike Fährmann
3284c62f22
ensure PathFormat.directory ends with a path separator
...
... plus some other small optimizations
5 years ago
Mike Fährmann
2495b99347
[postprocessor:classify] improve path generation ( fixes #138 )
...
It still doesn't work for converted ugoira animations thanks to how
those files are handled, but everything else, including files with
unknown or changing file extension, now works as it should.
5 years ago
Mike Fährmann
5f8621b29d
improve output of active post processor modules
5 years ago
Mike Fährmann
0bb873757a
update PathFormat class
...
- change 'has_extension' from a simple flag/bool to a field that
contains the original filename extension
- rename 'keywords' to 'kwdict' and some other stuff as well
- inline 'adjust_path()'
- put enumeration index before filename extension (#306 )
5 years ago
Mike Fährmann
a90280f4e7
[postprocessor:zip] add 'mode' option ( #355 )
5 years ago
Mike Fährmann
c41ff9441e
improve find() for downloaders and postprocessors
5 years ago
Mike Fährmann
16c582aaf9
implement 'mtime' post-processor ( #332 )
...
This can set a file's modification time according to a UNIX timestamp
or a datetime object from its metadata.
5 years ago
Mike Fährmann
a01f99728c
[postprocessor:zip] delete empty archives when done ( #316 )
5 years ago
Mike Fährmann
b1db194c14
[reactor] update and improve
...
- split 'tags' into a list
- parse 'date' into a datetime object
- fix webm/mp4 URLs
5 years ago
Mike Fährmann
523ebc9b0b
Fix serialization of 'datetime' objects in '--write-metadata'
...
Simplified universal serialization support in json.dump() can be achieved
by passing 'default=str', which was already the case in DataJob.run()
for -j/--dump-json, but not for the 'metadata' post-processor.
This commit introduces util.dump_json() that (more or less) unifies the
JSON output procedure of both --write-metadata and --dump-json.
(#251 , #252 )
5 years ago
Mike Fährmann
7973419b54
restrict downloader and postprocessor module imports
6 years ago
Mike Fährmann
8aba2bdebf
[postprocessor:metadata] add 'tags' and 'custom' modes ( #135 )
6 years ago
Mike Fährmann
5f38ac9609
[postprocessor:exec] add a better error message ( #155 )
6 years ago
Mike Fährmann
38500ad697
[postprocessor:metadata] first implementation ( #135 )
6 years ago
Mike Fährmann
79f6755c60
[postprocessor:classify] handle missing "extension" ( #138 )
6 years ago
Mike Fährmann
d3d7f01543
add 'prepare()' step for post-processors
...
This allows post-processors to modify the destination path before
checking if a file already exists.
6 years ago
Mike Fährmann
09d2f3e5e7
[postprocessor:ugoira] improve libx264 detection
6 years ago
Mike Fährmann
55f5c87160
[postprocessor:ugoira] add 'libx264-prevent-odd' option
...
A rather crude workaround for "width/height not divisible by 2"
errors when using libx264.
6 years ago
Mike Fährmann
e9ae6fd080
improve downloader/postprocessor module loading
...
- handle arguments of any type without propagating an exception
- prevent potential security risk through relative imports
6 years ago
Mike Fährmann
6ecb36d88c
[postprocessor:ugoira] add 'ffmpeg-output' option
6 years ago
Mike Fährmann
02a4a67f6d
[postprocessor:ugoira] support danbooru sources
6 years ago
Mike Fährmann
54a0d72dc8
[postprocessor:ugoira] improve frame rate handling
...
By default FFmpeg assumes a 25 FPS input frame rate, leading to dropped
frames if the source requires a higher frame rate than that.
This commit adds a `framerate` option (default "auto"), which allows to
automatically assign a (more or less) fitting frame rate based on
delays between ugoira frames and avoids dropped frames.
6 years ago
Mike Fährmann
4eb94aca17
[postprocessor:ugoira] pass '-f' if not present
6 years ago
Mike Fährmann
0c1c4557dd
[postprocessor:ugoira] add option for two-pass encoding
6 years ago
Mike Fährmann
eb7a1f3b98
[pixiv] rework ugoira handling
...
Frame information now gets attached to the ZIP file's keyword dict
instead of being written to a separate text file.
6 years ago
Mike Fährmann
613b692275
[postprocessor:ugoira] add a few options
...
- ffmpeg-location: path to the ffmpeg (or avconv) executable
- ffmpeg-args: additional command line args for ffmpeg
- extension: filename extension of the resulting video file
6 years ago
Mike Fährmann
a444755979
[postprocessor] add 'ugoira' to convert pixiv animations to webm
6 years ago
Mike Fährmann
eec081dd3e
[postprocessor:zip] delete directory ( #85 )
6 years ago
Mike Fährmann
baccf8a958
improve postprocessor handling
...
- add pathfmt argument for __init__()
- add finalization step
- add option to keep or delete zipped files
6 years ago
Mike Fährmann
2628911ba0
[pp:exec] add 'async' option
6 years ago
Mike Fährmann
37d97ff02c
[pp:classify] use temppath
6 years ago
Mike Fährmann
97189e50cd
[pp:zip] use temppath; add options
6 years ago
Mike Fährmann
d0ae3ed52c
[postprocessor] add 'zip' to write files to a ZIP archive
...
(#85 )
6 years ago
Mike Fährmann
ca4008e1c1
[postprocessor] add 'classify' to sort downloads by fileext
6 years ago
Mike Fährmann
d378c0a323
[postprocessor] add 'exec' to execute user-defined processes
6 years ago
Mike Fährmann
76c32d58e5
[postprocessor] initial code
6 years ago