e45792a5c4
[postprocessor:ugoira] insert extra frame
...
into files generated with the 'image2' demuxer to compensate for the
last frame not being shown for as long as it should.
This only happens for ugoira with non-uniform delays between frames and
only when 'repeat-last-frame' is enabled.
3 years ago
e718dd7b32
[postprocessor:ugoira] support setting timecodes with mkvmerge
...
by selecting the "mkvmerge" demuxer
(#1550 )
3 years ago
e7b30866d0
[postprocessor:mtime] fix timestamps from datetime objects ( #2307 )
...
'datetime.timestamp()', which got used to convert datetime objects to
POSIX timestamps, assumes naive datetimes represent LOCAL time, while
datetimes in 'date' metadata fields represent UTC time.
Ref: https://docs.python.org/3/library/datetime.html#datetime.datetime.timestamp
> Naive datetime instances are assumed to represent local time
> you can obtain the POSIX timestamp by … calculating the timestamp directly
3 years ago
40ce505420
[postprocessor:ugoira] add 'mtime' option ( #2307 )
3 years ago
9bd27b1b8d
[postprocessor:metadata] implement archive options ( #2421 )
...
'archive', 'archive-format', and 'archive-prefix'
3 years ago
59749555b1
[postprocessor:mtime] add 'event' option ( #2307 )
3 years ago
7958995398
[postprocessor:metadata] implement 'mtime' option ( #2307 )
3 years ago
df8050b81d
[postprocessor:compare] add 'equal' option ( #1592 )
...
Move functionality from cdd72e14
3 years ago
cdd72e1413
[postprocessor:compare] extend 'action' option ( #1592 )
...
allow setting it to "abort", "terminate", or "exit" as with 'skip'
3 years ago
74145467dd
move 'util.Formatter' into its own 'formatter' module
3 years ago
2ca011dfa8
add 'kwdict' argument to PathFormat.build_filename()
3 years ago
b93cbe6720
[postprocessor:zip] run 'finalize()' regardless of 'filter' ( #1460 )
3 years ago
ff8d0d7280
[postprocessor:metadata] ensure reliable tags order ( fixes #1598 )
3 years ago
4a747a31a3
[postprocessor:metadata] handle dicts in mode;tags ( fixes #1598 )
3 years ago
3cbbefd4ed
support 'filter' option for post processors ( #1460 )
3 years ago
bb50e85795
[postprocessor:ugoira] optimize writing ffconcat files
...
collect all content in-memory first and
write everything with a single 'write()'
3 years ago
5c18db5fab
[postprocessor:ugoira] use 'concat' demuxer by default ( #1550 )
...
'image2' with nanasecond mtime timestamps doesn't work on Windows
3 years ago
19a11fa11d
[postprocessor:ugoira] use frame extension from filenames ( #1550 )
3 years ago
74d97e6c6b
[postprocessor:ugoira] add 'repeat-last-frame' option ( #1550 )
3 years ago
be9547a5f3
[postprocessor:ugoira] use 'image2' demuxer ( #1550 )
...
Ensures exact frame timecodes with no duplicate frames.
Possible issues are the duration the last frame in an Ugoira with variable
frame durations is shown and insufficient timestamp precision of the
underlying file system (e.g. FAT32, ext3; works on ext4, tmpfs, NTFS).
3 years ago
e5123f56c9
fix crash when using --no-download with --ugoira-conv ( #1507 )
3 years ago
8821dceb79
use __import__() to dynamically load modules
4 years ago
1bd3d7cfb0
[postprocessor:metadata] call expand_path() on custom paths
...
(#1299 )
4 years ago
fbfcbcbf57
Merge branch '1.17.0'
4 years ago
b3cd970d87
[postprocessor:metadata] fix crash with 'extension-format'
...
Using the 'extension-format' option for events where no filename
extension is available caused a crash.
(fixes #1285 )
4 years ago
ebf417f31f
remove support for deprecated options
...
- instagram.highlights
- metadata.bypost
- exec.final
4 years ago
0fdaea00a3
[postprocessor:metadata] sanitize filenames
4 years ago
e8c64dd961
[postprocessor:exec] do not auto-add '{}' to command ( #1185 )
...
This was initially done to mimic youtube-dl's behavior and
implementation of --exec, and it seemed reasonable at the time.
4 years ago
0a3bbc9c63
[postprocessor:exec] update output
4 years ago
b6f1fe59cb
add deprecation warnings for exec.final and metadata.bypost
4 years ago
ca59bd691c
[postprocessor:metadata] add 'event' and 'filename' options
4 years ago
9c3568c397
[postprocessor:exec] add 'event' option
...
and remove 'final' option -- use '"event": "finalize"' instead.
4 years ago
9fffa9c343
rework post processor callbacks
4 years ago
d83b95fd28
[postprocessor:metadata] accept a string-list for 'content-format'
...
(closes #1080 )
4 years ago
fd0685d9b5
[postprocessor:zip] defer zip file creation ( fixes #968 )
...
don't try to create zip files on postprocessor construction,
wait until directory creation during file download,
4 years ago
c8787647ed
add global WINDOWS bool
4 years ago
ece73b5b2a
make 'path' and 'keywords' available in logging messages
...
Wrap all loggers used by job, extractor, downloader, and postprocessor
objects into a (custom) LoggerAdapter that provides access to the
underlying job, extractor, pathfmt, and kwdict objects and their
properties.
__init__() signatures for all downloader and postprocessor classes have
been changed to take the current Job object as their first argument,
instead of the current extractor or pathfmt.
(#574 , #575 )
4 years ago
4a3d2405de
[postprocessor:ugoira] small optimization
...
Use tuples instead of lists when extending the list of
command-line arguments.
5 years ago
254f7c3999
implement a post-processor module to compare file versions
...
(#530 )
5 years ago
760b9b4db4
add remove_file() and remove_directory() helpers
...
these functions call os.unlink() or os.rmdir()
while catching and suppressing potential OSErrors
5 years ago
87c8b89ddd
[postprocessor:metadata] add 'directory' option ( #520 )
5 years ago
63e6993716
merge 'bypost' functionality into metadata postprocessor
5 years ago
c0b9ad678d
Separate metadata from handle_url into handle_metadata, commenting
5 years ago
c20bb5c338
Naming convention, as per travis.
5 years ago
cfc70a97ab
Added an additional channel for downloading the metadata of an entire post or gallery.
5 years ago
26d2334550
[postprocessor:metadata] rename 'format' to 'content-format'
...
Just to be consistent with the other 'extension-format' option name,
and only 'format' is also still accepted.
5 years ago
a412531451
[postprocessor:metadata] implement 'extension-format' option
...
closes #477
5 years ago
3fc1e12949
[postprocessor:metadata] filter private entries
...
i.e. keys starting with an underscore
5 years ago
9e88e7a344
[postprocessor:exec] improve ( #421 , #413 )
...
- add 'final' option
- include job status in pp finalization
- improve and extend documentation
5 years ago
ad81c07204
[postprocessor] match logger names of downloader modules
...
The logger name for a postprocessor object got changed to
"postprocessor.<module-name>" instead of just
"postprocessor"
5 years ago
03bc8adfc7
[postprocessor:exec] run after file moved to target location
...
(#421 )
5 years ago
35958bebd4
[postprocessor:exec] fix filename quoting on Windows ( #421 )
5 years ago
b06c372e4d
[postprocessor:exec] improve; add command-line option ( #421 )
5 years ago
5fcebb69c2
[postprocessor:ugoira] improve error messages ( #406 )
5 years ago
60c8e090da
[postprocessor:zip] fix archive names ( closes #397 )
...
Remove the trailing path separator introduced in 3284c62
5 years ago
3284c62f22
ensure PathFormat.directory ends with a path separator
...
... plus some other small optimizations
5 years ago
2495b99347
[postprocessor:classify] improve path generation ( fixes #138 )
...
It still doesn't work for converted ugoira animations thanks to how
those files are handled, but everything else, including files with
unknown or changing file extension, now works as it should.
5 years ago
5f8621b29d
improve output of active post processor modules
5 years ago
0bb873757a
update PathFormat class
...
- change 'has_extension' from a simple flag/bool to a field that
contains the original filename extension
- rename 'keywords' to 'kwdict' and some other stuff as well
- inline 'adjust_path()'
- put enumeration index before filename extension (#306 )
5 years ago
a90280f4e7
[postprocessor:zip] add 'mode' option ( #355 )
5 years ago
c41ff9441e
improve find() for downloaders and postprocessors
5 years ago
16c582aaf9
implement 'mtime' post-processor ( #332 )
...
This can set a file's modification time according to a UNIX timestamp
or a datetime object from its metadata.
5 years ago
a01f99728c
[postprocessor:zip] delete empty archives when done ( #316 )
5 years ago
b1db194c14
[reactor] update and improve
...
- split 'tags' into a list
- parse 'date' into a datetime object
- fix webm/mp4 URLs
5 years ago
523ebc9b0b
Fix serialization of 'datetime' objects in '--write-metadata'
...
Simplified universal serialization support in json.dump() can be achieved
by passing 'default=str', which was already the case in DataJob.run()
for -j/--dump-json, but not for the 'metadata' post-processor.
This commit introduces util.dump_json() that (more or less) unifies the
JSON output procedure of both --write-metadata and --dump-json.
(#251 , #252 )
5 years ago
7973419b54
restrict downloader and postprocessor module imports
6 years ago
8aba2bdebf
[postprocessor:metadata] add 'tags' and 'custom' modes ( #135 )
6 years ago
5f38ac9609
[postprocessor:exec] add a better error message ( #155 )
6 years ago
38500ad697
[postprocessor:metadata] first implementation ( #135 )
6 years ago
79f6755c60
[postprocessor:classify] handle missing "extension" ( #138 )
6 years ago
d3d7f01543
add 'prepare()' step for post-processors
...
This allows post-processors to modify the destination path before
checking if a file already exists.
6 years ago
09d2f3e5e7
[postprocessor:ugoira] improve libx264 detection
6 years ago
55f5c87160
[postprocessor:ugoira] add 'libx264-prevent-odd' option
...
A rather crude workaround for "width/height not divisible by 2"
errors when using libx264.
6 years ago
e9ae6fd080
improve downloader/postprocessor module loading
...
- handle arguments of any type without propagating an exception
- prevent potential security risk through relative imports
6 years ago
6ecb36d88c
[postprocessor:ugoira] add 'ffmpeg-output' option
6 years ago
02a4a67f6d
[postprocessor:ugoira] support danbooru sources
6 years ago
54a0d72dc8
[postprocessor:ugoira] improve frame rate handling
...
By default FFmpeg assumes a 25 FPS input frame rate, leading to dropped
frames if the source requires a higher frame rate than that.
This commit adds a `framerate` option (default "auto"), which allows to
automatically assign a (more or less) fitting frame rate based on
delays between ugoira frames and avoids dropped frames.
6 years ago
4eb94aca17
[postprocessor:ugoira] pass '-f' if not present
6 years ago
0c1c4557dd
[postprocessor:ugoira] add option for two-pass encoding
6 years ago
eb7a1f3b98
[pixiv] rework ugoira handling
...
Frame information now gets attached to the ZIP file's keyword dict
instead of being written to a separate text file.
6 years ago
613b692275
[postprocessor:ugoira] add a few options
...
- ffmpeg-location: path to the ffmpeg (or avconv) executable
- ffmpeg-args: additional command line args for ffmpeg
- extension: filename extension of the resulting video file
6 years ago
a444755979
[postprocessor] add 'ugoira' to convert pixiv animations to webm
6 years ago
eec081dd3e
[postprocessor:zip] delete directory ( #85 )
6 years ago
baccf8a958
improve postprocessor handling
...
- add pathfmt argument for __init__()
- add finalization step
- add option to keep or delete zipped files
6 years ago
2628911ba0
[pp:exec] add 'async' option
6 years ago
37d97ff02c
[pp:classify] use temppath
6 years ago
97189e50cd
[pp:zip] use temppath; add options
6 years ago
d0ae3ed52c
[postprocessor] add 'zip' to write files to a ZIP archive
...
(#85 )
6 years ago
ca4008e1c1
[postprocessor] add 'classify' to sort downloads by fileext
6 years ago
d378c0a323
[postprocessor] add 'exec' to execute user-defined processes
6 years ago
76c32d58e5
[postprocessor] initial code
6 years ago
Mike Fährmann