Mike Fährmann
8821dceb79
use __import__() to dynamically load modules
4 years ago
Mike Fährmann
1bd3d7cfb0
[postprocessor:metadata] call expand_path() on custom paths
...
(#1299 )
4 years ago
Mike Fährmann
fbfcbcbf57
Merge branch '1.17.0'
4 years ago
Mike Fährmann
b3cd970d87
[postprocessor:metadata] fix crash with 'extension-format'
...
Using the 'extension-format' option for events where no filename
extension is available caused a crash.
(fixes #1285 )
4 years ago
Mike Fährmann
ebf417f31f
remove support for deprecated options
...
- instagram.highlights
- metadata.bypost
- exec.final
4 years ago
Mike Fährmann
0fdaea00a3
[postprocessor:metadata] sanitize filenames
4 years ago
Mike Fährmann
e8c64dd961
[postprocessor:exec] do not auto-add '{}' to command ( #1185 )
...
This was initially done to mimic youtube-dl's behavior and
implementation of --exec, and it seemed reasonable at the time.
4 years ago
Mike Fährmann
0a3bbc9c63
[postprocessor:exec] update output
4 years ago
Mike Fährmann
b6f1fe59cb
add deprecation warnings for exec.final and metadata.bypost
4 years ago
Mike Fährmann
ca59bd691c
[postprocessor:metadata] add 'event' and 'filename' options
4 years ago
Mike Fährmann
9c3568c397
[postprocessor:exec] add 'event' option
...
and remove 'final' option -- use '"event": "finalize"' instead.
4 years ago
Mike Fährmann
9fffa9c343
rework post processor callbacks
4 years ago
Mike Fährmann
d83b95fd28
[postprocessor:metadata] accept a string-list for 'content-format'
...
(closes #1080 )
4 years ago
Mike Fährmann
fd0685d9b5
[postprocessor:zip] defer zip file creation ( fixes #968 )
...
don't try to create zip files on postprocessor construction,
wait until directory creation during file download,
4 years ago
Mike Fährmann
c8787647ed
add global WINDOWS bool
4 years ago
Mike Fährmann
ece73b5b2a
make 'path' and 'keywords' available in logging messages
...
Wrap all loggers used by job, extractor, downloader, and postprocessor
objects into a (custom) LoggerAdapter that provides access to the
underlying job, extractor, pathfmt, and kwdict objects and their
properties.
__init__() signatures for all downloader and postprocessor classes have
been changed to take the current Job object as their first argument,
instead of the current extractor or pathfmt.
(#574 , #575 )
4 years ago
Mike Fährmann
4a3d2405de
[postprocessor:ugoira] small optimization
...
Use tuples instead of lists when extending the list of
command-line arguments.
5 years ago
Mike Fährmann
254f7c3999
implement a post-processor module to compare file versions
...
(#530 )
5 years ago
Mike Fährmann
760b9b4db4
add remove_file() and remove_directory() helpers
...
these functions call os.unlink() or os.rmdir()
while catching and suppressing potential OSErrors
5 years ago
Mike Fährmann
87c8b89ddd
[postprocessor:metadata] add 'directory' option ( #520 )
5 years ago
Mike Fährmann
63e6993716
merge 'bypost' functionality into metadata postprocessor
5 years ago
Gio
c0b9ad678d
Separate metadata from handle_url into handle_metadata, commenting
5 years ago
Gio
c20bb5c338
Naming convention, as per travis.
5 years ago
Gio
cfc70a97ab
Added an additional channel for downloading the metadata of an entire post or gallery.
5 years ago
Mike Fährmann
26d2334550
[postprocessor:metadata] rename 'format' to 'content-format'
...
Just to be consistent with the other 'extension-format' option name,
and only 'format' is also still accepted.
5 years ago
Mike Fährmann
a412531451
[postprocessor:metadata] implement 'extension-format' option
...
closes #477
5 years ago
Mike Fährmann
3fc1e12949
[postprocessor:metadata] filter private entries
...
i.e. keys starting with an underscore
5 years ago
Mike Fährmann
9e88e7a344
[postprocessor:exec] improve ( #421 , #413 )
...
- add 'final' option
- include job status in pp finalization
- improve and extend documentation
5 years ago
Mike Fährmann
ad81c07204
[postprocessor] match logger names of downloader modules
...
The logger name for a postprocessor object got changed to
"postprocessor.<module-name>" instead of just
"postprocessor"
5 years ago
Mike Fährmann
03bc8adfc7
[postprocessor:exec] run after file moved to target location
...
(#421 )
5 years ago
Mike Fährmann
35958bebd4
[postprocessor:exec] fix filename quoting on Windows ( #421 )
5 years ago
Mike Fährmann
b06c372e4d
[postprocessor:exec] improve; add command-line option ( #421 )
5 years ago
Mike Fährmann
5fcebb69c2
[postprocessor:ugoira] improve error messages ( #406 )
5 years ago
Mike Fährmann
60c8e090da
[postprocessor:zip] fix archive names ( closes #397 )
...
Remove the trailing path separator introduced in 3284c62
before
adding the archive's filename extension.
[ci skip]
5 years ago
Mike Fährmann
3284c62f22
ensure PathFormat.directory ends with a path separator
...
... plus some other small optimizations
5 years ago
Mike Fährmann
2495b99347
[postprocessor:classify] improve path generation ( fixes #138 )
...
It still doesn't work for converted ugoira animations thanks to how
those files are handled, but everything else, including files with
unknown or changing file extension, now works as it should.
5 years ago
Mike Fährmann
5f8621b29d
improve output of active post processor modules
5 years ago
Mike Fährmann
0bb873757a
update PathFormat class
...
- change 'has_extension' from a simple flag/bool to a field that
contains the original filename extension
- rename 'keywords' to 'kwdict' and some other stuff as well
- inline 'adjust_path()'
- put enumeration index before filename extension (#306 )
5 years ago
Mike Fährmann
a90280f4e7
[postprocessor:zip] add 'mode' option ( #355 )
5 years ago
Mike Fährmann
c41ff9441e
improve find() for downloaders and postprocessors
5 years ago
Mike Fährmann
16c582aaf9
implement 'mtime' post-processor ( #332 )
...
This can set a file's modification time according to a UNIX timestamp
or a datetime object from its metadata.
5 years ago
Mike Fährmann
a01f99728c
[postprocessor:zip] delete empty archives when done ( #316 )
5 years ago
Mike Fährmann
b1db194c14
[reactor] update and improve
...
- split 'tags' into a list
- parse 'date' into a datetime object
- fix webm/mp4 URLs
5 years ago
Mike Fährmann
523ebc9b0b
Fix serialization of 'datetime' objects in '--write-metadata'
...
Simplified universal serialization support in json.dump() can be achieved
by passing 'default=str', which was already the case in DataJob.run()
for -j/--dump-json, but not for the 'metadata' post-processor.
This commit introduces util.dump_json() that (more or less) unifies the
JSON output procedure of both --write-metadata and --dump-json.
(#251 , #252 )
5 years ago
Mike Fährmann
7973419b54
restrict downloader and postprocessor module imports
6 years ago
Mike Fährmann
8aba2bdebf
[postprocessor:metadata] add 'tags' and 'custom' modes ( #135 )
6 years ago
Mike Fährmann
5f38ac9609
[postprocessor:exec] add a better error message ( #155 )
6 years ago
Mike Fährmann
38500ad697
[postprocessor:metadata] first implementation ( #135 )
6 years ago
Mike Fährmann
79f6755c60
[postprocessor:classify] handle missing "extension" ( #138 )
6 years ago
Mike Fährmann
d3d7f01543
add 'prepare()' step for post-processors
...
This allows post-processors to modify the destination path before
checking if a file already exists.
6 years ago