6ecb710a7f
add '-P/--postprocessor' command-line option
3 years ago
83fc4c1098
update post processor config capabilities
...
This change makes it possible to specify just the name of a post processor
in the "postprocessors" list instead of a dict with all of its options.
The options for it will then be taken from inside the "postprocessor"
block similar to "extractor", "downloader", or "output" blocks.
This makes it possible to for example override the default settings for
--write-metadata by specifying a custom "metadata" block, or to set a
custom post processor block ("cbz") and then use it by referencing just
its name in "postprocessors" lists.
{
"postprocessor":
{
"metadata": {
"name": "metadata",
"event": "post",
"filename": "{tweet_id|post_id|id}.json"
},
"cbz": {
"name" : "zip",
"compression": "store",
"extension" : "cbz"
}
}
}
3 years ago
4a747a31a3
[postprocessor:metadata] handle dicts in mode;tags ( fixes #1598 )
3 years ago
a3bf878329
[idolcomplex] improve and fix pagination ( #1601 )
...
always rely on the 'next-page-url' value and its query parameters
3 years ago
3cbbefd4ed
support 'filter' option for post processors ( #1460 )
3 years ago
4cf40434d7
initial support for conditional filenames ( #1394 )
3 years ago
0abad8bc12
implement 'compile_expression()'
3 years ago
e39c4633ba
[cyberdrop] b64decode -> a2b_base64
3 years ago
f3d692a378
improve docs for danbooru and e621 passwords ( #1593 , #1596 )
3 years ago
407627ec86
[foolfuuka] support 'archive.wakarimasen.moe' ( closes #1595 )
3 years ago
78f89d2e61
[idolcomplex] fix pagination ( closes #1594 )
3 years ago
f14a36cfc9
release version 1.17.5
3 years ago
52052a0e1a
[manganelo] update domain to 'manganato.com'
3 years ago
c80b18a477
[weibo] extend 'retweets' option ( closes #1542 )
...
Setting 'retweets' to "original" will use metadata from the
original posts, and not from the retweeted ones.
3 years ago
c0fa5058da
[kemonoparty] actually add a 'type' metadata field ( #1556 )
3 years ago
bb50e85795
[postprocessor:ugoira] optimize writing ffconcat files
...
collect all content in-memory first and
write everything with a single 'write()'
3 years ago
5c18db5fab
[postprocessor:ugoira] use 'concat' demuxer by default ( #1550 )
...
'image2' with nanasecond mtime timestamps doesn't work on Windows
3 years ago
19a11fa11d
[postprocessor:ugoira] use frame extension from filenames ( #1550 )
3 years ago
74d97e6c6b
[postprocessor:ugoira] add 'repeat-last-frame' option ( #1550 )
3 years ago
be9547a5f3
[postprocessor:ugoira] use 'image2' demuxer ( #1550 )
...
Ensures exact frame timecodes with no duplicate frames.
Possible issues are the duration the last frame in an Ugoira with variable
frame durations is shown and insufficient timestamp precision of the
underlying file system (e.g. FAT32, ext3; works on ext4, tmpfs, NTFS).
3 years ago
b8de0671e5
update description for 'instagram.include'
...
- add double quotes for "reels"
- add "tagged"
3 years ago
264beb8556
recognize v2.mangapark URLs ( #1578 )
...
* recognize v2.mangapark URLs
* update mangapark root url to use the v2 subdomain
3 years ago
e6811c7450
[pixiv] implement 'max-posts' option ( #1558 )
...
* implement max-rank for pixiv
* rename to max-posts and make more generic
3 years ago
8a909e478d
[imagebam] fix extraction of NSFW images ( #1534 )
3 years ago
b5affc62aa
[twitter] rename 'text-only' to 'text-tweets' ( #570 )
3 years ago
da6806a161
fix job tests for Python 3.4 and 3.5
...
assert_called() and assert_not_called() got added in Python 3.6
3 years ago
724ca61f36
[twitter] add 'text-only' option ( #570 )
3 years ago
8fd8126117
fix ISO 639-1 code for Japanese
...
"jp" -> "ja"
3 years ago
2c60c7d798
[reactor] skip deleted/empty posts
3 years ago
af9dba4684
add DataJob tests
3 years ago
532ac79fb0
update extractor test results
3 years ago
d7bc4a2b8b
[500px] update query hashes
3 years ago
0f35aca728
[aryion] minor code updates
3 years ago
2eb46452ad
[aryion] update 'needle' to not skip text posts ( fixes #1568 )
...
on "Latest Updates" pages
"class='thumb scrollthumb' href='/g4/view/" and
"class='thumb' href='/g4/view/" both end with
"thumb' href='/g4/view/"
3 years ago
adf4d661b3
use '_extractor' info in UrlJobs
3 years ago
4fc9668922
[imgur] update URL patterns ( #1561 )
3 years ago
1eabfa5c7a
[pillowfort] implement login with username & password ( #846 )
3 years ago
24dd10ac3c
[patreon] extract user defined 'tags' ( #1539 , closes #1540 )
3 years ago
a7e4917ee1
[pillowfort] add 'inline' option ( #846 )
...
to support images present in a post's 'content',
but not listed in 'media'.
also separates the file hash present at the beginning
of each 'filename' into its own field.
3 years ago
efa6cc8ec3
[pillowfort] add 'external' option ( #846 )
...
for links to external Twitter posts etc.
3 years ago
394fbb5f56
[twitter] strip useless t.co links ( #1532 )
...
The 'full_text' of Tweets with media content usually ends with a t.co
link to itself. This commit removes those.
3 years ago
3a7c3ff138
support XDG_CONFIG_HOME ( closes #1545 )
...
This will only load either
${XDG_CONFIG_HOME}/gallery-dl/config.json or
${HOME}/.config/gallery-dl/config.json
if XDG_CONFIG_HOME is empty, never both.
3 years ago
41457dbb1b
[twitter] resolve t.co URLs in 'content' ( #1532 )
3 years ago
2b5d80862e
[kemonoparty] add 'type' metadata field ( #1556 )
...
'file', 'attachment', or 'inline'
3 years ago
17b0ccb071
[twitter] add missing retweet media entities ( fixes #1555 )
...
from the original tweets
3 years ago
5eeaaee01d
[pixiv] add 'metadata' option ( #1551 )
3 years ago
0717456b4e
[kemonoparty] add 'metadata' option ( closes #1548 )
...
to fetch creator names with an additional HTTP request
3 years ago
559462789d
add some tests for job.py
3 years ago
b50b8e6cf4
refactor applying 'parent-…' options
3 years ago
7ab8374385
add 'parent-skip' option ( #1399 )
3 years ago
Mike Fährmann