Mike Fährmann
00fbc942e5
add 'timedelta' class to the global namespace in filter expressions
...
(https://github.com/mikf/gallery-dl/issues/2308#issuecomment-1046135924 )
3 years ago
Mike Fährmann
7958995398
[postprocessor:metadata] implement 'mtime' option ( #2307 )
3 years ago
Mike Fährmann
64cf26eaf4
allow specifying sleep-* options as string
...
either as single value or as range: "3.5", "2.1 - 5.0"
3 years ago
Mike Fährmann
010d65dcec
extend blacklist/whitelist syntax ( #2025 )
...
Each entry in such a list can now also include a subcategory
'<category>:<subcategory>'
and it is possible to use '*' or an empty string as placeholder
'*:<subcategory>', ':<subcategory>', '<category>:*'
For example
"blacklist": "imgur,*:tag,gfycat:user" or
"blacklist": ["imgur", "*:tag", "gfycat:user"]
will filter all 'imgur' extractors, all extractors with a 'tag'
subcategory (e.g. https://danbooru.donmai.us/posts?tags=bonocho ),
and all 'gfycat' user extractors.
3 years ago
Mike Fährmann
c22ff97743
remove 'unit' argument from 'util.format_value()'
3 years ago
Mike Fährmann
cad85640de
move 'util.PathFormat' into its own 'path' module
...
to prevent circular imports between 'formatter' and 'util'
3 years ago
Mike Fährmann
74145467dd
move 'util.Formatter' into its own 'formatter' module
3 years ago
Mike Fährmann
e69ee41f25
implement 'page-reverse' option ( #1854 )
3 years ago
Mike Fährmann
c9e6693530
allow specifying a minimum/maximum for 'sleep-*' options ( #1835 )
...
for example '"sleep-request": [5.0, 10.0]' to wait between 5 and 10
seconds between each HTTP request
3 years ago
Mike Fährmann
292fffc83c
add 'j' format string conversion
...
to convert to a JSON formatted string
3 years ago
Mike Fährmann
d3eab417ed
implement a 'path-strip' option
3 years ago
Mike Fährmann
2792ed6e4b
implement 'util.format_value()'
3 years ago
Mike Fährmann
13d4045a8a
add 'archive-prefix' option ( #1711 )
3 years ago
Mike Fährmann
9e42cd58ea
replace ChainPredicate class with 'functools.partial'
3 years ago
Mike Fährmann
1b2f9050fb
rename all instances of 'kwds' to 'kwdict'
3 years ago
Mike Fährmann
0179581340
add 'T' format string conversion ( #1646 )
...
to convert 'date'/datetime to timestamp
3 years ago
Mike Fährmann
befe635022
cache parsed Formatter functions
3 years ago
Mike Fährmann
79b7ee2712
use 'functools.partial' in '_build_cleanfunc' when possible
...
makes calls to the returned function a slight bit faster (~10%)
3 years ago
Mike Fährmann
ceaf7fd989
optimize 'base-directory' initialization and usage
...
apply 'clean_path()' only once
3 years ago
Mike Fährmann
2ca011dfa8
add 'kwdict' argument to PathFormat.build_filename()
3 years ago
Mike Fährmann
fd00d47116
implement conditional directories ( #1394 )
...
They work the same way as conditional filenames (84d2e640
), e.g.
"directory": {
"score >= 20": ["high score"],
"score >= 5" : ["mid score"],
"" : ["{category}", "default"]
}
3 years ago
Mike Fährmann
def0148582
restructure code in PathFormat constructor
3 years ago
Mike Fährmann
84d2e64024
combine conditional filenames into `filename` option ( #1394 )
3 years ago
Mike Fährmann
4cf40434d7
initial support for conditional filenames ( #1394 )
3 years ago
Mike Fährmann
0abad8bc12
implement 'compile_expression()'
3 years ago
Mike Fährmann
8fd8126117
fix ISO 639-1 code for Japanese
...
"jp" -> "ja"
3 years ago
Mike Fährmann
c5ca7905ce
add 'noop()' and 'identity()' functions
3 years ago
Mike Fährmann
bff71cde80
implement 'util.unique_squence()'
4 years ago
Mike Fährmann
92071d02f4
fix crash when 'base-directory' is an empty string ( #1339 )
4 years ago
Mike Fährmann
970fc2b2b5
allow setting 'filename' & '(base-)directory' to default
...
by setting them to 'None'/'null'
4 years ago
Mike Fährmann
91308140ec
make 'generate_token()' compatible with Python 3.4
4 years ago
Mike Fährmann
780b6adb91
rename 'generate_csrf_token()' to just 'generate_token()'
...
and add a 'size' argument
4 years ago
Mike Fährmann
79501a356f
fix crash when 'path-restrict' is an object/dict
...
This basically reverts commit 5818c928
(#1234 )
4 years ago
Mike Fährmann
5d4494b15f
add "ascii" as a special 'path-restrict' value
4 years ago
Mike Fährmann
5818c928c4
refactor 'path-restrict' parsing
4 years ago
Mike Fährmann
aac00a2024
add 'd' conversion for format strings
...
to convert a timestamp to a formattable 'datetime' object.
For example '{created_at!d:%Y-%m-%d}'
transforms the timestamp in 'created_at' into a 'datetime' object
and then formats its content using '%Y-%m-%d' as template.
1262304000 -> datetime(2010, 1, 1) -> "2010-01-01"
4 years ago
Mike Fährmann
511d8d3fa3
increase SQLite connection timeouts ( #1173 )
4 years ago
Mike Fährmann
9b1bd09454
change 'extension-map' default
...
Replace all JPEG filename extensions with 'jpg'.
4 years ago
Mike Fährmann
e3480bc8de
implement 'extension-map' option ( #318 )
4 years ago
Mike Fährmann
c3f01dc4e6
implement 'util.unique()'
4 years ago
Mike Fährmann
de4a1e45c9
improve 'generate_csrf_token()'
...
no need to use hashlib.md5()
4 years ago
Mike Fährmann
ec61696316
add 't' format string conversion ( closes #1065 )
...
to Trim whitespace from the beginning and end of strings.
Example: '{field!t}' becomes 'foo' for 'field' == " \nfoo\t\r"
4 years ago
Mike Fährmann
1b1cf01d0d
add a general 'generate_csrf_token()' function
4 years ago
Mike Fährmann
d5fa716d89
fix crash when using 'skip=false' and archive ( fixes #1023 )
...
Separating the archive check from pathfmt.exists() in b5243297
had some unintended side effects.
It is also not possible to monkey-patch a dunder method like
__contains__ because of the special method lookup that gets
performed for them.
4 years ago
Mike Fährmann
65744a7a31
use alternative for all falsey values in format strings
...
… and not just None (#525 )
It would be better to consistently use None for all non-existent
fields and/or fields without a valid value, but this is a good
enough workaround for now.
4 years ago
Mike Fährmann
b5243297ff
write skipped files to archive ( closes #550 )
4 years ago
Mike Fährmann
4d8b3e4f70
defer directory creation ( fixes #722 )
...
Only call os.makedirs() before a file is getting downloaded,
and not immediately for every Directory message.
4 years ago
Mike Fährmann
1ae1df0d27
update '--write-pages' ( #737 )
...
- fix infinite recursion for responses with multiple entries in
'history'
- hide values of Set-Cookie headers
- only write the response content by default
(use '-o write-pages=all' to also include HTTP headers)
4 years ago
Mike Fährmann
1fcf938f9c
implement a general 'delete_items()' function
4 years ago
Mike Fährmann
ddc253cf9a
implement a 'path-replace' option ( #662 , #755 )
4 years ago