f7383a56f8
wrap filters/conditionals in a try-except block
...
allows accessing undefined fields without exception or locals().get(…)
but hides mistakes/typos/etc by evaluating to False without feedback
performance loss compared to the previous version without try-except
is negligible (~20ns for me)
2 months ago
7614bc458e
[util] extend CustomNone with comparison operators
4 months ago
88f94190f4
[archive] move DownloadArchive into its own module
4 months ago
9a8403917a
restore LD_LIBRARY_PATH for PyInstaller builds ( #5421 )
6 months ago
8a11b72253
remove extractor/test.py ( #4504 )
7 months ago
a24b82e67d
add 'util.repeat()'
9 months ago
e256434c9e
use custom HTTPBasicAuth class
...
to support LazyPrompt as password
and to generate the Authorization header only once
instead of for every request
10 months ago
bdebe4597a
fix util.dump_response to work with bytes as header values
10 months ago
9dd5cb8c8a
interactively prompt for passwords on login when none is provided
10 months ago
4700051562
rework and extend input file processing ( #4732 )
...
- add 2 command-line options to modify input file contents
- -I/--input-file-comment
- -x/--input-file-delete
- implement InputManager class
- move code from util.py to __init__.py
(mainly to avoid import cycles)
10 months ago
72b18d701f
represent util.NONE as 'null' in JSON output
...
was '"None"' before
11 months ago
899df8f237
remove another '*' for keyword-only arguments
...
076380e0
1 year ago
26d06e0bb2
move executable check into util.py
2 years ago
d788e6c60c
implement 'globals' option
2 years ago
3d8777fbc1
move user agent string to util.py
2 years ago
56039d2456
add 'hash_md5' and 'hash_sha1' functions ( #3679 )
...
... to global eval namespace
2 years ago
d4232f3a8b
implement restarting an extractor ( #3338 )
2 years ago
1aae72773f
put argument init on separate lines
2 years ago
5503ac4d5e
replace json.dumps with direct calls to JSONEncoder.encode
2 years ago
dd884b02ee
replace json.loads with direct calls to JSONDecoder.decode
2 years ago
762a68996b
implement 'archive-pragma' option
2 years ago
d6793b2c7d
include request body in 'write-pages=all' output
2 years ago
d651d45239
implement specifying ranges in slice notation ( #918 , #2865 )
...
e.g.
- '1:101' or ':101' or ':101:' for files 1 to 100
- '1::2' or '::2' for every second file
- '1:101:5' or ':101:5' for files 1, 6, 11, ..., 91, 96
(the second argument specifies the first index NOT included)
2 years ago
3616adfc75
implement '--range' with Python ranges
2 years ago
1800bd7d14
allow '*-filter' options to be a list of expressions
2 years ago
43c211f1a7
extend and rename util.CustomNone
2 years ago
39d9c362e4
include 'http-metadata' in '-K' output
2 years ago
597b63d922
move git head functionality to function in util.py
2 years ago
455e34113e
Improve compatibility of DownloadArchive ( #3078 )
...
Other programs can add additional columns to the table without affecting
gallery-dl
2 years ago
a6e2d96dde
fix bug when processing input file comments ( #2808 )
...
and move 'parse_inputfile()' to util.py
2 years ago
dd3a6a9fd1
make 'enumerate_reversed()' work with generators ( #2795 )
2 years ago
a96f9d29b5
create missing directories for archive files ( #2597 )
2 years ago
c4b9f7bab8
update functions working with cookies.txt files
...
- rename
- load_cookiestxt -> cookiestxt_load
- save_cookiestxt -< cookiestxt_store
- in cookiestxt_load, add cookies directly to a cookie jar
instead of storing them in a list first
- other unnoticeable performance increases
2 years ago
4c8e28db82
avoid circular import in util.py
2 years ago
61afa973b7
use extended format strings for archive keys
2 years ago
ca3a364db7
fix build_duration_func() ( #2533 )
...
for extractors with request_interval_min > 0
2 years ago
7fe54bab2a
attempt to fix some issues with 'contains()' ( #2446 )
...
add a third argument that gets used
when the values o search are given as a string
2 years ago
d78a2c7163
re.escape() arguments for 'contains()' ( #2446 )
2 years ago
413b77757b
implement 'contains()' ( #2446 )
...
and add it to globals() in compiled expressions for --filter etc
3 years ago
29db716a63
implement 'datetime_to_timestamp()'
...
and rename 'to_timestamp()'
to the more descriptive 'datetime_to_timestamp_string()'
3 years ago
9bd27b1b8d
[postprocessor:metadata] implement archive options ( #2421 )
...
'archive', 'archive-format', and 'archive-prefix'
3 years ago
8295bc6d97
fix loading/storing cookies without domain
3 years ago
47cf05c4ab
refactor proxy handling code ( #2357 )
...
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
- add 'downloader.proxy' option
- '-o extractor.proxy=–PROXY_URL -o downloader.proxy=null'
now has the same effect as youtube-dl's '--geo-verification-proxy'
3 years ago
00fbc942e5
add 'timedelta' class to the global namespace in filter expressions
...
(https://github.com/mikf/gallery-dl/issues/2308#issuecomment-1046135924 )
3 years ago
7958995398
[postprocessor:metadata] implement 'mtime' option ( #2307 )
3 years ago
64cf26eaf4
allow specifying sleep-* options as string
...
either as single value or as range: "3.5", "2.1 - 5.0"
3 years ago
010d65dcec
extend blacklist/whitelist syntax ( #2025 )
...
Each entry in such a list can now also include a subcategory
'<category>:<subcategory>'
and it is possible to use '*' or an empty string as placeholder
'*:<subcategory>', ':<subcategory>', '<category>:*'
For example
"blacklist": "imgur,*:tag,gfycat:user" or
"blacklist": ["imgur", "*:tag", "gfycat:user"]
will filter all 'imgur' extractors, all extractors with a 'tag'
subcategory (e.g. https://danbooru.donmai.us/posts?tags=bonocho ),
and all 'gfycat' user extractors.
3 years ago
c22ff97743
remove 'unit' argument from 'util.format_value()'
3 years ago
cad85640de
move 'util.PathFormat' into its own 'path' module
...
to prevent circular imports between 'formatter' and 'util'
3 years ago
74145467dd
move 'util.Formatter' into its own 'formatter' module
3 years ago
Mike Fährmann