Mike Fährmann
d7a2c73274
[util] let a CustomNone instance be equal to itself
2 months ago
Mike Fährmann
f7383a56f8
wrap filters/conditionals in a try-except block
...
allows accessing undefined fields without exception or locals().get(…)
but hides mistakes/typos/etc by evaluating to False without feedback
performance loss compared to the previous version without try-except
is negligible (~20ns for me)
2 months ago
Mike Fährmann
7614bc458e
[util] extend CustomNone with comparison operators
4 months ago
Mike Fährmann
88f94190f4
[archive] move DownloadArchive into its own module
4 months ago
Mike Fährmann
9a8403917a
restore LD_LIBRARY_PATH for PyInstaller builds ( #5421 )
6 months ago
Mike Fährmann
8a11b72253
remove extractor/test.py ( #4504 )
7 months ago
Mike Fährmann
a24b82e67d
add 'util.repeat()'
9 months ago
Mike Fährmann
e256434c9e
use custom HTTPBasicAuth class
...
to support LazyPrompt as password
and to generate the Authorization header only once
instead of for every request
10 months ago
Mike Fährmann
bdebe4597a
fix util.dump_response to work with bytes as header values
10 months ago
Mike Fährmann
9dd5cb8c8a
interactively prompt for passwords on login when none is provided
10 months ago
Mike Fährmann
4700051562
rework and extend input file processing ( #4732 )
...
- add 2 command-line options to modify input file contents
- -I/--input-file-comment
- -x/--input-file-delete
- implement InputManager class
- move code from util.py to __init__.py
(mainly to avoid import cycles)
10 months ago
Mike Fährmann
72b18d701f
represent util.NONE as 'null' in JSON output
...
was '"None"' before
11 months ago
Mike Fährmann
899df8f237
remove another '*' for keyword-only arguments
...
076380e0
1 year ago
Mike Fährmann
26d06e0bb2
move executable check into util.py
2 years ago
Mike Fährmann
d788e6c60c
implement 'globals' option
2 years ago
Mike Fährmann
3d8777fbc1
move user agent string to util.py
2 years ago
Mike Fährmann
56039d2456
add 'hash_md5' and 'hash_sha1' functions ( #3679 )
...
... to global eval namespace
2 years ago
Mike Fährmann
d4232f3a8b
implement restarting an extractor ( #3338 )
2 years ago
Mike Fährmann
1aae72773f
put argument init on separate lines
2 years ago
Mike Fährmann
5503ac4d5e
replace json.dumps with direct calls to JSONEncoder.encode
2 years ago
Mike Fährmann
dd884b02ee
replace json.loads with direct calls to JSONDecoder.decode
2 years ago
Mike Fährmann
762a68996b
implement 'archive-pragma' option
2 years ago
Mike Fährmann
d6793b2c7d
include request body in 'write-pages=all' output
2 years ago
Mike Fährmann
d651d45239
implement specifying ranges in slice notation ( #918 , #2865 )
...
e.g.
- '1:101' or ':101' or ':101:' for files 1 to 100
- '1::2' or '::2' for every second file
- '1:101:5' or ':101:5' for files 1, 6, 11, ..., 91, 96
(the second argument specifies the first index NOT included)
2 years ago
Mike Fährmann
3616adfc75
implement '--range' with Python ranges
2 years ago
Mike Fährmann
1800bd7d14
allow '*-filter' options to be a list of expressions
2 years ago
Mike Fährmann
43c211f1a7
extend and rename util.CustomNone
2 years ago
Mike Fährmann
39d9c362e4
include 'http-metadata' in '-K' output
2 years ago
Mike Fährmann
597b63d922
move git head functionality to function in util.py
2 years ago
ClosedPort22
455e34113e
Improve compatibility of DownloadArchive ( #3078 )
...
Other programs can add additional columns to the table without affecting
gallery-dl
2 years ago
Mike Fährmann
a6e2d96dde
fix bug when processing input file comments ( #2808 )
...
and move 'parse_inputfile()' to util.py
2 years ago
Mike Fährmann
dd3a6a9fd1
make 'enumerate_reversed()' work with generators ( #2795 )
2 years ago
Mike Fährmann
a96f9d29b5
create missing directories for archive files ( #2597 )
2 years ago
Mike Fährmann
c4b9f7bab8
update functions working with cookies.txt files
...
- rename
- load_cookiestxt -> cookiestxt_load
- save_cookiestxt -< cookiestxt_store
- in cookiestxt_load, add cookies directly to a cookie jar
instead of storing them in a list first
- other unnoticeable performance increases
2 years ago
Mike Fährmann
4c8e28db82
avoid circular import in util.py
2 years ago
Mike Fährmann
61afa973b7
use extended format strings for archive keys
2 years ago
Mike Fährmann
ca3a364db7
fix build_duration_func() ( #2533 )
...
for extractors with request_interval_min > 0
2 years ago
Mike Fährmann
7fe54bab2a
attempt to fix some issues with 'contains()' ( #2446 )
...
add a third argument that gets used
when the values o search are given as a string
2 years ago
Mike Fährmann
d78a2c7163
re.escape() arguments for 'contains()' ( #2446 )
2 years ago
Mike Fährmann
413b77757b
implement 'contains()' ( #2446 )
...
and add it to globals() in compiled expressions for --filter etc
3 years ago
Mike Fährmann
29db716a63
implement 'datetime_to_timestamp()'
...
and rename 'to_timestamp()'
to the more descriptive 'datetime_to_timestamp_string()'
3 years ago
Mike Fährmann
9bd27b1b8d
[postprocessor:metadata] implement archive options ( #2421 )
...
'archive', 'archive-format', and 'archive-prefix'
3 years ago
Mike Fährmann
8295bc6d97
fix loading/storing cookies without domain
3 years ago
Mike Fährmann
47cf05c4ab
refactor proxy handling code ( #2357 )
...
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
- add 'downloader.proxy' option
- '-o extractor.proxy=–PROXY_URL -o downloader.proxy=null'
now has the same effect as youtube-dl's '--geo-verification-proxy'
3 years ago
Mike Fährmann
00fbc942e5
add 'timedelta' class to the global namespace in filter expressions
...
(https://github.com/mikf/gallery-dl/issues/2308#issuecomment-1046135924 )
3 years ago
Mike Fährmann
7958995398
[postprocessor:metadata] implement 'mtime' option ( #2307 )
3 years ago
Mike Fährmann
64cf26eaf4
allow specifying sleep-* options as string
...
either as single value or as range: "3.5", "2.1 - 5.0"
3 years ago
Mike Fährmann
010d65dcec
extend blacklist/whitelist syntax ( #2025 )
...
Each entry in such a list can now also include a subcategory
'<category>:<subcategory>'
and it is possible to use '*' or an empty string as placeholder
'*:<subcategory>', ':<subcategory>', '<category>:*'
For example
"blacklist": "imgur,*:tag,gfycat:user" or
"blacklist": ["imgur", "*:tag", "gfycat:user"]
will filter all 'imgur' extractors, all extractors with a 'tag'
subcategory (e.g. https://danbooru.donmai.us/posts?tags=bonocho ),
and all 'gfycat' user extractors.
3 years ago
Mike Fährmann
c22ff97743
remove 'unit' argument from 'util.format_value()'
3 years ago
Mike Fährmann
cad85640de
move 'util.PathFormat' into its own 'path' module
...
to prevent circular imports between 'formatter' and 'util'
3 years ago