e326029355
build path when skipping archived files
...
fixes bug from 8124c16a
2 years ago
171262c1b6
[instagram] remove login support
...
broken feature that I cannot get to work anymore
2 years ago
8124c16a50
split 'build_path' from 'set_filename' and 'set_extension'
...
Do not automatically build a new path
when setting file metadata or updating its extension.
2 years ago
39d9c362e4
include 'http-metadata' in '-K' output
2 years ago
e2401c96ee
[postprocessor:metadata] add '"mode": "jsonl"'
2 years ago
895f36e53b
[postprocessor:metadata] add 'open' and 'encoding' options
2 years ago
93e6bd6847
[uploadir] use utf-8 filenames ( #3162 )
2 years ago
870e6a48a0
implement 'http-metadata' option
...
or at least attempt to.
2 years ago
b7a83ac726
[uploadir] update ( #3162 )
...
- prevent extra HTTP request from redirects
- add 'id' metadata field
- set 'filename_fmt' and 'archive_fmt'
2 years ago
ccb80f1b8b
[uploadir] add support for 'uploadir.com' ( #3162 )
2 years ago
b0cb4a1b9c
replace 'text.extract()' with 'text.extr()' where possible
2 years ago
eb33e6cf2d
add 'text.extr()'
...
a stripped-down version of text.extract() that
- always returns a string (like 'extract_from')
- only returns a string
- does not deal with 'pos' arguments
- is ~20% faster
2 years ago
597b63d922
move git head functionality to function in util.py
2 years ago
4fd3c893fa
[booru] adjust/match '_tags' and '_notes' code
2 years ago
88954aa2e4
[gelbooru_v02] implement 'notes' extraction
...
same code as for 'moebooru' works here as well
2 years ago
942bc84962
add '--chunk-size' command-line option ( #3143 )
2 years ago
79a9fc6e45
add '--user-agent' command-line option
2 years ago
b357c2dcf8
build executables on GitHub Actions with Python 3.11
2 years ago
4e80d3210e
[tumblr] Fallback to `gifv` when possible ( #3095 ) ( #3159 )
2 years ago
9d3f86dbcd
[twitter] update URL for syndication API ( #3160 )
...
Twitter changed the URL format to access tweet data through their syndication API.
2 years ago
c01cad599a
[lolisafe] add support for xbunkr ( #3156 )
2 years ago
9fc142d27b
[mastodon] add "remote_instance" field ( #3119 )
...
Example Usage:
If the url is "mastodon:https://mastodon.example.org/@VoteChess@botsin.space the "remote_instance" will be "botsin.space"
...
"directory": ["mastodon", "{remote_instance|instance}", "{account[username]!l}"]
...
2 years ago
bca9f965e5
[downloader:http] add 'chunk-size' option ( #3143 )
...
and double the previous default from 16384 (2**14) to 32768 (2**15)
2 years ago
2a1cb403ee
Revert "[Deviantart] [ #1776 ] Remove the "you need session cookies to download mature scraps" warning ( #1777 )"
...
This reverts commit 1f02878351
2 years ago
460095adca
update downloader tests
2 years ago
0059e2bfe7
[downloader:http] add MIME type and signature for .avif files
2 years ago
f687e64513
[downloader:http] refactor file signature checks
...
use functions/lambdas instead of startswith()
2 years ago
86790da2d5
update Cloudflare IUAM detection
...
again
2 years ago
c12a97bcde
[postprocessor] add 'post-after' event ( #3117 )
2 years ago
775895f44b
[booru] refactor 'tags' and 'notes' extraction
...
- move HTML request for post pages into its own function
- move gelbooru_v02.py notes extraction to gelbooru.py
since it only works there
- clean up some code
2 years ago
48bbe1ccf6
update CHANGELOG and README for 1.23.5
2 years ago
0f9dfb7e62
[instagram] Fix AttributeError on user stories extraction ( #3123 )
2 years ago
7a9cb6d7bf
update CHANGELOG and README for 1.23.4
2 years ago
f81dd5297a
[skeb] fix extraction ( #3112 )
...
'completed_at' is no longer included in API responses
2 years ago
b337e51e91
run flake8 on all .py files
2 years ago
fb2dbb04e2
[moebooru] extract 'notes' ( #3094 )
2 years ago
4e26bf98f5
[aibooru] support 'safe' subdomain ( #3110 )
2 years ago
f037429fa4
attempt to improve '-K' output for lists
...
- use [N] instead if [] to indicate a Number needs to be placed there
- enumerate list items
2 years ago
e140b85342
reword error text for unsupported URLs
2 years ago
5c31791b3c
[mastodon] support '/web/' URLs ( #3109 )
2 years ago
9a2cfd4421
[mastodon] support cross-instance user references ( #3109 )
2 years ago
58d97188b4
[mastodon] add 'bookmark' extractor ( #3109 )
2 years ago
c8af1f585a
fix pypy version specifier
2 years ago
46b64251eb
[bcy] fix extraction ( #3103 )
...
- fix regex for non-watermarked images
- fetch data from '/item/detail' pages for all other posts,
since '/apiv3/user/selfPosts' only has incomplete data
2 years ago
77173694d5
[kemonoparty] fix 'dms' extraction ( #3106 )
2 years ago
f168ec9572
[instagram] extract 'coauthors' metadata ( #3107 )
2 years ago
7c6af27eb8
[tumblr] add 'fallback-*' options ( #2957 )
...
specifically 'fallback-delay' and 'fallback-retries'
and change default number of retries to 2 (down from 3)
2 years ago
0e5d2e128b
upgrade actions versions
2 years ago
2952b8891f
test with python 3.11
2 years ago
56396ed261
update trove classifiers
2 years ago
Mike Fährmann