d0b160461a
terrible workaround for errors with 'http-metadata' ( #3334 )
2 years ago
20e12b5d7c
[nitter] support '/i/user/' URLs ( #3310 )
...
as well as using 'id:<userid>' as username
not all nitter instances seem to support '/i/user/' ...
2 years ago
fceaee3c4f
[lolisafe] remove zz.ht
2 years ago
4554c43d5f
[bunkr] use 'media-files' servers for more file types
2 years ago
4bc756dfe0
[2chen] fix extraction ( #3356 )
...
update 'archive_fmt'
update tests
update 'board' regex
2 years ago
54844944ab
[pixhost] add 'gallery' support ( #3353 )
2 years ago
213676c785
[fapachi] add 'post' and 'user' extractors ( #3347 )
...
* [fapachi] add 'post' and 'user' extractors
* [fapachi] add 'keyword' to test
* [fapachi] remove whitespaces
2 years ago
a18511e346
[nitter] retry downloads on 404 ( #3313 )
2 years ago
80102fa367
[downloader:http] add 'retry-codes' option ( #3313 )
2 years ago
88610c3478
[patreon] update API query parameters
2 years ago
c19b1f03b9
[patreon] fix '403 Forbidden' errors
...
send 'Content-Type' headers for API requests
2 years ago
b4253f69c9
[downloader:http] fix ZeroDivisionError ( #3328 )
...
ensure 'time_elapsed' only get used as divisor
when it is greater than zero
2 years ago
fc34f76cc5
[bunkr] fix video downloads ( #3326 )
...
by sending 'https://stream.bunkr.is/ ' as Referer header
2 years ago
86a396e086
[bcy] fix JSONDecodeError ( #3321 )
2 years ago
5b9a22af7f
[patreon] improve 'campaign_id' extraction ( #3235 )
2 years ago
1bdd0e4338
[nitter] support '/i/web/' Tweet URLs ( #3310 )
2 years ago
7e277d0f7d
[weibo] add 'count' metadata field ( #3305 )
...
or '{status[count]}', as most metadata for weibo is inside 'status'
2 years ago
4287a93202
[nitter] handle base64-encoded filenames
2 years ago
b14b33f19e
Implement `version-metadata` option ( #3201 )
2 years ago
a6305d031c
[hitomi] apply format check for every image ( #3030 ) ( #3280 )
2 years ago
a7c7953107
[reddit] use 'dash_url' for videos ( #3258 ) ( #3306 )
...
* use fallback_url for reddit_video to fix issue 3258
* changed to dash_url to include audio
* update
- use [] instead of .get
- catch TypeErrors in case one of the elements is not a dict
Co-authored-by: InterruptSpeed <steven@docherty.ca>
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
0e75358af8
[twitter] fix using user IDs for suspended accounts
2 years ago
c25905641e
[weibo] fix bug with empty 'playback_list' ( #3301 )
2 years ago
6cb12f513b
[nitter] support quoted Tweets
...
- distinguish between regular and quoted Tweets and media
- add 'quoted' option and metadata field
2 years ago
aabfa7cf34
[nitter] fix direct Tweet links
2 years ago
a41d093bb1
[nitter] add 'retweets' option ( #3278 )
2 years ago
3d6489a4c0
[nitter] update 'user' and 'author'
2 years ago
e99ce99284
[danbooru] remove stray 'print()'
2 years ago
ed49e63d95
[nitter] set 'hlsPlayback' cookie
2 years ago
e081b1fac4
[nitter] sanitize filenames ( #3294 )
2 years ago
e31d12139c
[nitter] add 'videos' option ( #3279 )
...
with the same semantics as for twitter
2 years ago
8c4e21b110
[itaku] remove 'Extreme' rating ( #3287 )
2 years ago
72c5d26e85
[hotleak] fix UnboundLocalError ( #3288 , #3293 )
2 years ago
501d9bccfe
[artstation] add 'max-posts' option ( #3270 )
2 years ago
b1ad6f2289
[artstation] add 'pro-first' option ( #3273 )
2 years ago
5a17e15b76
[pixiv] preserve 'tags' order ( #3266 )
...
for '"tags": "translated"'
As it turns out, set() does *not* preserve insertion order.
2 years ago
1392b44bfe
[inkbunny] provide additional metadata ( #3274 )
...
- 'pool_id' for pools
- 'favs_user_id' for favorites
- 'search[...]' for searches
2 years ago
42481aed59
[formatter] implement 'S' format specifier ( #3266 )
...
to Sort lists
2 years ago
8a021e4ee4
release version 1.24.0
2 years ago
6b97dcf2e0
[postprocessor:metadata] add 'private' option
2 years ago
a24dcbe802
[twitter] fix login ( #3220 )
...
Using an email as 'username' seems to no longer be possible,
as Twitter will always additionally ask for username or phone number
when providing an email address as 'username'.
2 years ago
985fd398f5
[ytdl] update 'parse_bytes' location ( #3256 )
...
https://github.com/yt-dlp/yt-dlp/commit/64c464a
2 years ago
226d778294
do not try to fetch 'http-metadata' for ytdl URLs ( #3257 )
2 years ago
133412bd62
remove previous 'http-metadata' entries from kwdict
2 years ago
53a5d95b7d
[instagram] skip private check for avatars ( #3255 )
2 years ago
08fd1ff835
[twitter] add 'avatar' and 'background' extractors ( #349 , #3023 )
2 years ago
46d811bac0
add loaded config files to debug output
2 years ago
4c6379e9d5
fix typo
2 years ago
6379157543
[instagram] use REST API by default
...
regardless of logged in status
2 years ago
f87cfa5f66
[downloader:http] add signature check for .mp4 files
2 years ago
7897f68225
[wallhaven] update 'user' extractor ( #3226 )
...
* [wallhaven] update 'user' extractor
* [wallhaven] update 'configuration.rst'
add 'extractor.wallhaven.include' entry
* [wallhaven] add 'wallhaven.include' in gallery-dl.conf
2 years ago
5a68b5cb3c
[wallhaven] add 'user' extractor ( #3213 )
2 years ago
442b03f7c3
[khinsider] fix song extraction ( #3219 )
2 years ago
eaae4d9b65
[pixiv] stop with error for invalid search/ranking parameters
...
instead of falling back to defaults
2 years ago
368f156378
[pixiv] rankings: add support for the new daily AI and daily AI R18
...
(#3214 , #3221 )
In remembrance of @thatfuckingbird
2 years ago
6c153750fa
[nitter] add extractors for Nitter instances ( #2696 )
2 years ago
374f14c28c
fix repeating paths for skipped files ( #3203 )
...
fixing the fix from e3260293
2 years ago
9f06e79868
implement '"user-agent": "browser"' ( #2636 )
2 years ago
70c7fbe89a
[instagram] add 'guide' extractor ( #3192 )
2 years ago
93ea8ca8e3
[imxto] extract additional metadata ( #3175 )
2 years ago
e3abab8629
[weibo] send 'Referer' headers ( #3188 )
2 years ago
6423f990de
[realbooru] fix 'tags' extraction ( #2530 )
2 years ago
ecad02cf3f
[realbooru] fix download URLs ( #2530 )
2 years ago
a4ff20cf16
[downloader:http] fix issues from inaccurate 'time.sleep()'
...
(#3143 )
Reverts part of c59b98c8
2 years ago
15cd114c9c
[twitter] update bookmarks pagination ( #3172 )
...
Do not stop when there aren't any tweets in a batch,
but only when the same cursor value appears twice in a row.
2 years ago
550f90ab56
delay enabling .part files when 'http-metadata' is set
...
otherwise 'build_path' gets called before all metadata is collected
2 years ago
20fbba9d7c
[exhentai] add metadata to search results ( #3181 )
...
'gallery_id' and 'gallery_token'
2 years ago
6a0c5e34f4
[exhentai] fix pagination ( #3181 )
2 years ago
05255f5be0
add 'default' argument to 'text.extr()'
2 years ago
e326029355
build path when skipping archived files
...
fixes bug from 8124c16a
2 years ago
171262c1b6
[instagram] remove login support
...
broken feature that I cannot get to work anymore
2 years ago
8124c16a50
split 'build_path' from 'set_filename' and 'set_extension'
...
Do not automatically build a new path
when setting file metadata or updating its extension.
2 years ago
39d9c362e4
include 'http-metadata' in '-K' output
2 years ago
e2401c96ee
[postprocessor:metadata] add '"mode": "jsonl"'
2 years ago
895f36e53b
[postprocessor:metadata] add 'open' and 'encoding' options
2 years ago
93e6bd6847
[uploadir] use utf-8 filenames ( #3162 )
2 years ago
870e6a48a0
implement 'http-metadata' option
...
or at least attempt to.
2 years ago
b7a83ac726
[uploadir] update ( #3162 )
...
- prevent extra HTTP request from redirects
- add 'id' metadata field
- set 'filename_fmt' and 'archive_fmt'
2 years ago
ccb80f1b8b
[uploadir] add support for 'uploadir.com' ( #3162 )
2 years ago
b0cb4a1b9c
replace 'text.extract()' with 'text.extr()' where possible
2 years ago
eb33e6cf2d
add 'text.extr()'
...
a stripped-down version of text.extract() that
- always returns a string (like 'extract_from')
- only returns a string
- does not deal with 'pos' arguments
- is ~20% faster
2 years ago
597b63d922
move git head functionality to function in util.py
2 years ago
4fd3c893fa
[booru] adjust/match '_tags' and '_notes' code
2 years ago
88954aa2e4
[gelbooru_v02] implement 'notes' extraction
...
same code as for 'moebooru' works here as well
2 years ago
942bc84962
add '--chunk-size' command-line option ( #3143 )
2 years ago
79a9fc6e45
add '--user-agent' command-line option
2 years ago
4e80d3210e
[tumblr] Fallback to `gifv` when possible ( #3095 ) ( #3159 )
2 years ago
9d3f86dbcd
[twitter] update URL for syndication API ( #3160 )
...
Twitter changed the URL format to access tweet data through their syndication API.
2 years ago
c01cad599a
[lolisafe] add support for xbunkr ( #3156 )
2 years ago
9fc142d27b
[mastodon] add "remote_instance" field ( #3119 )
...
Example Usage:
If the url is "mastodon:https://mastodon.example.org/@VoteChess@botsin.space the "remote_instance" will be "botsin.space"
...
"directory": ["mastodon", "{remote_instance|instance}", "{account[username]!l}"]
...
2 years ago
bca9f965e5
[downloader:http] add 'chunk-size' option ( #3143 )
...
and double the previous default from 16384 (2**14) to 32768 (2**15)
2 years ago
2a1cb403ee
Revert "[Deviantart] [ #1776 ] Remove the "you need session cookies to download mature scraps" warning ( #1777 )"
...
This reverts commit 1f02878351
2 years ago
0059e2bfe7
[downloader:http] add MIME type and signature for .avif files
2 years ago
f687e64513
[downloader:http] refactor file signature checks
...
use functions/lambdas instead of startswith()
2 years ago
86790da2d5
update Cloudflare IUAM detection
...
again
2 years ago
c12a97bcde
[postprocessor] add 'post-after' event ( #3117 )
2 years ago
775895f44b
[booru] refactor 'tags' and 'notes' extraction
...
- move HTML request for post pages into its own function
- move gelbooru_v02.py notes extraction to gelbooru.py
since it only works there
- clean up some code
2 years ago
0f9dfb7e62
[instagram] Fix AttributeError on user stories extraction ( #3123 )
2 years ago
f81dd5297a
[skeb] fix extraction ( #3112 )
...
'completed_at' is no longer included in API responses
2 years ago
b337e51e91
run flake8 on all .py files
2 years ago
Mike Fährmann