257e9fb435
[gelbooru] improve pagination logic for meta tags ( #5478 )
...
similar to 494acabd38
5 months ago
a2affdcef3
re-order verbose and warning options
5 months ago
a3f580254a
Add warnings flag
...
This commit adds a warnings flag
It can be combined with -q / --quiet to display warnings.
The intent is to provide a silent option that still surfaces
warning and error messages so that they are visible in logs.
5 months ago
0d72789aa3
merge #5461 : [cookies] use tempfile when saving cookies.txt files
5 months ago
63ac06643f
compute tempfile path only once
5 months ago
5227bb6b1d
[text] catch general Exceptions
5 months ago
141a93c8fd
[docs] update docs/configuration links ( #5059 , #5369 , #5423 )
5 months ago
85bbb59483
[twitter] implement 'relogin' option ( #5445 )
5 months ago
e02d2ff45d
[tapas] add 'creator' extractor ( #5306 )
5 months ago
35d4a706ae
[pixiv:novel] add 'covers' option ( #5373 )
5 months ago
b57051719f
[wikimedia] support wiki.gg wikis
5 months ago
40c1a8e471
[wikimedia] fix exception for files with empty 'metadata'
5 months ago
0e730ba980
[pp:mtime] do not overwrite '_mtime' for None values ( #5439 )
6 months ago
647a87d17c
[twitter] match '/photo/' Tweet URLs ( #5443 )
...
fixes regression introduced in 40c05535
6 months ago
3346a377b3
[pp:ugoira] log errors for general exceptions
6 months ago
40bd145637
remove 'contextlib' imports
6 months ago
9a8403917a
restore LD_LIBRARY_PATH for PyInstaller builds ( #5421 )
6 months ago
86a97d8e27
[kemonoparty] fix crash on posts with missing datetime info ( #5422 )
6 months ago
ef8f02c3d9
[oauth] use Extractor.request() for HTTP requests ( #5433 )
...
Enables using proxies and general network options.
6 months ago
095e5ded6f
[reddit] support comment embeds ( #5366 )
6 months ago
64948f2c09
[foolfuuka] improve 'board' pattern & support pages ( #5408 )
6 months ago
0c17884673
store 'match' and 'groups' in Extractor objects
6 months ago
bdbc8be5f3
[twitter] prevent crash when extracting 'birdwatch' metadata ( #5403 )
6 months ago
ef0c90414c
[wikimedia] suppress exception for entries without 'imageinfo' ( #5384 )
6 months ago
9cce461627
[kemonoparty] add 'announcements' option ( #5262 )
...
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-2015919188
6 months ago
72ac2c750d
[kemonoparty:favorite] support 'sort' and 'order' query params ( #5375 )
6 months ago
d1d017ab5d
merge #5372 : [twitter] match Tweet URLs with query parameters ( #5371 )
...
fixes regression introduced in 40c05535
6 months ago
423599ce95
[twitter] fix pattern for single tweet ( #5371 )
...
- Add optional slash
- Update tests to include some non-standard tweet URLs
6 months ago
15a4bc2584
[kemonoparty] fix KeyError for empty files ( #5368 )
6 months ago
c3bafd6a2b
release version 1.26.9
6 months ago
925123e007
[deviantart] handle CloudFront blocks in general ( #5363 )
...
This was already done for non-OAuth requests (#655 )
but CF is now blocking OAuth API requests as well.
6 months ago
31e7ca73b6
[gelbooru] add 'order-posts' option for favorites ( #5220 )
6 months ago
4b6f47e571
[pornhub:gif] extract 'viewkey' and 'timestamp' metadata ( #4463 )
...
https://github.com/mikf/gallery-dl/issues/4463#issuecomment-2014550302
6 months ago
9f73fac5ef
[steamgriddb] raise proper exception for deleted assets
6 months ago
fe9171508f
[kemonoparty] fix exception for '/revision/' URLs
...
caused by 03a9ce98
6 months ago
a650fd3177
[deviantart] improve 'index' extraction for stash files ( #5335 )
6 months ago
1d6260f456
[bunkr] remove 'description' metadata
...
album descriptions are no longer available on album pages
and the previous code erroneously returned just '0'
6 months ago
32262a048b
[idolcomplex] fix metadata extraction
...
- replace legacy 'id' vales with alphanumeric ones, since the former are
no longer available
- approximate 'vote_average', since the real value is no longer
available
- fix 'vote_count'
6 months ago
77ab015df2
[idolcomplex] support new pool URLs
6 months ago
1b34d5ac40
[subscribestar] fix 'date' metadata
6 months ago
ddb2edfd32
[formatter] fix local DST datetime offsets for ':O'
...
'O' would get the *current* local UTC offset and apply it to all
'datetime' objects it gets applied to.
This would result in a wrong offset if the current offset includes
DST and the target 'datetime' does not or vice-versa.
'O' now determines the correct local UTC offset while respecting DST for
each individual 'datetime'.
6 months ago
56f2d5a5f2
fapello.py Fullsize image
...
by remove ".md" and ".th" in image url, it will download fullsize of images
6 months ago
eb673a7204
Update fapello.py
...
get fullsize image instead resized
6 months ago
6d93295fea
[gelbooru] add 'date_favorited' metadata field
6 months ago
0d69af94d5
[gelbooru] detect returned favorites order ( #5220 )
6 months ago
e1c51c0dfb
[vipergirls] add 'domain' option ( #4166 )
6 months ago
ae116812cc
[vipergirls] add 'like' option ( #4166 )
6 months ago
da6ba60331
[bluesky] add 'instance' metadata field ( #4438 )
6 months ago
5c1f5861b6
[flickr] add 'contexts' option ( #5324 )
6 months ago
26bc2d55f4
[hiperdex] update URL patterns & fix 'manga' metadata ( #5340 )
6 months ago
8e694d85c4
[twitter] add 'birdwatch' metadata field ( #5317 )
...
should probably get a better name,
but this is what it's called internally by Twitter
6 months ago
b8e7be225c
merge #5333 : [imagefap] fix folder extractor
6 months ago
99c53f7fa8
Fix imagefap extrcator
6 months ago
03a9ce9832
[kemonoparty] add 'order-revisions' option ( #5334 )
6 months ago
1418c0ce38
[kemonoparty] add 'revision_count' metadata field ( #5334 )
6 months ago
ddee5ae7c4
[gofile] fix extraction
6 months ago
5716430c35
[deviantart:stash] recognize 'deviantart.com/stash/…' URLs
6 months ago
76683c5f5c
[deviantart:stash] fix 'index' metadata ( #5335 )
6 months ago
5158cbb4c1
[weibo] rework pagination logic ( #4168 )
...
don't automatically stop when receiving an empty status list
shouldn't improve 'tabtype=feed' results, but at least 'tabtype=album'
ones and others using cursors won't end prematurely
6 months ago
d53db6e11a
[twitter] handle "account is temporarily locked" errors ( #5300 )
...
- display proper error message
- add 'locked' option
6 months ago
108abab537
[twitter] add 'protected' metadata field ( #5327 )
...
for 'author' and 'user'
6 months ago
dde822e69a
[instagram] change 'posts are private' exception to a warning ( #5322 )
6 months ago
e40973118d
merge #5323 : [instagram] default 'likes' to '0'
6 months ago
b188cb2729
merge #5321 : [mastodon] fix handling null 'moved' account field
6 months ago
c210c7a4ed
[instagram] default posts like_count to zero
...
I don't know when/why this happens and for private account reasons can't
add an example here, but i had this error out and it feels like a safe
fix to default to `0` here.
6 months ago
225d849139
[mastodon] fix handling null 'moved' account field
6 months ago
106dfdb4c3
cleanup sleep-request retry delay code
...
more lines but easier to read I'd say
6 months ago
4565de3c8e
[reddit] provide 'gif' and 'mp4' variant fallbacks ( #5315 )
6 months ago
5842e4928d
[imgur] fail downloads when redirected to 'removed.png' ( #5308 )
6 months ago
ac4e29f70a
[lensdump] support more direct link formats ( #5293 )
6 months ago
146459056c
[reddit] provide 'fallback_url' as video fallback ( #5296 )
7 months ago
d3003f8531
merge #5270 : [imagefap] add 'folder' metadata
7 months ago
05331f9cf1
[imagefap] flake8, cleanup, tests
7 months ago
40c0553523
[twitter] add 'quotes' extractor ( #5262 )
...
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-1981571924
It's implemented as a search for 'quoted_tweet_id:…' on Twitter.
7 months ago
790c0ffb8d
[lensdump] recognize direct image links ( #5293
7 months ago
6d9e3c0eb1
[skeb] add extractor for followed users ( #5290 )
...
needs 'Authorization' header from browser session
-o headers.Authorization="Bearer ey…"
7 months ago
ace16f00f5
[weibo] fix retweets ( #2825 , #3874 , #5263 )
...
- handle 快转 retweets
- disable 'retweets' by default
- skip all retweet media when 'retweets' are disabled
- extract all retweet media when 'retweets' is set to "original"
7 months ago
0676a9d6ec
[weibo] fix 'livephoto' filename extensions ( #5287 )
7 months ago
a8027745e3
[downloader:http] add MIME type and signature for .mov files ( #5287 )
7 months ago
db507e30c7
[pixiv] fix novel text extraction ( #5285 )
...
change to '/webview/v2/novel'
since '/v1/novel/text' does not work anymore
7 months ago
296f20e630
[warosu] fix 'board_name' metadata
7 months ago
24873c2724
[warosu] fix crash for threads with deleted posts ( #5289 )
7 months ago
f296067797
[naver] unescape post 'title' and 'description'
7 months ago
a71cdab53e
merge #5126 : [naver] fix EUC-KR encoding issue in old image URLs
7 months ago
a8d3efbb99
[naver] simplify code + add test
7 months ago
f64fb8f239
[naver] EUC-KR encoding issue in old image URLs Fix
...
Around October 2010, the image server URL format and file name
encoding changed from EUC-KR to UTF-8.
Modified to detect old URL format and decode image URLs into EUC-KR
- (lint with flake8) Customize conditions
Wrap lines smaller than 79 characters
- (lint with flake8) Customize conditions (2nd try)
- One import per line
- Indent on consecutive lines
- (lint with flake8) Customize conditions (3rd try)
- E128 continuation line under-indented for visual indent
- E123 closing bracket does not match indentation of opening bracket's line
- Update naver.py
Check encoding for all image URLs
7 months ago
7b28418f69
[naver] recognize '.naver' URLs
...
https://blog.naver.com/PostView.naver ?…
7 months ago
a767832332
[deviantart:avatar] ignore default avatars ( #5276 )
7 months ago
0cbc910905
[deviantart:avatar] fix 'index' for avatars without '?' ( #5276 )
7 months ago
6482bbc525
[bluesky] handle different 'embed' structure
7 months ago
1115dccd0d
[bluesky] fix feeds and lists
...
bug introduced in 495c9ee1
7 months ago
1a9b9aa310
[artstation] support video clips ( #2566 , #3309 , #3911 )
...
- add 'videos' and 'previews' options
- fix 403 errors for video previews
7 months ago
f8b037ed40
[Imagefap] Add folder metadata
...
[Imagefap] Add "folder" metadata when downloading a folder or user profile.
No additional request is made to the server.
Use for example with the following configuration :
"parent-metadata": true
"directory":["{category}", "{uploader}", "{folder}", "{gallery_id} {title}"]
7 months ago
982880615d
[deviantart] prevent unnecessary API requests ( #4995 )
...
… when using 'comments-avatars'
This also has the added benefit of making it possible to download
comment avatars from users without a valid user profile entry,
like deleted users.
7 months ago
25d2854272
[deviantart] add 'comments-avatars' option ( #4995 )
7 months ago
218ec1a9ee
[instagram] raise proper error for missing 'reels_media' ( #5257 )
7 months ago
82c73c77b0
[redgifs] make 'date' available for directories ( #5262 )
...
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-1973975415
7 months ago
cf9e99c07b
[artstation] support collections ( #146 )
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1972101003
7 months ago
32ec695195
merge #5256 : [wikimedia] add azurlane.koumakan.jp
7 months ago
88a06df165
[wikimedia] add azurlane.koumakan.jp to presets
7 months ago
1db0a587f3
[nitter] ignore invalid Tweets ( #5253 )
...
like "Load newest"
7 months ago
76581c13f7
handle URLs without '/' after their TLD ( #5252 )
7 months ago
a00b171d4e
[bluesky] wait unitl 'RateLimit-Reset' on 429 responses
7 months ago
7d874e2497
[bluesky] improve API error messages
7 months ago
d921d860f1
automatically create directory path for logging files ( #5249 )
7 months ago
495c9ee126
[bluesky] add 'reposts' option ( #4438 , #5248 )
7 months ago
c8b591303f
[paheal] cleanup
7 months ago
8a11b72253
remove extractor/test.py ( #4504 )
7 months ago
fde9e25c9f
[tests:kemonoparty] '.party' -> '.su'
7 months ago
311a21bfb2
[bluesky] fix '/follows' not spawning child extractors ( #5246 )
7 months ago
d3dca68225
[xvideos] fix galleries with more than 500 images ( #5244 )
7 months ago
13443f40a3
[xvideos] support '/channels/' URLs ( #5244 )
7 months ago
c60ebc6519
[deviantart] improve fetching extended metadata ( #5175 )
...
use multiple metadata API calls per chunk of deviations if necessary
7 months ago
cc6b9e4c18
[zerochan] use API by default ( #3669 )
...
add 'pagination' option
7 months ago
a2b55d5dde
[skeb] retry 429 responses containing a 'request_key' cookie ( #5210 )
7 months ago
e51ee6b132
fix HttpError.status value
...
'response' with error status code evaluates to False
7 months ago
b4c46de4b8
merge #5224 : [artstation] update URL patterns to recognize usernames with dashes
7 months ago
962f55cc68
[artstation] fix handling usernames with dashes
7 months ago
fe7e2281ac
[nijie] increase default delay between requests ( #5221 )
...
1-2s is not enough
7 months ago
a34312e3ac
[instagram] make accessing 'like_count' non-fatal ( #5218 )
7 months ago
741fd00cec
[deviantart] extend 'metadata' option ( #5175 )
...
alloe fetching extended metadata in addition to the usual
'description', 'tags', etc by setting 'metadata' to a list of
'camera', 'stats', 'submission', 'collection', and 'gallery'
for example "metadata": "stats,submission"
7 months ago
fc46177578
release version 1.26.8
7 months ago
8a63801311
[vsco] add 'spaces' extractor ( #5202 )
...
for spaces listed on a user page
7 months ago
ccb413df71
[wikimedia] support 'pidgi.net' and 'bulbapedia.bulbagarden.net' ( #5205 , #5206 )
7 months ago
7033cc14e9
[vsco] add 'space' extractor ( #5202 )
7 months ago
770aec922d
[fapachi] ignore empty entries
7 months ago
ee7c054855
[bluesky] add 'search' extractor ( #4438 )
...
Both https://bsky.app/search?q=QUERY and https://bsky.app/search/QUERY
are recognized as search URLs, where QUERY gets forwarded unmodified as
'q' parameter for app.bsky.feed.searchPosts .
User searches are not supported yet.
7 months ago
91e5c4fdfe
[bluesky] add 'avatar' and 'background' extractors ( #4438 )
7 months ago
24c1317e0d
[batoto] fix crash when manga/chapter contains a '-' ( #5200 )
7 months ago
0abd9723af
[bluesky] add 'metadata' option ( #4438 )
...
allow extracting 'user' metadata and
make 'facets' extraction optional
7 months ago
7e036ea290
[bluesky] add 'depth' option ( #4438 )
...
and reduce default depth and parentHeight values
7 months ago
42335ea880
[zerochan] fix skipping every other post
7 months ago
c97b92cc35
[fanbox] add 'home' and 'supporting' extractors ( #5138 )
7 months ago
04e4ffc64c
[deviantart] combine 'png' option with 'quality' ( #4846 )
...
"quality": "png" to download PNGs instead og JPEGs
7 months ago
9cc4ec2c58
[deviantart] add 'png' option ( #4846 )
7 months ago
966c8608e6
[deviantart] move image content extraction into separate function
7 months ago
1d1ffe3317
[pornpics] update 'channel' extraction & add test
...
change 'channel' to a list, since extracting both 'channel' and
'channels' does not really work with text.extract_from()
7 months ago
32472d7d6c
Add support for multi channels
7 months ago
139ff3f6ab
[kemonoparty] add 'posts' extractor ( #5194 )
7 months ago
814ad9321e
[deviantart] skip locked/blurred posts ( #4567 , #5193 )
7 months ago
f7f8ef8684
[twitter] support communities ( #4913 )
7 months ago
cae77e85f8
[twitter] update query hashes
...
... as well as 'variables' and 'features' values
also remove unused legacy API code
7 months ago
06cb518d97
[bunkr] fix extraction ( #5088 , #5151 , #5153 )
...
- remove legacy code
- map legacy domains to bunkr.sk
- use input URL domain for newer domains
- update tests (some files got slightly modified or deleted)
7 months ago
dcc6e3f65c
merge #5134 : [bunkr] add new bunkr domains ( #5130 )
7 months ago
4641937ca3
[imagetwist] add 'gallery' extractor ( #5190 )
7 months ago
fde82ab0ce
[imagechest] add 'user' extractor ( #5143 )
7 months ago
4474cea31b
merge #5187 : [skeb] add 'num' and 'count' metadata fields
7 months ago
4cfceb23cb
[skeb] rename 'data' -> 'file' & add tests
7 months ago
44a1a66dac
merge #5186 : Fix filename formatting silently failing under certain circumstances
7 months ago
c83d0a1596
[weibo] add 'gifs' option ( #5183 )
7 months ago
f9a8e8cacf
[skeb] add 'num' and 'count' metadata fields
7 months ago
909830f8ea
fix filename formatting silently failing under certain circumstances
7 months ago
af61d2b037
[wikimedia] combine most wikimedia.org sites ( #1443 )
...
add wikidata.org and wikivoyage.org
7 months ago
c7d17f1111
[bluesky] extract 'hashtags', 'mentions', and 'uris' metadata ( #4438 )
7 months ago
55bbd49a0e
[bluesky] download images in original resolution ( #4438 )
...
at least up to 2000 px
7 months ago
6414dc6bca
[idolcomplex] fix pagination for tags containing ':' ( #5171 )
7 months ago
5c2a2321a2
[bluesky] update refresh token after using it ( #4438 )
7 months ago
9c10be54fb
[bluesky] add 'following' extractor ( #4438 )
7 months ago
86ce35d6a1
[bluesky] simplify 'pattern'
7 months ago
da292ded4e
[bluesky] add 'list' extractor ( #4438 )
7 months ago
004bf7bb38
[bluesky] add 'feed' extractor ( #4438 )
7 months ago
6aea818d4e
[bluesky] allow using DIDs as user handles ( #4438 )
7 months ago
aee5580c62
[idolcomplex] extract 'id_alnum' metadata ( #5171 )
7 months ago
cf7d6be2d4
[bluesky] initial support ( #4438 , #4708 , #4722 , #5047 )
8 months ago
6ef143ea31
[idolcomplex] support alphanumeric post IDs ( #5171 )
8 months ago
6e928300bc
[flickr] handle non-JSON errors ( #5131 )
8 months ago
90ac6d7375
[wikimedia] use '/api.php' as default API path
8 months ago
d7823b9f81
[pinterest] fix section URLs for boards with /?# in name ( #5104 )
8 months ago
de752eb7b1
[naverwebtoon] support '/webtoon/' paths for all comics ( #5123 )
8 months ago
0dacb2b24c
[downloader:http] remove 'pyopenssl' import ( #5156 )
8 months ago
d9d0601ab1
break up line to fit 80 char
8 months ago
6bcd3c9380
[bunkr] add new bunkr domains ( #5130 )
8 months ago
62d6f5f8d2
[luscious] fix IndexError for files without thumbnail ( #5122 )
8 months ago
22647c2626
[naverwebtoon] fix 'title' for comics with empty tags ( #5120 )
8 months ago
3433481dd2
[gofile] update 'website_token' extraction
8 months ago
1f7101d606
[archivedmoe] fix thebarchive webm URLs ( #5116 )
8 months ago
34a4ddc399
[sankaku] add 'id-format' option ( #5073 )
8 months ago
afd20ef42c
[kemonoparty] implement filtering duplicate revisions ( #5013 )
...
set 'revisions' to '"unique"' to have it ignore duplicate revisions
8 months ago
c28475d325
[kemonoparty] fix deleting 'name' in orginal objects ( #5103 )
...
... when computing 'revision_hash'
regression caused by 3d68eda4
8 months ago
beacfa7436
[bunkr] update domain to 'bunkr.sk' ( #5114 )
8 months ago
0502256251
release version 1.26.7
8 months ago
67c99b1366
[patreon] prevent HttpError for stream.mux.com URLs
8 months ago
f3ad91b44f
[bunkr] update domain ( #5088 )
8 months ago
c7a42880ab
[wikimedia] support fandom wikis ( #1443 , #2677 , #3378 )
...
Wikis hosted on fandom.com are just wikimedia instances
and support its API.
8 months ago
5bf156f0b1
merge #5094 : [webtoons] fix extracting comic and episode name with commas
8 months ago
df718887c2
[webtoons] fix extracting comic and episode name with commas
8 months ago
6eb62f2140
Combine lh*(-**).googleusercontent.com URL regex into one line.
...
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
8 months ago
a6fed628dd
[blogger] Fix lh*.googleusercontent.com forward slash bug, add support for lh*-**.googleusercontent.com
...
Some URLs use "lh(number)-(locale).googleusercontent.com" format, so I added support for those.
Also, "lh(number).googleusercontent.com" formats were broken because the regex was looking for a second forward slash.
Examples:
lh7.googleusercontent.com
lh7-us.googleusercontent.com
8 months ago
6f8592eaff
[hbrowse] remove from modules list
8 months ago
acc94ac187
[realbooru] fix extraction
...
revert ac97aca99c
8 months ago
9599151118
[issuu] fix extraction
8 months ago
9ca6117c67
[hbrowse] remove module
...
website gone
8 months ago
375eefb886
[chevereto] remove 'pixl.li'
...
"Pixl is closing down"
"All images will be deleted January 1st."
8 months ago
321861af7e
[erome] fix 'count' metadata
8 months ago
b41d9bf616
[paheal] fix 'source' metadata
8 months ago
b0a441f1e3
[nitter] remove 'nitter.lacontrevoie.fr'
...
"Fermeture de Nitter / Closing down Nitter"
8 months ago
a1c1e80f67
[giantessbooru] update domain
8 months ago
2007cb2f59
[tests] check extractor category values
8 months ago
fc4e737f67
[wikimedia] include 'sha1' in default filenames
8 months ago
44f2c15a04
[wikimedia] handle 'File:' paths
8 months ago
93b4120e77
[gelbooru] support 'all' and empty tag ( #5076 )
8 months ago
a416d4c3d5
[sankaku] support post URLs with alphanumeric IDs ( #5073 )
8 months ago
ea553a1d55
[wikimedia] generalize ( #1443 )
...
- support mediawiki.org
- support mariowiki.com (#3660 )
- combine code into a single extractor
(use prefix as subcategory)
- handle non-wiki instances
- unescape titles
8 months ago
89066844f4
add 'config_instance' method
...
to allow for a more streamlined access to BaseExtractor instance options
8 months ago
c3c1635ef3
[wikimedia] update
...
- rewrite using BaseExtractor
- support most Wiki* domains
- update docs/supportedsites
- add tests
8 months ago
221f54309c
[wikimedia] Improved archive identifiers
8 months ago
e33056adcd
[wikimedia] Add Wikipedia/Wikimedia extractor
8 months ago
3d68eda4ab
[kemonoparty] add 'revision_hash' metadata ( #4706 , #4727 , #5013 )
...
A SHA1 hexdigest of other relevant metadata fields like
title, content, file and attachment URLs.
This value does NOT reflect which revisions are listed on the website.
Neither does 'edited' or any other metadata field (combinations).
8 months ago
799a8206ad
merge #5061 : [webtoons] extract more metadata
...
- author_name
- comic_name
- episode_name
- username
8 months ago
8ffa0cd3c8
[webtoons] small optimization
...
don't extract the entire 'author_area' and
avoid creating a second 'text.extract_from()' object
8 months ago
59cf4b3884
merge #4444 : [2ch] add 'thread' and 'board' extractors ( #1009 , #3540 )
8 months ago
90b382304a
[deviantart] fix KeyError: 'premium_folder_data' ( #5063 )
8 months ago
4cedf378d5
[deviantart] fix AttributeError for URLs without username ( #5065 )
...
caused by 4f367145
8 months ago
68196589c4
[2ch] update
...
- simplify extractor code
- more metadata
- add tests
8 months ago
6c4abc982e
[2ch] add 'thread' and 'board' extractors
...
- [2ch] add thread extractor
- [2ch] add board extractor
- [2ch] add new entry to supported sites
8 months ago
bb446b1598
[webtoons] extract more metadata
8 months ago
355b909f46
merge #5041 : [steamgriddb] add support ( #5033 )
8 months ago
71e2c3e5a2
merge #5037 : [hatenablog] add support ( #5036 )
8 months ago
9f53daabb8
[hatenablog] implement additional suggestion
8 months ago
293f1559df
[hatenablog] implement suggestions
8 months ago
65f42442f5
[steamgriddb] implement another suggestion
8 months ago
8995fd5f01
[steamgriddb] implement suggestions
8 months ago
b1c175fdd1
allow using an empty string as argument for -D/--directory
8 months ago
2dcfb012ea
[patreon] download 'm3u8' manifests with ytdl
8 months ago
1c68b7df01
[patreon] fix KeyError ( #5048 )
8 months ago
2191e29e14
[nijie] fix image URL for single image posts ( #5049 )
8 months ago
bbf96753e2
[gelbooru] only log "Incomplete API response" for favorites ( #5045 )
8 months ago
39904c9e4e
[deviantart:avatar] add 'formats' option ( #4995 )
8 months ago
5c43098a1a
[twitter] revert to using 'media' timeline by default ( #4953 )
...
This reverts commit a94f944148
8 months ago
5f9a98cf0f
[deviantart:avatar] fix exception when 'comments' are enabled ( #4995 )
8 months ago
887ade30a5
[batoto] support more mirror domains ( #5042 )
8 months ago
0a382a5092
[batoto] improve 'manga_id' extraction ( #5042 )
8 months ago
100966b122
[steamgriddb] fix linting error
9 months ago
2ccb7d3bd3
[steamgriddb] add support
9 months ago
ec958a26bc
[fuskator] make metadata extraction non-fatal ( #5039 )
...
- prevent KeyErrors
- prevent HTTP redirect
- return file URLs as list
9 months ago
2cfe788f93
[hatenablog] fix extractor naming errors
9 months ago
be6949c55d
[hatenablog] fix linting error
9 months ago
61f3b2f820
[hatenablog] add support
9 months ago
657ed93a22
[batoto] improve v2 manga URL pattern
...
and add tests
9 months ago
50eef1b5cc
merge #5029 : [pixiv] update App API headers
9 months ago
33f228756a
[mangadex] add 'list' extractor ( #5025 )
...
supports listing manga and chapters from list feed
9 months ago
db8de13537
[vk] transform image URLs to non-blurred versions ( #5017 )
...
apply the same filter from before d85e66bc
9 months ago
6e10260fb0
release version 1.26.6
9 months ago
d0d199414f
modify useragent of pixiv
9 months ago
cbfb7bfdf1
[gelbooru] display error for invalid API responses ( #4903 )
9 months ago
c25bdbae91
[komikcast] fix 'manga' extractor ( #5027 )
9 months ago
8e1a2b5446
[komikcast] update domain to 'komikcast.lol' ( #5027 )
9 months ago
a441249ea2
merge #4979 : [batoto] add 'chapter' and 'manga' extractors ( #1434 , #2111 )
9 months ago
b11c352d66
[bato] rename to 'batoto'
...
to use the same category name as the previous bato.to site
9 months ago
3aa24c3744
[bato] simplify and update
9 months ago
11150a7d72
[nudecollect] remove module
9 months ago
c158927c38
merge #5016 : [zzup] add 'gallery' extractor ( #4517 , #4604 , #4659 , #4863 )
9 months ago
e61f016465
[szurubooru] support 'snootbooru.com' ( #5023 )
9 months ago
b4bcf40278
[weibo] fix AttributeError in 'user' extractor ( #5022 )
...
yet another bug caused by a383eca7
9 months ago
0ab0a10d2d
[jpgfish] update domain
9 months ago
0f30136109
[zzup] add 'gallery' extractor
9 months ago
a86775f617
[gelbooru] fix 'favorite' extractor ( #4903 )
...
lots of +1/-1 and </<= mistakes
9 months ago
7eaf648f2e
[fanbox] add 'metadata' option ( #4921 )
...
extracts 'plan' and extended 'user' metadata
9 months ago
00570028a3
[cookies] fix macOS Firefox profile path
...
85b33f5c16
9 months ago
4f3671458e
[deviantart] add 'avatar' and 'background' extractors ( #4995 )
9 months ago
9fa4f54c24
[twitter] raise error for invalid 'strategy' values ( #4953 )
9 months ago
516c69297d
[manganelo] fix extraction & recognize '.to' TLDs ( #5005 )
9 months ago
63f649cd92
[idolcomplex] fix extraction & update URL patterns ( #5002 )
9 months ago
b6903a4c90
[nijie] add 'count' metadata field
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1812849102
9 months ago
b93b351db9
merge #4962 : [poringa] add support ( #4675 )
9 months ago
9f21c839ad
[poringa] improvements and fixes
...
- add 'num' and 'count' metadata fields
- prevent crash for "private" posts
- prevent crash when there's no 'main-info'
- update tests
9 months ago
00d83d9588
[rule34us] add fallback for 'video-cdn1' videos ( #4985 )
9 months ago
085411f3f1
[rule34] recognize URLs with 'www' subdomain ( #4984 )
9 months ago
9f5051e4ed
merge #4981 : [pinterest] add 'count' metadata field
9 months ago
f6ce870885
Better variable names
9 months ago
3553025584
Removed f-strings
9 months ago
f36dafad06
improve 'include' handling ( #4982 )
...
- remove spaces when given as string
- warn about invalid vales
9 months ago
375f2db4c2
[pinterest] add count metadata field
9 months ago
e348da7a06
[poringa] add support
9 months ago
2c3f171d65
Fix python 3.5 linting issue
9 months ago
06ff1d3a3c
Replace text.extract with extr
9 months ago
9c1ce28f68
[bato] Added mangatoto alias
9 months ago
663b8d789a
Fix linting
9 months ago
74c225f94e
[bato] add support
9 months ago
f9544194c0
[paheal] restore 'extension' metadata ( #4976 )
9 months ago
77d46e6f0c
[lynxchan] update 'bbw-chan' domain ( #4970 )
9 months ago
766316e436
[imagechest] fix loading more than 10 images in a gallery ( #4469 )
9 months ago
6840717745
release version 1.26.5
9 months ago
108c978073
merge #4919 : [postmill] add support ( #4917 )
9 months ago
8a42ea736a
[postmill] implement suggestions
9 months ago
c184454efb
[shimmie2] small optimizations
...
- unroll/remove loop
- avoid copy
9 months ago
7cd0211cc9
[shimmie2] autodetect single or double quotes
9 months ago
2a60645095
[deviantart] set 'is_original' for intermediary URLs to 'false'
9 months ago
01bb75f6cb
merge #4945 : {shimmie2[ support 'rule34hentai.net' ( #861 , #4789 )
9 months ago
79e4606893
[rule34hentai] cleanup
...
- fix using 'self._posts_rule34hentai'
- fix 'file_url' for posts
- update docs/supportedsites
- add tests
9 months ago
ef370df41d
[shimmie2] support 'rule34hentai.net'
...
- Add files via upload
- Update shimmie2.py
- Update shimme2.py
- Delete gallery_dl/extractor/shimme2.py
- spacefix
- Update shimmie2.py
- Update shimmie2.py
- flask warnings1
- Update shimmie2.py
- Update shimmie2.py
9 months ago
627ed794a2
[danbooru] provide 'tags' as list ( #4942 )
...
keep the old 'tag_string' values around, similar to sankaku
a lot of repeat code ...
would be a lot less bad if "".split(" ") returned an empty list
9 months ago
fbebc58189
[deviantart] add 'intermediary' option ( #4955 )
9 months ago
75fa1a5553
[pinterest] remove login code
...
this has been broken since forever
and is still "protected" by an invisible recaptcha check
9 months ago
92ff99c8e5
[twitter] remove 'syndication' option ( #3889 )
9 months ago
a75f85a2c2
[twitter] remove 'date_liked' ( #3850 , #4108 , #4657 )
...
Twitter's 'sortIndex' can't be used to calculate the timestamp
of when a Tweet was liked anymore.
9 months ago
a94f944148
[twitter] default to 'tweets' timeline when 'replies' are enabled ( #4953 )
9 months ago
a30a3e44d5
[nijie] move 'username required' out of _login_impl
9 months ago
57fc6fcf83
replace '24*3600' with '86400'
...
and generalize cache maxage values
9 months ago
1f9b16a70b
replace static 'sleep-request' defaults with dynamic ones
9 months ago
b127321b5c
[exhentai] only show 'using e-hentai.org' warning for exh domains
9 months ago
e097aaf64a
[exhentai] output continuation URL when interrupted ( #4782 )
9 months ago
99aa923322
[inkbunny] improve '/submissionsviewall.php' patterns ( #4934 )
...
allow 'mode=…' to be in any position
don't require it to be somewhere in the middle
9 months ago
3f9c113d78
[mastodon] Support non-numeric status IDs ( #4936 )
9 months ago
2852404e49
[inkbunny] add 'unread' extractor ( #4934 )
9 months ago
8b87a5330d
[inkbunny] stop pagination on empty results
9 months ago
6cd5e6adad
[patreon] fix bootstrap data extraction ( #4904 )
9 months ago
aac8bb4eae
[deviantart] simplify 9951c112
9 months ago
9951c112f8
[deviantart] workaround for integer client_id values ( #4924 )
9 months ago
a37b7759bc
[myhentaigallery] recognize '/g/' URLs ( #4920 )
9 months ago
da76e13e3b
[tumblr] fix exception after waiting for rate limit ( #4916 )
...
use a loop instead of recursive function calls
9 months ago
fbe14a2745
[postmill] add support
9 months ago
d59d4ebff4
[tumblr] support infinite 'fallback-retries'
9 months ago
2d5cda2b92
[exhentai] fix TypeError for infinite 'fallback-retries' ( #4911 )
9 months ago
a24b82e67d
add 'util.repeat()'
9 months ago
92fbf09643
remove single quotes in some logging messages ( #4908 )
...
('FileNotFoundError: [Errno 2] No such file or directory: ''')
->
(FileNotFoundError: [Errno 2] No such file or directory: '')
9 months ago
7dd79eee93
save cookies to tempfile, then rename
...
avoids wiping the cookies file if the disk is full
9 months ago
1d5ee4239d
[docker] let metadata-action automatically generate 'latest' tags
9 months ago
28d60e3546
release version 1.26.4
9 months ago
9a001fa6e4
merge #4906 : [patreon] fix bootstrap data extraction ( #4904 )
9 months ago
66cbe9da41
- fix style check failure "line to long"
9 months ago
244444b194
- adapt code to current code style
9 months ago
fd06255f93
- reformat and refactor to pass tests
9 months ago
5ff7106d4f
- add code for the situation when Patreon is using window.patreon = wrapInProxy({"bootstrap":' to store metadata
...
- refactor code to make it more readable
- output page content when the HTML structure is unknown (to make debugging easier)
9 months ago
75697dfb26
implement -e/--error-file as a logging handler
...
similar to --write-unsupported
10 months ago
ac22bbe80c
[twitter] retry API requests only for Timeout errors ( #4811 )
10 months ago
c55955db03
[twitter] quick and dirty fix for /media changes ( #4898 )
10 months ago
9a8dc6b02b
[exhentai] add 'fallback-retries' option ( #4792 )
10 months ago
bf74eb5c46
merge #4886 : [urlgalleries] add 'gallery' extractor ( #919 , #1184 , #2905 )
10 months ago
c29ae9af08
[urlgalleries] simplify + resolve redirects
10 months ago
042a9da451
add 'output.errorfile' config option
10 months ago
e256434c9e
use custom HTTPBasicAuth class
...
to support LazyPrompt as password
and to generate the Authorization header only once
instead of for every request
10 months ago
bdebe4597a
fix util.dump_response to work with bytes as header values
10 months ago
6a4218aa23
handle 'json' parameter in Extractor.request() manually
...
Mainly to allow passing custom classes like util.LazyPrompt,
but also to simplify and streamline how requests handles it.
10 months ago
9dd5cb8c8a
interactively prompt for passwords on login when none is provided
10 months ago
99b76628f7
implement '-e/--error-file' command-line option ( #4732 )
...
copying per-URL options from regular, read-only input files
does currently not work
10 months ago
4eb3590103
[nijie] fix image URLs of multi-image posts ( #4876 )
10 months ago
a4e6ea667b
[twitter] retry API calls when their response contains errors ( #4811 )
10 months ago
cf5702c843
[twitter] generalize "Login Required" error ( #4734 , #4324 )
10 months ago
ecaa0feb5d
[urlgalleries] add support
10 months ago
1770c31e63
[urlgalleries] add support
10 months ago
da0da0faaa
[exhentai] store more cookies when logging in ( #4881 )
...
include 'igneous', 'hath_perks', etc
and not just 'ipb_member_id' and 'ipb_pass_hash' like before
10 months ago
43ca49c1b4
[github] add workflow to build and push docker images
...
heavily inspired by and adapted from
https://github.com/danbooru/danbooru/blob/master/.github/workflows/docker-build.yaml
10 months ago
4dde36889c
release version 1.26.3
10 months ago
c83fbe6c2d
merge #4855 : [nitter] fix video extraction ( #4853 )
10 months ago
013ca21543
[idolcomplex] update to site layout changes
10 months ago
1e9bacd169
[nitter] fix video extraction
10 months ago
9f3368c46f
[pornhub] fix 'user' metadata for gifs
10 months ago
bdb3ce7217
[foolslide] remove 'powermanga.org'
10 months ago
d9734ce008
[cyberdrop] update to site layout changes
10 months ago
Mike Fährmann