Mike Fährmann
c97b92cc35
[fanbox] add 'home' and 'supporting' extractors ( #5138 )
7 months ago
Mike Fährmann
04e4ffc64c
[deviantart] combine 'png' option with 'quality' ( #4846 )
...
"quality": "png" to download PNGs instead og JPEGs
7 months ago
Mike Fährmann
9cc4ec2c58
[deviantart] add 'png' option ( #4846 )
7 months ago
Mike Fährmann
966c8608e6
[deviantart] move image content extraction into separate function
7 months ago
Mike Fährmann
1d1ffe3317
[pornpics] update 'channel' extraction & add test
...
change 'channel' to a list, since extracting both 'channel' and
'channels' does not really work with text.extract_from()
7 months ago
cc1234
32472d7d6c
Add support for multi channels
7 months ago
Mike Fährmann
139ff3f6ab
[kemonoparty] add 'posts' extractor ( #5194 )
7 months ago
Mike Fährmann
814ad9321e
[deviantart] skip locked/blurred posts ( #4567 , #5193 )
7 months ago
Mike Fährmann
f7f8ef8684
[twitter] support communities ( #4913 )
7 months ago
Mike Fährmann
cae77e85f8
[twitter] update query hashes
...
... as well as 'variables' and 'features' values
also remove unused legacy API code
7 months ago
Mike Fährmann
06cb518d97
[bunkr] fix extraction ( #5088 , #5151 , #5153 )
...
- remove legacy code
- map legacy domains to bunkr.sk
- use input URL domain for newer domains
- update tests (some files got slightly modified or deleted)
7 months ago
Mike Fährmann
dcc6e3f65c
merge #5134 : [bunkr] add new bunkr domains ( #5130 )
7 months ago
Mike Fährmann
4641937ca3
[imagetwist] add 'gallery' extractor ( #5190 )
7 months ago
Mike Fährmann
fde82ab0ce
[imagechest] add 'user' extractor ( #5143 )
7 months ago
Mike Fährmann
4474cea31b
merge #5187 : [skeb] add 'num' and 'count' metadata fields
7 months ago
Mike Fährmann
4cfceb23cb
[skeb] rename 'data' -> 'file' & add tests
7 months ago
Mike Fährmann
44a1a66dac
merge #5186 : Fix filename formatting silently failing under certain circumstances
7 months ago
Mike Fährmann
c83d0a1596
[weibo] add 'gifs' option ( #5183 )
7 months ago
blankie
f9a8e8cacf
[skeb] add 'num' and 'count' metadata fields
7 months ago
blankie
909830f8ea
fix filename formatting silently failing under certain circumstances
7 months ago
Mike Fährmann
af61d2b037
[wikimedia] combine most wikimedia.org sites ( #1443 )
...
add wikidata.org and wikivoyage.org
7 months ago
Mike Fährmann
c7d17f1111
[bluesky] extract 'hashtags', 'mentions', and 'uris' metadata ( #4438 )
7 months ago
Mike Fährmann
55bbd49a0e
[bluesky] download images in original resolution ( #4438 )
...
at least up to 2000 px
7 months ago
Mike Fährmann
6414dc6bca
[idolcomplex] fix pagination for tags containing ':' ( #5171 )
7 months ago
Mike Fährmann
5c2a2321a2
[bluesky] update refresh token after using it ( #4438 )
7 months ago
Mike Fährmann
9c10be54fb
[bluesky] add 'following' extractor ( #4438 )
7 months ago
Mike Fährmann
86ce35d6a1
[bluesky] simplify 'pattern'
7 months ago
Mike Fährmann
da292ded4e
[bluesky] add 'list' extractor ( #4438 )
7 months ago
Mike Fährmann
004bf7bb38
[bluesky] add 'feed' extractor ( #4438 )
7 months ago
Mike Fährmann
6aea818d4e
[bluesky] allow using DIDs as user handles ( #4438 )
7 months ago
Mike Fährmann
aee5580c62
[idolcomplex] extract 'id_alnum' metadata ( #5171 )
7 months ago
Mike Fährmann
cf7d6be2d4
[bluesky] initial support ( #4438 , #4708 , #4722 , #5047 )
8 months ago
Mike Fährmann
6ef143ea31
[idolcomplex] support alphanumeric post IDs ( #5171 )
8 months ago
Mike Fährmann
6e928300bc
[flickr] handle non-JSON errors ( #5131 )
8 months ago
Mike Fährmann
90ac6d7375
[wikimedia] use '/api.php' as default API path
8 months ago
Mike Fährmann
d7823b9f81
[pinterest] fix section URLs for boards with /?# in name ( #5104 )
8 months ago
Mike Fährmann
de752eb7b1
[naverwebtoon] support '/webtoon/' paths for all comics ( #5123 )
8 months ago
Mike Fährmann
0dacb2b24c
[downloader:http] remove 'pyopenssl' import ( #5156 )
8 months ago
Jeff Mercado
d9d0601ab1
break up line to fit 80 char
8 months ago
Jeff Mercado
6bcd3c9380
[bunkr] add new bunkr domains ( #5130 )
8 months ago
Mike Fährmann
62d6f5f8d2
[luscious] fix IndexError for files without thumbnail ( #5122 )
8 months ago
Mike Fährmann
22647c2626
[naverwebtoon] fix 'title' for comics with empty tags ( #5120 )
8 months ago
Mike Fährmann
3433481dd2
[gofile] update 'website_token' extraction
8 months ago
Mike Fährmann
1f7101d606
[archivedmoe] fix thebarchive webm URLs ( #5116 )
8 months ago
Mike Fährmann
34a4ddc399
[sankaku] add 'id-format' option ( #5073 )
8 months ago
Mike Fährmann
afd20ef42c
[kemonoparty] implement filtering duplicate revisions ( #5013 )
...
set 'revisions' to '"unique"' to have it ignore duplicate revisions
8 months ago
Mike Fährmann
c28475d325
[kemonoparty] fix deleting 'name' in orginal objects ( #5103 )
...
... when computing 'revision_hash'
regression caused by 3d68eda4
dict.copy() only creates a shallow copy
I know that and still managed to get I wrong ...
8 months ago
Mike Fährmann
beacfa7436
[bunkr] update domain to 'bunkr.sk' ( #5114 )
8 months ago
Mike Fährmann
0502256251
release version 1.26.7
8 months ago
Mike Fährmann
67c99b1366
[patreon] prevent HttpError for stream.mux.com URLs
8 months ago
Mike Fährmann
f3ad91b44f
[bunkr] update domain ( #5088 )
8 months ago
Mike Fährmann
c7a42880ab
[wikimedia] support fandom wikis ( #1443 , #2677 , #3378 )
...
Wikis hosted on fandom.com are just wikimedia instances
and support its API.
8 months ago
Mike Fährmann
5bf156f0b1
merge #5094 : [webtoons] fix extracting comic and episode name with commas
8 months ago
blankie
df718887c2
[webtoons] fix extracting comic and episode name with commas
8 months ago
Wiiplay123
6eb62f2140
Combine lh*(-**).googleusercontent.com URL regex into one line.
...
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
8 months ago
Wiiplay123
a6fed628dd
[blogger] Fix lh*.googleusercontent.com forward slash bug, add support for lh*-**.googleusercontent.com
...
Some URLs use "lh(number)-(locale).googleusercontent.com" format, so I added support for those.
Also, "lh(number).googleusercontent.com" formats were broken because the regex was looking for a second forward slash.
Examples:
lh7.googleusercontent.com
lh7-us.googleusercontent.com
8 months ago
Mike Fährmann
6f8592eaff
[hbrowse] remove from modules list
8 months ago
Mike Fährmann
acc94ac187
[realbooru] fix extraction
...
revert ac97aca99c
8 months ago
Mike Fährmann
9599151118
[issuu] fix extraction
8 months ago
Mike Fährmann
9ca6117c67
[hbrowse] remove module
...
website gone
8 months ago
Mike Fährmann
375eefb886
[chevereto] remove 'pixl.li'
...
"Pixl is closing down"
"All images will be deleted January 1st."
8 months ago
Mike Fährmann
321861af7e
[erome] fix 'count' metadata
8 months ago
Mike Fährmann
b41d9bf616
[paheal] fix 'source' metadata
8 months ago
Mike Fährmann
b0a441f1e3
[nitter] remove 'nitter.lacontrevoie.fr'
...
"Fermeture de Nitter / Closing down Nitter"
8 months ago
Mike Fährmann
a1c1e80f67
[giantessbooru] update domain
8 months ago
Mike Fährmann
2007cb2f59
[tests] check extractor category values
8 months ago
Mike Fährmann
fc4e737f67
[wikimedia] include 'sha1' in default filenames
8 months ago
Mike Fährmann
44f2c15a04
[wikimedia] handle 'File:' paths
8 months ago
Mike Fährmann
93b4120e77
[gelbooru] support 'all' and empty tag ( #5076 )
8 months ago
Mike Fährmann
a416d4c3d5
[sankaku] support post URLs with alphanumeric IDs ( #5073 )
8 months ago
Mike Fährmann
ea553a1d55
[wikimedia] generalize ( #1443 )
...
- support mediawiki.org
- support mariowiki.com (#3660 )
- combine code into a single extractor
(use prefix as subcategory)
- handle non-wiki instances
- unescape titles
8 months ago
Mike Fährmann
89066844f4
add 'config_instance' method
...
to allow for a more streamlined access to BaseExtractor instance options
8 months ago
Mike Fährmann
c3c1635ef3
[wikimedia] update
...
- rewrite using BaseExtractor
- support most Wiki* domains
- update docs/supportedsites
- add tests
8 months ago
Ailothaen
221f54309c
[wikimedia] Improved archive identifiers
8 months ago
Ailothaen
e33056adcd
[wikimedia] Add Wikipedia/Wikimedia extractor
8 months ago
Mike Fährmann
3d68eda4ab
[kemonoparty] add 'revision_hash' metadata ( #4706 , #4727 , #5013 )
...
A SHA1 hexdigest of other relevant metadata fields like
title, content, file and attachment URLs.
This value does NOT reflect which revisions are listed on the website.
Neither does 'edited' or any other metadata field (combinations).
8 months ago
Mike Fährmann
799a8206ad
merge #5061 : [webtoons] extract more metadata
...
- author_name
- comic_name
- episode_name
- username
8 months ago
Mike Fährmann
8ffa0cd3c8
[webtoons] small optimization
...
don't extract the entire 'author_area' and
avoid creating a second 'text.extract_from()' object
8 months ago
Mike Fährmann
59cf4b3884
merge #4444 : [2ch] add 'thread' and 'board' extractors ( #1009 , #3540 )
8 months ago
Mike Fährmann
90b382304a
[deviantart] fix KeyError: 'premium_folder_data' ( #5063 )
8 months ago
Mike Fährmann
4cedf378d5
[deviantart] fix AttributeError for URLs without username ( #5065 )
...
caused by 4f367145
8 months ago
Mike Fährmann
68196589c4
[2ch] update
...
- simplify extractor code
- more metadata
- add tests
8 months ago
hunter-gatherer8
6c4abc982e
[2ch] add 'thread' and 'board' extractors
...
- [2ch] add thread extractor
- [2ch] add board extractor
- [2ch] add new entry to supported sites
8 months ago
blankie
bb446b1598
[webtoons] extract more metadata
8 months ago
Mike Fährmann
355b909f46
merge #5041 : [steamgriddb] add support ( #5033 )
8 months ago
Mike Fährmann
71e2c3e5a2
merge #5037 : [hatenablog] add support ( #5036 )
8 months ago
blankie
9f53daabb8
[hatenablog] implement additional suggestion
8 months ago
blankie
293f1559df
[hatenablog] implement suggestions
8 months ago
blankie
65f42442f5
[steamgriddb] implement another suggestion
8 months ago
blankie
8995fd5f01
[steamgriddb] implement suggestions
8 months ago
Mike Fährmann
b1c175fdd1
allow using an empty string as argument for -D/--directory
8 months ago
Mike Fährmann
2dcfb012ea
[patreon] download 'm3u8' manifests with ytdl
8 months ago
Mike Fährmann
1c68b7df01
[patreon] fix KeyError ( #5048 )
8 months ago
Mike Fährmann
2191e29e14
[nijie] fix image URL for single image posts ( #5049 )
8 months ago
Mike Fährmann
bbf96753e2
[gelbooru] only log "Incomplete API response" for favorites ( #5045 )
8 months ago
Mike Fährmann
39904c9e4e
[deviantart:avatar] add 'formats' option ( #4995 )
8 months ago
Mike Fährmann
5c43098a1a
[twitter] revert to using 'media' timeline by default ( #4953 )
...
This reverts commit a94f944148
.
8 months ago
Mike Fährmann
5f9a98cf0f
[deviantart:avatar] fix exception when 'comments' are enabled ( #4995 )
8 months ago
Mike Fährmann
887ade30a5
[batoto] support more mirror domains ( #5042 )
8 months ago
Mike Fährmann
0a382a5092
[batoto] improve 'manga_id' extraction ( #5042 )
8 months ago
blankie
100966b122
[steamgriddb] fix linting error
8 months ago
blankie
2ccb7d3bd3
[steamgriddb] add support
8 months ago
Mike Fährmann
ec958a26bc
[fuskator] make metadata extraction non-fatal ( #5039 )
...
- prevent KeyErrors
- prevent HTTP redirect
- return file URLs as list
9 months ago
blankie
2cfe788f93
[hatenablog] fix extractor naming errors
9 months ago
blankie
be6949c55d
[hatenablog] fix linting error
9 months ago
blankie
61f3b2f820
[hatenablog] add support
9 months ago
Mike Fährmann
657ed93a22
[batoto] improve v2 manga URL pattern
...
and add tests
9 months ago
Mike Fährmann
50eef1b5cc
merge #5029 : [pixiv] update App API headers
9 months ago
Mike Fährmann
33f228756a
[mangadex] add 'list' extractor ( #5025 )
...
supports listing manga and chapters from list feed
9 months ago
Mike Fährmann
db8de13537
[vk] transform image URLs to non-blurred versions ( #5017 )
...
apply the same filter from before d85e66bc
9 months ago
Mike Fährmann
6e10260fb0
release version 1.26.6
9 months ago
Se AKi
d0d199414f
modify useragent of pixiv
9 months ago
Mike Fährmann
cbfb7bfdf1
[gelbooru] display error for invalid API responses ( #4903 )
9 months ago
Mike Fährmann
c25bdbae91
[komikcast] fix 'manga' extractor ( #5027 )
9 months ago
Mike Fährmann
8e1a2b5446
[komikcast] update domain to 'komikcast.lol' ( #5027 )
9 months ago
Mike Fährmann
a441249ea2
merge #4979 : [batoto] add 'chapter' and 'manga' extractors ( #1434 , #2111 )
9 months ago
Mike Fährmann
b11c352d66
[bato] rename to 'batoto'
...
to use the same category name as the previous bato.to site
9 months ago
Mike Fährmann
3aa24c3744
[bato] simplify and update
9 months ago
Mike Fährmann
11150a7d72
[nudecollect] remove module
9 months ago
Mike Fährmann
c158927c38
merge #5016 : [zzup] add 'gallery' extractor ( #4517 , #4604 , #4659 , #4863 )
9 months ago
Mike Fährmann
e61f016465
[szurubooru] support 'snootbooru.com' ( #5023 )
9 months ago
Mike Fährmann
b4bcf40278
[weibo] fix AttributeError in 'user' extractor ( #5022 )
...
yet another bug caused by a383eca7
9 months ago
Mike Fährmann
0ab0a10d2d
[jpgfish] update domain
9 months ago
enduser420
0f30136109
[zzup] add 'gallery' extractor
9 months ago
Mike Fährmann
a86775f617
[gelbooru] fix 'favorite' extractor ( #4903 )
...
lots of +1/-1 and </<= mistakes
9 months ago
Mike Fährmann
7eaf648f2e
[fanbox] add 'metadata' option ( #4921 )
...
extracts 'plan' and extended 'user' metadata
9 months ago
Mike Fährmann
00570028a3
[cookies] fix macOS Firefox profile path
...
85b33f5c16
9 months ago
Mike Fährmann
4f3671458e
[deviantart] add 'avatar' and 'background' extractors ( #4995 )
9 months ago
Mike Fährmann
9fa4f54c24
[twitter] raise error for invalid 'strategy' values ( #4953 )
9 months ago
Mike Fährmann
516c69297d
[manganelo] fix extraction & recognize '.to' TLDs ( #5005 )
9 months ago
Mike Fährmann
63f649cd92
[idolcomplex] fix extraction & update URL patterns ( #5002 )
9 months ago
Mike Fährmann
b6903a4c90
[nijie] add 'count' metadata field
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1812849102
9 months ago
Mike Fährmann
b93b351db9
merge #4962 : [poringa] add support ( #4675 )
9 months ago
Mike Fährmann
9f21c839ad
[poringa] improvements and fixes
...
- add 'num' and 'count' metadata fields
- prevent crash for "private" posts
- prevent crash when there's no 'main-info'
- update tests
9 months ago
Mike Fährmann
00d83d9588
[rule34us] add fallback for 'video-cdn1' videos ( #4985 )
9 months ago
Mike Fährmann
085411f3f1
[rule34] recognize URLs with 'www' subdomain ( #4984 )
9 months ago
Mike Fährmann
9f5051e4ed
merge #4981 : [pinterest] add 'count' metadata field
9 months ago
bug-assassin
f6ce870885
Better variable names
9 months ago
bug-assassin
3553025584
Removed f-strings
9 months ago
Mike Fährmann
f36dafad06
improve 'include' handling ( #4982 )
...
- remove spaces when given as string
- warn about invalid vales
9 months ago
blankie
375f2db4c2
[pinterest] add count metadata field
9 months ago
Antonio
e348da7a06
[poringa] add support
9 months ago
bug-assassin
2c3f171d65
Fix python 3.5 linting issue
9 months ago
bug-assassin
06ff1d3a3c
Replace text.extract with extr
9 months ago
bug-assassin
9c1ce28f68
[bato] Added mangatoto alias
9 months ago
bug-assassin
663b8d789a
Fix linting
9 months ago
bug-assassin
74c225f94e
[bato] add support
9 months ago
Mike Fährmann
f9544194c0
[paheal] restore 'extension' metadata ( #4976 )
9 months ago
Mike Fährmann
77d46e6f0c
[lynxchan] update 'bbw-chan' domain ( #4970 )
9 months ago
Mike Fährmann
766316e436
[imagechest] fix loading more than 10 images in a gallery ( #4469 )
9 months ago
Mike Fährmann
6840717745
release version 1.26.5
9 months ago
Mike Fährmann
108c978073
merge #4919 : [postmill] add support ( #4917 )
9 months ago
blankie
8a42ea736a
[postmill] implement suggestions
9 months ago
Mike Fährmann
c184454efb
[shimmie2] small optimizations
...
- unroll/remove loop
- avoid copy
9 months ago
Mike Fährmann
7cd0211cc9
[shimmie2] autodetect single or double quotes
9 months ago
Mike Fährmann
2a60645095
[deviantart] set 'is_original' for intermediary URLs to 'false'
9 months ago
Mike Fährmann
01bb75f6cb
merge #4945 : {shimmie2[ support 'rule34hentai.net' ( #861 , #4789 )
9 months ago
Mike Fährmann
79e4606893
[rule34hentai] cleanup
...
- fix using 'self._posts_rule34hentai'
- fix 'file_url' for posts
- update docs/supportedsites
- add tests
9 months ago
bun-dev
ef370df41d
[shimmie2] support 'rule34hentai.net'
...
- Add files via upload
- Update shimmie2.py
- Update shimme2.py
- Delete gallery_dl/extractor/shimme2.py
- spacefix
- Update shimmie2.py
- Update shimmie2.py
- flask warnings1
- Update shimmie2.py
- Update shimmie2.py
9 months ago
Mike Fährmann
627ed794a2
[danbooru] provide 'tags' as list ( #4942 )
...
keep the old 'tag_string' values around, similar to sankaku
a lot of repeat code ...
would be a lot less bad if "".split(" ") returned an empty list
9 months ago
Mike Fährmann
fbebc58189
[deviantart] add 'intermediary' option ( #4955 )
9 months ago
Mike Fährmann
75fa1a5553
[pinterest] remove login code
...
this has been broken since forever
and is still "protected" by an invisible recaptcha check
9 months ago
Mike Fährmann
92ff99c8e5
[twitter] remove 'syndication' option ( #3889 )
9 months ago
Mike Fährmann
a75f85a2c2
[twitter] remove 'date_liked' ( #3850 , #4108 , #4657 )
...
Twitter's 'sortIndex' can't be used to calculate the timestamp
of when a Tweet was liked anymore.
9 months ago
Mike Fährmann
a94f944148
[twitter] default to 'tweets' timeline when 'replies' are enabled ( #4953 )
9 months ago
Mike Fährmann
a30a3e44d5
[nijie] move 'username required' out of _login_impl
9 months ago
Mike Fährmann
57fc6fcf83
replace '24*3600' with '86400'
...
and generalize cache maxage values
9 months ago
Mike Fährmann
1f9b16a70b
replace static 'sleep-request' defaults with dynamic ones
9 months ago
Mike Fährmann
b127321b5c
[exhentai] only show 'using e-hentai.org' warning for exh domains
9 months ago
Mike Fährmann
e097aaf64a
[exhentai] output continuation URL when interrupted ( #4782 )
9 months ago
Mike Fährmann
99aa923322
[inkbunny] improve '/submissionsviewall.php' patterns ( #4934 )
...
allow 'mode=…' to be in any position
don't require it to be somewhere in the middle
9 months ago
Mike Fährmann
3f9c113d78
[mastodon] Support non-numeric status IDs ( #4936 )
9 months ago
Mike Fährmann
2852404e49
[inkbunny] add 'unread' extractor ( #4934 )
9 months ago
Mike Fährmann
8b87a5330d
[inkbunny] stop pagination on empty results
9 months ago
Mike Fährmann
6cd5e6adad
[patreon] fix bootstrap data extraction ( #4904 )
9 months ago
Mike Fährmann
aac8bb4eae
[deviantart] simplify 9951c112
9 months ago
Mike Fährmann
9951c112f8
[deviantart] workaround for integer client_id values ( #4924 )
9 months ago
Mike Fährmann
a37b7759bc
[myhentaigallery] recognize '/g/' URLs ( #4920 )
9 months ago
Mike Fährmann
da76e13e3b
[tumblr] fix exception after waiting for rate limit ( #4916 )
...
use a loop instead of recursive function calls
9 months ago
blankie
fbe14a2745
[postmill] add support
9 months ago
Mike Fährmann
d59d4ebff4
[tumblr] support infinite 'fallback-retries'
9 months ago
Mike Fährmann
2d5cda2b92
[exhentai] fix TypeError for infinite 'fallback-retries' ( #4911 )
9 months ago
Mike Fährmann
a24b82e67d
add 'util.repeat()'
9 months ago
Mike Fährmann
92fbf09643
remove single quotes in some logging messages ( #4908 )
...
('FileNotFoundError: [Errno 2] No such file or directory: ''')
->
(FileNotFoundError: [Errno 2] No such file or directory: '')
9 months ago
Luc Ritchie
7dd79eee93
save cookies to tempfile, then rename
...
avoids wiping the cookies file if the disk is full
9 months ago
Mike Fährmann
1d5ee4239d
[docker] let metadata-action automatically generate 'latest' tags
9 months ago
Mike Fährmann
28d60e3546
release version 1.26.4
9 months ago
Mike Fährmann
9a001fa6e4
merge #4906 : [patreon] fix bootstrap data extraction ( #4904 )
9 months ago
Tobi823
66cbe9da41
- fix style check failure "line to long"
9 months ago
Tobi823
244444b194
- adapt code to current code style
9 months ago
Tobi823
fd06255f93
- reformat and refactor to pass tests
9 months ago
Tobi823
5ff7106d4f
- add code for the situation when Patreon is using window.patreon = wrapInProxy({"bootstrap":' to store metadata
...
- refactor code to make it more readable
- output page content when the HTML structure is unknown (to make debugging easier)
9 months ago
Mike Fährmann
75697dfb26
implement -e/--error-file as a logging handler
...
similar to --write-unsupported
10 months ago
Mike Fährmann
ac22bbe80c
[twitter] retry API requests only for Timeout errors ( #4811 )
10 months ago
Mike Fährmann
c55955db03
[twitter] quick and dirty fix for /media changes ( #4898 )
10 months ago
Mike Fährmann
9a8dc6b02b
[exhentai] add 'fallback-retries' option ( #4792 )
10 months ago
Mike Fährmann
bf74eb5c46
merge #4886 : [urlgalleries] add 'gallery' extractor ( #919 , #1184 , #2905 )
10 months ago
Mike Fährmann
c29ae9af08
[urlgalleries] simplify + resolve redirects
10 months ago
Mike Fährmann
042a9da451
add 'output.errorfile' config option
10 months ago
Mike Fährmann
e256434c9e
use custom HTTPBasicAuth class
...
to support LazyPrompt as password
and to generate the Authorization header only once
instead of for every request
10 months ago
Mike Fährmann
bdebe4597a
fix util.dump_response to work with bytes as header values
10 months ago
Mike Fährmann
6a4218aa23
handle 'json' parameter in Extractor.request() manually
...
Mainly to allow passing custom classes like util.LazyPrompt,
but also to simplify and streamline how requests handles it.
10 months ago
Mike Fährmann
9dd5cb8c8a
interactively prompt for passwords on login when none is provided
10 months ago
Mike Fährmann
99b76628f7
implement '-e/--error-file' command-line option ( #4732 )
...
copying per-URL options from regular, read-only input files
does currently not work
10 months ago
Mike Fährmann
4eb3590103
[nijie] fix image URLs of multi-image posts ( #4876 )
10 months ago
Mike Fährmann
a4e6ea667b
[twitter] retry API calls when their response contains errors ( #4811 )
10 months ago
Mike Fährmann
cf5702c843
[twitter] generalize "Login Required" error ( #4734 , #4324 )
10 months ago
jsouthgb
ecaa0feb5d
[urlgalleries] add support
10 months ago
jsouthgb
1770c31e63
[urlgalleries] add support
10 months ago
Mike Fährmann
da0da0faaa
[exhentai] store more cookies when logging in ( #4881 )
...
include 'igneous', 'hath_perks', etc
and not just 'ipb_member_id' and 'ipb_pass_hash' like before
10 months ago
Mike Fährmann
43ca49c1b4
[github] add workflow to build and push docker images
...
heavily inspired by and adapted from
https://github.com/danbooru/danbooru/blob/master/.github/workflows/docker-build.yaml
10 months ago
Mike Fährmann
4dde36889c
release version 1.26.3
10 months ago
Mike Fährmann
c83fbe6c2d
merge #4855 : [nitter] fix video extraction ( #4853 )
10 months ago
Mike Fährmann
013ca21543
[idolcomplex] update to site layout changes
10 months ago
enduser420
1e9bacd169
[nitter] fix video extraction
10 months ago
Mike Fährmann
9f3368c46f
[pornhub] fix 'user' metadata for gifs
10 months ago
Mike Fährmann
bdb3ce7217
[foolslide] remove 'powermanga.org'
10 months ago
Mike Fährmann
d9734ce008
[cyberdrop] update to site layout changes
10 months ago
Mike Fährmann
8ac68ffba2
[hentaicosplays] force 'https://' for download URLs
10 months ago
Mike Fährmann
fc1101779c
[hiperdex] fix 'manga' metadata
10 months ago
Mike Fährmann
d119507037
[imagefap] fix single image resolution
...
Downloading from a single image page like
https://www.imagefap.com/photo/123456789/
returned only the thumbnail URL.
10 months ago
Mike Fährmann
311ec1d9ef
[mangaread] fix extraction
10 months ago
Mike Fährmann
7608201a44
[tumblr] fix 'day' extractor
...
another bug caused by a383eca7
10 months ago
Mike Fährmann
c8c744a7c0
[webtoons] fix pagination when receiving an HTTP redirect
10 months ago
Mike Fährmann
23cd17997d
[wallpapercave] fix extraction
10 months ago
Mike Fährmann
5b979b5706
[xvideos] fix metadata extraction
10 months ago
Mike Fährmann
adc3aa0b77
[zerochan] fix metadata extraction
...
author, path, tags
10 months ago
Mike Fährmann
f9dac43be9
[warosu] fix file URLs
10 months ago
Mike Fährmann
645b4627ef
[sankaku] update URL patterns
10 months ago
Mike Fährmann
1ae43d8123
merge #4841 : [fapello] support '.su' TLD ( #4840 )
10 months ago
Mike Fährmann
b43be67206
[exhentai] add 'gp' option ( #4576 )
10 months ago
Mike Fährmann
cb9a1176e6
[pixeldrain] add 'api-key' option ( #4839 )
10 months ago
Mike Fährmann
e1404827a6
[pixeldrain] add 'file' and 'album' extractors ( #4839 )
10 months ago
enduser420
2402162e8a
[fapello] support '.su' TLD
10 months ago
Mike Fährmann
725c8dd55a
[tmohentai] 'categories' -> 'genres'
...
quite likely that the site meant 'genres' by "Genders"
10 months ago
Mike Fährmann
ce7c4cb544
merge #4832 : [tmohentai] add 'gallery' extractor ( #4808 )
10 months ago
Mike Fährmann
c4a201ed42
[tmohentai] simplify + tests
10 months ago
Mike Fährmann
e17a48fe56
[blogger] inherit from BaseExtractor
...
- support www.micmicidol.club (#4759 )
10 months ago
jsouthgb
714b1a7089
[tmohentai] simplify url matching
10 months ago
jsouthgb
31963fa947
[tmohentai] inherit from GalleryExtractor. refactor metadata.
10 months ago
Mike Fährmann
0fa85360a0
merge #4812 : [erome] add 'count' metadata field
10 months ago
Mike Fährmann
a43cf78bb7
[erome] tests
10 months ago
Mike Fährmann
aea15f6d17
add 'metadata-extractor' option ( #4549 )
10 months ago
Mike Fährmann
34a387b6e2
support 'metadata-*' names for '*-metadata' options
...
For example, instead of 'url-metadata' it is now also possible to use
'metadata-url' as option name.
- metadata-url
- metadata-path
- metadata-http
- metadata-version
- metadata-parent
10 months ago
Mike Fährmann
e97d7b1c85
[exhentai] fix empty api_url with '"source": "hitomi"' ( #4829 )
10 months ago
jsouthgb
ed965eecbb
[tmohentai] refactor to str.format for backwards compatibility
10 months ago
jsouthgb
dad7ba1d58
[tmohentai] fix edge cases. updated archive_fmt and filename_fmt
10 months ago
jsouthgb
286d0cb098
[tmohentai] add support
10 months ago
Mike Fährmann
b714df5a16
disable 'downloader.progress' when using -q/--quiet ( #4810 )
...
it didn't produce any output since output.mode is set to to "null",
but it caused some unnecessary function calls
10 months ago
Mike Fährmann
07cb584231
[behance] add 'modules' option ( #4799 )
10 months ago
Mike Fährmann
6a753d9ff3
[behance] support 'text' modules ( #4799 )
10 months ago
Mike Fährmann
ea78f67860
[downloader:http] skip files not passing filesize-min/-max ( #4821 )
...
instead of failing the download
10 months ago
Mike Fährmann
8bf161e574
reorder post processing options shown by --help
10 months ago
Mike Fährmann
168331d147
replace '--ugoira-conv' etc with a general '--ugoira'
...
update --ugoira webm to use the same FFmpeg args as Danbooru
--ugoira-conv -> --ugoira vp8
--ugoira-conv-lossless -> --ugoira vp9-lossless
--ugoira-conv-copy -> --ugoira copy
(--ugoira-conv and co still work as before,
but --help now lists only --ugoira)
10 months ago
Mike Fährmann
97357e65ee
replace '--mtime-from-date' with a more generic '--mtime'
...
--mtime-from-date -> --mtime date
for the same effect as before
(--mtime-from-date also still works,
but --help now lists only --mtime)
10 months ago
Mike Fährmann
387c8b0950
reword some (internal) option text
10 months ago
jsouthgb
c6ad9bcd9b
[erome] add "count" for albums
10 months ago
Mike Fährmann
51e377e612
add '--cbz' command-line option
10 months ago
Mike Fährmann
4700051562
rework and extend input file processing ( #4732 )
...
- add 2 command-line options to modify input file contents
- -I/--input-file-comment
- -x/--input-file-delete
- implement InputManager class
- move code from util.py to __init__.py
(mainly to avoid import cycles)
10 months ago
Mike Fährmann
17e710c4bf
[oauth] warn when cache is enabled but not writeable ( #4771 )
10 months ago
Mike Fährmann
2e4bf54644
[hentaifoundry] check for and update expired sessions ( #4694 )
10 months ago
Mike Fährmann
0435c6e603
[exhentai] handle 'Downloading … requires GP' errors ( #4576 , #4763 )
10 months ago
Mike Fährmann
4288cea94a
[mastodon] fix reblogs ( #4580 )
10 months ago
Mike Fährmann
7a0f145cbe
[twitter] ignore promoted Tweets ( #4790 , #3894 )
...
add 'ads' option in case someone actually wants to
download promoted content for whatever reason
10 months ago
Mike Fährmann
e8b5e59a08
[weibo] detect redirects to login page ( #4773 )
10 months ago
Mike Fährmann
5e58d2b455
[instagram] fix exception on empty 'video_versions' ( #4795 )
10 months ago
Mike Fährmann
807ddde7e1
release version 1.26.2
11 months ago
Mike Fährmann
6402f2950f
[pp:metadata] ignore non-string tag values ( #4764 )
11 months ago
Mike Fährmann
61d6558322
[exhentai] try to avoid 'DH_KEY_TOO_SMALL' errors ( #1021 , #4593 )
11 months ago
Mike Fährmann
69b931b9bb
[exhentai] provide fallback URLs ( #1021 , #4745 )
11 months ago
Mike Fährmann
007c433677
[patreon] support 'id:<campaign_id>' in place of a user name
...
https://patreon.com/id:12345
… and remove 'campaign-id' config option
11 months ago
Mike Fährmann
3984a49abf
[nijie] set 1-2s delay between requests to avoid 429 errors
11 months ago
Mike Fährmann
dd14adccf6
[pixiv] allow cookies for non-OAuth URLs ( #4760 )
11 months ago
Mike Fährmann
caf31e751c
[kemonoparty] limit 'title' length ( #4741 )
11 months ago
Mike Fährmann
43d0c49d7e
[exhentai] fix original image URLs ( #4754 )
11 months ago
Mike Fährmann
43a3d93467
merge #4755 : [twitter] recognize fixupx.com URLs
11 months ago
Mike Fährmann
fc8f86bf24
[hitomi] recognize 'imageset' gallery URLs ( #4756 )
11 months ago
Mike Fährmann
91e20eb59b
[fantia] simplify 'tags' to a list of strings ( #4752 )
11 months ago
Mike Fährmann
72b18d701f
represent util.NONE as 'null' in JSON output
...
was '"None"' before
11 months ago
thatfuckingbird
44d7964c09
[twitter] recognize fixupx.com URLs
11 months ago
Mike Fährmann
56cd9d408d
[weibo] fix Sina Visitor request
11 months ago
Mike Fährmann
68e72a836c
[exhentai] fix extraction ( #4730 )
...
- update to new API response layout
- use proper API server URL
- fix 'filesize' metadata
11 months ago
Mike Fährmann
fd8f58ad76
[behance] unescape embed URLs ( #4742 )
11 months ago
Mike Fährmann
ca1d5c2c0c
merge #4738 : [patreon] parse new bootstrap data format ( #4736 )
11 months ago
Mike Fährmann
4730de163f
[patreon] refactor _extract_bootstrap()
11 months ago
Mike Fährmann
e46efbd5b5
prevent crash when 'stdout.line_buffering' is not defined ( #642 )
11 months ago
Mike Fährmann
c9a2be36d4
[sankaku] support '/posts/' tag search URLs ( #4740 )
11 months ago
Tobias Hellmann
28ada11cba
Try to parse newer HTTP response from Patreon
11 months ago
Mike Fährmann
fd36eafe32
[twitter] restore truncated retweet text ( #3430 , #4690 )
11 months ago
Mike Fährmann
218295a4c6
[twitter] fix avatars without 'date' information ( #4696 )
11 months ago
Mike Fährmann
969be65d0b
[instagram] update API headers
11 months ago
Mike Fährmann
d0effcae20
[kemonoparty] add 'revision_index' metadata field ( #4727 )
11 months ago
Mike Fährmann
3bbaa875f1
[kemonoparty] fix parsing of non-standard 'dates' ( #4676 )
11 months ago
Mike Fährmann
75dec71253
[idolcomplex] disable Referer headers by default ( #4726 )
11 months ago
Mike Fährmann
a09df34bcf
merge #4714 : [4archive] add 'thread' and 'board' extractors
...
(#1262 , #2418 , #4400 , #4710 )
11 months ago
enduser420
acb713b95a
[4archive] update
11 months ago
Mike Fährmann
6766877524
merge #4693 : [reddit] support Reddit Mobile share links
11 months ago
Mike Fährmann
1042278bec
[misskey] support 'misskey.design' ( #4713 )
11 months ago
Mike Fährmann
12a800ce21
[patreon] improve 'campaign_id' handling ( #4699 , #4715 )
...
- add ways to directly specify a 'campaign_id'
- 'campaign-id' config option
- 'c' or 'campaign_id' URL query parameter
- more descriptive error messages
- show 'campaign_id' value in debug log
11 months ago
Mike Fährmann
31dbbffc0b
[twitter] cache 'user_by_…' results ( #4719 )
11 months ago
enduser420
c0714d5585
[4archive] add 'thread' and 'board' extractors
11 months ago
inty
b68aad3dab
[reddit] implement Reddit Mobile share links
11 months ago
Mike Fährmann
95a74be2a5
release version 1.26.1
11 months ago
Mike Fährmann
de224ef3e4
[cookies] include exception in fallback warning
11 months ago
Mike Fährmann
7958ab1946
[newgrounds] support 'imageData' files ( #4642 )
11 months ago
Mike Fährmann
b52fd91ac6
[sankaku] support '/posts/' URLs ( #4688 )
11 months ago
Mike Fährmann
b8674776e9
[4chanarchives] disable Referer headers by default ( #4686 )
11 months ago
Mike Fährmann
78493f0870
[bunkr] fix '/d/' file URLs ( #4685 )
11 months ago
Mike Fährmann
b2c3db3e24
[bunkr] add extractor for media URLs ( #4684 )
11 months ago
Mike Fährmann
0d52b775cb
[kemonoparty] add 'revisions' option ( #4498 , #4597 )
11 months ago
Mike Fährmann
6e830ffc9e
[kemonoparty] support post searches ( #3385 , #4057 )
11 months ago
Mike Fährmann
aaf539009b
[kemonoparty] initial support for post revisions ( #4498 , #4597 )
...
- single revision
https://kemono.party/SERVICE/user/12345/post/12345/revision/12345
- all revisions
https://kemono.party/SERVICE/user/12345/post/12345/revisions
11 months ago
Mike Fährmann
174191cb79
[kemonoparty] restore discord pagination ( #4676 )
11 months ago
Mike Fährmann
c9a976d8a6
[kemonoparty] various updates and fixes ( #4676 , #4681 )
...
- fix pagination
- fix 'date' metadata
- fix discord channel API endpoint
11 months ago
Klion Xu
dc1c2139b1
fix line too long
11 months ago
Klion Xu
6b22af9720
[kemonoparty] update API endpoint ( #4676 )
11 months ago
Mike Fährmann
bfdc07632a
[deviantart] expand nested comment replies ( #4653 )
11 months ago
Mike Fährmann
390d14dbcc
[chevereto] support 'img.kiwi' and 'deltaporno.com' ( #4664 , #1381 )
11 months ago
Mike Fährmann
727c8eec6c
merge #4667 : [redgifs] fix 'niches' extraction ( #4666 )
11 months ago
Mike Fährmann
2911ed1240
[chevereto] add generic extractors ( #4664 )
...
- support jpgfish
- support pixl.li / pixl.is (#3179 , #4357 )
11 months ago
enduser420
db3363ac0b
[redgifs] fix 'niches' extraction
11 months ago
Mike Fährmann
ade8347ead
[kemonoparty] fix DM dates
11 months ago
Mike Fährmann
6dfe200ae4
[kemonoparty] support discord URLs with channel IDs ( #4662 )
11 months ago
Mike Fährmann
c6a3892210
[imgbb] update username extraction ( #4626 )
11 months ago
Mike Fährmann
830a48bca4
[fantia] bad workaround for 833dce14
( #4627 )
...
at least this makes "filter": "content_num == content_count+1"
with "event": "post-after" work
11 months ago
Mike Fährmann
13ce3a9acb
[warosu] fix extraction ( #4634 )
11 months ago
Mike Fährmann
c4c4e4d2f4
[newgrounds] improve 'art-image' extraction ( #4642 )
...
- download files in original resolution
- replace .webp with extension of first file
11 months ago
Mike Fährmann
833dce141f
[fantia] add 'content_count' and 'content_num' metadata fields ( #4627 )
11 months ago
Mike Fährmann
2d41702762
[deviantart] implement '"group": "skip"' ( #4630 )
11 months ago
Mike Fährmann
992e86ec94
[deviantart] disable 'jwt' ( #4652 )
11 months ago
Mike Fährmann
2974b8e3c8
[moebooru] add 'metadata' option ( #4646 )
...
for extended 'pool' metadata
11 months ago
Mike Fährmann
d194ea68a9
[cookies] open cookie databases in read-only mode
...
bypasses the need to copy the entire database file
might solve #4195
11 months ago
Mike Fährmann
8bb7243c10
[reddit] fix wrong previews ( #4649 )
...
caused by a failed comment URL
using the main submission's preview as fallback
14af15bd
4963bb9b
12 months ago
Mike Fährmann
08bdde5aac
merge #4619 : [twitter] add 'sensitive' metadata field
12 months ago
Mike Fährmann
f3d6aaff13
[twitter] rename to 'sensitive'; use 'tget()'
12 months ago
Mike Fährmann
95c280c59b
[imgbb] update pagination end condition ( #4626 )
12 months ago
Mike Fährmann
2e350dd82a
merge #4626 : [imgbb] fix 'user' extraction, add 'displayname'
12 months ago
Mike Fährmann
a2daa9befe
[imgbb] fix flake8 and username order
12 months ago
Mike Fährmann
67ba4ee842
[pp:exec] support more replacement fields for '--exec' ( #4633 )
...
- {_directory}
- {_filename}
- {_path} (alias for {})
12 months ago
Mike Fährmann
9a008523ac
[hentaifoundry] fix '.swf' file downloads ( #4641 )
12 months ago
Mike Fährmann
15f940819b
[newgrounds] support 'art-image' files ( #4642 )
12 months ago
Mike Fährmann
63db54b905
[patreon] update 'campaign_id' path ( #4639 )
12 months ago
HRXN
b846f56c3a
[imgbb] Fix `user` extraction, add `displayname`
12 months ago
Mike Fährmann
efaab4fbfa
[twitter] fix crash due to missing 'source' ( #4620 )
...
regression caused by 06aaedde
12 months ago
Nahida
3438a3098d
[twitter] add possible_sensitive field
12 months ago
Mike Fährmann
85357c1ef8
release version 1.26.0
12 months ago
Mike Fährmann
64dbc58a5a
[deviantart] update Eclipse API endpoints 2 ( #4615 )
12 months ago
Mike Fährmann
84fbbd96aa
[shimmie2] remove 'meme.museum'
12 months ago
Mike Fährmann
aa77fda78c
[instagram] better error message for invalid users ( #4606 )
12 months ago
Mike Fährmann
482f002e1f
[nsfwalbum] detect '/error.jpg' images ( #4598 )
12 months ago