663b8d789a
Fix linting
9 months ago
74c225f94e
[bato] add support
9 months ago
f9544194c0
[paheal] restore 'extension' metadata ( #4976 )
9 months ago
77d46e6f0c
[lynxchan] update 'bbw-chan' domain ( #4970 )
9 months ago
766316e436
[imagechest] fix loading more than 10 images in a gallery ( #4469 )
9 months ago
108c978073
merge #4919 : [postmill] add support ( #4917 )
9 months ago
8a42ea736a
[postmill] implement suggestions
9 months ago
c184454efb
[shimmie2] small optimizations
...
- unroll/remove loop
- avoid copy
9 months ago
7cd0211cc9
[shimmie2] autodetect single or double quotes
9 months ago
2a60645095
[deviantart] set 'is_original' for intermediary URLs to 'false'
9 months ago
01bb75f6cb
merge #4945 : {shimmie2[ support 'rule34hentai.net' ( #861 , #4789 )
9 months ago
79e4606893
[rule34hentai] cleanup
...
- fix using 'self._posts_rule34hentai'
- fix 'file_url' for posts
- update docs/supportedsites
- add tests
9 months ago
ef370df41d
[shimmie2] support 'rule34hentai.net'
...
- Add files via upload
- Update shimmie2.py
- Update shimme2.py
- Delete gallery_dl/extractor/shimme2.py
- spacefix
- Update shimmie2.py
- Update shimmie2.py
- flask warnings1
- Update shimmie2.py
- Update shimmie2.py
9 months ago
627ed794a2
[danbooru] provide 'tags' as list ( #4942 )
...
keep the old 'tag_string' values around, similar to sankaku
a lot of repeat code ...
would be a lot less bad if "".split(" ") returned an empty list
9 months ago
fbebc58189
[deviantart] add 'intermediary' option ( #4955 )
9 months ago
75fa1a5553
[pinterest] remove login code
...
this has been broken since forever
and is still "protected" by an invisible recaptcha check
9 months ago
92ff99c8e5
[twitter] remove 'syndication' option ( #3889 )
9 months ago
a75f85a2c2
[twitter] remove 'date_liked' ( #3850 , #4108 , #4657 )
...
Twitter's 'sortIndex' can't be used to calculate the timestamp
of when a Tweet was liked anymore.
9 months ago
a94f944148
[twitter] default to 'tweets' timeline when 'replies' are enabled ( #4953 )
9 months ago
a30a3e44d5
[nijie] move 'username required' out of _login_impl
9 months ago
57fc6fcf83
replace '24*3600' with '86400'
...
and generalize cache maxage values
9 months ago
1f9b16a70b
replace static 'sleep-request' defaults with dynamic ones
9 months ago
b127321b5c
[exhentai] only show 'using e-hentai.org' warning for exh domains
9 months ago
e097aaf64a
[exhentai] output continuation URL when interrupted ( #4782 )
9 months ago
99aa923322
[inkbunny] improve '/submissionsviewall.php' patterns ( #4934 )
...
allow 'mode=…' to be in any position
don't require it to be somewhere in the middle
9 months ago
3f9c113d78
[mastodon] Support non-numeric status IDs ( #4936 )
9 months ago
2852404e49
[inkbunny] add 'unread' extractor ( #4934 )
9 months ago
8b87a5330d
[inkbunny] stop pagination on empty results
9 months ago
6cd5e6adad
[patreon] fix bootstrap data extraction ( #4904 )
9 months ago
aac8bb4eae
[deviantart] simplify 9951c112
9 months ago
9951c112f8
[deviantart] workaround for integer client_id values ( #4924 )
9 months ago
a37b7759bc
[myhentaigallery] recognize '/g/' URLs ( #4920 )
9 months ago
da76e13e3b
[tumblr] fix exception after waiting for rate limit ( #4916 )
...
use a loop instead of recursive function calls
9 months ago
fbe14a2745
[postmill] add support
9 months ago
d59d4ebff4
[tumblr] support infinite 'fallback-retries'
9 months ago
2d5cda2b92
[exhentai] fix TypeError for infinite 'fallback-retries' ( #4911 )
9 months ago
92fbf09643
remove single quotes in some logging messages ( #4908 )
...
('FileNotFoundError: [Errno 2] No such file or directory: ''')
->
(FileNotFoundError: [Errno 2] No such file or directory: '')
9 months ago
9a001fa6e4
merge #4906 : [patreon] fix bootstrap data extraction ( #4904 )
9 months ago
66cbe9da41
- fix style check failure "line to long"
9 months ago
244444b194
- adapt code to current code style
9 months ago
fd06255f93
- reformat and refactor to pass tests
9 months ago
5ff7106d4f
- add code for the situation when Patreon is using window.patreon = wrapInProxy({"bootstrap":' to store metadata
...
- refactor code to make it more readable
- output page content when the HTML structure is unknown (to make debugging easier)
9 months ago
ac22bbe80c
[twitter] retry API requests only for Timeout errors ( #4811 )
10 months ago
c55955db03
[twitter] quick and dirty fix for /media changes ( #4898 )
10 months ago
9a8dc6b02b
[exhentai] add 'fallback-retries' option ( #4792 )
10 months ago
bf74eb5c46
merge #4886 : [urlgalleries] add 'gallery' extractor ( #919 , #1184 , #2905 )
10 months ago
c29ae9af08
[urlgalleries] simplify + resolve redirects
10 months ago
e256434c9e
use custom HTTPBasicAuth class
...
to support LazyPrompt as password
and to generate the Authorization header only once
instead of for every request
10 months ago
6a4218aa23
handle 'json' parameter in Extractor.request() manually
...
Mainly to allow passing custom classes like util.LazyPrompt,
but also to simplify and streamline how requests handles it.
10 months ago
9dd5cb8c8a
interactively prompt for passwords on login when none is provided
10 months ago
4eb3590103
[nijie] fix image URLs of multi-image posts ( #4876 )
10 months ago
a4e6ea667b
[twitter] retry API calls when their response contains errors ( #4811 )
10 months ago
cf5702c843
[twitter] generalize "Login Required" error ( #4734 , #4324 )
10 months ago
ecaa0feb5d
[urlgalleries] add support
10 months ago
1770c31e63
[urlgalleries] add support
10 months ago
da0da0faaa
[exhentai] store more cookies when logging in ( #4881 )
...
include 'igneous', 'hath_perks', etc
and not just 'ipb_member_id' and 'ipb_pass_hash' like before
10 months ago
c83fbe6c2d
merge #4855 : [nitter] fix video extraction ( #4853 )
10 months ago
013ca21543
[idolcomplex] update to site layout changes
10 months ago
1e9bacd169
[nitter] fix video extraction
10 months ago
9f3368c46f
[pornhub] fix 'user' metadata for gifs
10 months ago
bdb3ce7217
[foolslide] remove 'powermanga.org'
10 months ago
d9734ce008
[cyberdrop] update to site layout changes
10 months ago
8ac68ffba2
[hentaicosplays] force 'https://' for download URLs
10 months ago
fc1101779c
[hiperdex] fix 'manga' metadata
10 months ago
d119507037
[imagefap] fix single image resolution
...
Downloading from a single image page like
https://www.imagefap.com/photo/123456789/
returned only the thumbnail URL.
10 months ago
311ec1d9ef
[mangaread] fix extraction
10 months ago
7608201a44
[tumblr] fix 'day' extractor
...
another bug caused by a383eca7
10 months ago
c8c744a7c0
[webtoons] fix pagination when receiving an HTTP redirect
10 months ago
23cd17997d
[wallpapercave] fix extraction
10 months ago
5b979b5706
[xvideos] fix metadata extraction
10 months ago
adc3aa0b77
[zerochan] fix metadata extraction
...
author, path, tags
10 months ago
f9dac43be9
[warosu] fix file URLs
10 months ago
645b4627ef
[sankaku] update URL patterns
10 months ago
1ae43d8123
merge #4841 : [fapello] support '.su' TLD ( #4840 )
10 months ago
b43be67206
[exhentai] add 'gp' option ( #4576 )
10 months ago
cb9a1176e6
[pixeldrain] add 'api-key' option ( #4839 )
10 months ago
e1404827a6
[pixeldrain] add 'file' and 'album' extractors ( #4839 )
10 months ago
2402162e8a
[fapello] support '.su' TLD
10 months ago
725c8dd55a
[tmohentai] 'categories' -> 'genres'
...
quite likely that the site meant 'genres' by "Genders"
10 months ago
ce7c4cb544
merge #4832 : [tmohentai] add 'gallery' extractor ( #4808 )
10 months ago
c4a201ed42
[tmohentai] simplify + tests
10 months ago
e17a48fe56
[blogger] inherit from BaseExtractor
...
- support www.micmicidol.club (#4759 )
10 months ago
714b1a7089
[tmohentai] simplify url matching
10 months ago
31963fa947
[tmohentai] inherit from GalleryExtractor. refactor metadata.
10 months ago
0fa85360a0
merge #4812 : [erome] add 'count' metadata field
10 months ago
a43cf78bb7
[erome] tests
10 months ago
34a387b6e2
support 'metadata-*' names for '*-metadata' options
...
For example, instead of 'url-metadata' it is now also possible to use
'metadata-url' as option name.
- metadata-url
- metadata-path
- metadata-http
- metadata-version
- metadata-parent
10 months ago
e97d7b1c85
[exhentai] fix empty api_url with '"source": "hitomi"' ( #4829 )
10 months ago
ed965eecbb
[tmohentai] refactor to str.format for backwards compatibility
10 months ago
dad7ba1d58
[tmohentai] fix edge cases. updated archive_fmt and filename_fmt
10 months ago
286d0cb098
[tmohentai] add support
10 months ago
07cb584231
[behance] add 'modules' option ( #4799 )
10 months ago
6a753d9ff3
[behance] support 'text' modules ( #4799 )
10 months ago
c6ad9bcd9b
[erome] add "count" for albums
10 months ago
17e710c4bf
[oauth] warn when cache is enabled but not writeable ( #4771 )
10 months ago
2e4bf54644
[hentaifoundry] check for and update expired sessions ( #4694 )
10 months ago
0435c6e603
[exhentai] handle 'Downloading … requires GP' errors ( #4576 , #4763 )
10 months ago
4288cea94a
[mastodon] fix reblogs ( #4580 )
10 months ago
7a0f145cbe
[twitter] ignore promoted Tweets ( #4790 , #3894 )
...
add 'ads' option in case someone actually wants to
download promoted content for whatever reason
10 months ago
e8b5e59a08
[weibo] detect redirects to login page ( #4773 )
10 months ago
5e58d2b455
[instagram] fix exception on empty 'video_versions' ( #4795 )
10 months ago
61d6558322
[exhentai] try to avoid 'DH_KEY_TOO_SMALL' errors ( #1021 , #4593 )
11 months ago
69b931b9bb
[exhentai] provide fallback URLs ( #1021 , #4745 )
11 months ago
007c433677
[patreon] support 'id:<campaign_id>' in place of a user name
...
https://patreon.com/id:12345
… and remove 'campaign-id' config option
11 months ago
3984a49abf
[nijie] set 1-2s delay between requests to avoid 429 errors
11 months ago
dd14adccf6
[pixiv] allow cookies for non-OAuth URLs ( #4760 )
11 months ago
caf31e751c
[kemonoparty] limit 'title' length ( #4741 )
11 months ago
43d0c49d7e
[exhentai] fix original image URLs ( #4754 )
11 months ago
43a3d93467
merge #4755 : [twitter] recognize fixupx.com URLs
11 months ago
fc8f86bf24
[hitomi] recognize 'imageset' gallery URLs ( #4756 )
11 months ago
91e20eb59b
[fantia] simplify 'tags' to a list of strings ( #4752 )
11 months ago
44d7964c09
[twitter] recognize fixupx.com URLs
11 months ago
56cd9d408d
[weibo] fix Sina Visitor request
11 months ago
68e72a836c
[exhentai] fix extraction ( #4730 )
...
- update to new API response layout
- use proper API server URL
- fix 'filesize' metadata
11 months ago
fd8f58ad76
[behance] unescape embed URLs ( #4742 )
11 months ago
ca1d5c2c0c
merge #4738 : [patreon] parse new bootstrap data format ( #4736 )
11 months ago
4730de163f
[patreon] refactor _extract_bootstrap()
11 months ago
c9a2be36d4
[sankaku] support '/posts/' tag search URLs ( #4740 )
11 months ago
28ada11cba
Try to parse newer HTTP response from Patreon
11 months ago
fd36eafe32
[twitter] restore truncated retweet text ( #3430 , #4690 )
11 months ago
218295a4c6
[twitter] fix avatars without 'date' information ( #4696 )
11 months ago
969be65d0b
[instagram] update API headers
11 months ago
d0effcae20
[kemonoparty] add 'revision_index' metadata field ( #4727 )
11 months ago
3bbaa875f1
[kemonoparty] fix parsing of non-standard 'dates' ( #4676 )
11 months ago
75dec71253
[idolcomplex] disable Referer headers by default ( #4726 )
11 months ago
a09df34bcf
merge #4714 : [4archive] add 'thread' and 'board' extractors
...
(#1262 , #2418 , #4400 , #4710 )
11 months ago
acb713b95a
[4archive] update
11 months ago
6766877524
merge #4693 : [reddit] support Reddit Mobile share links
11 months ago
1042278bec
[misskey] support 'misskey.design' ( #4713 )
11 months ago
12a800ce21
[patreon] improve 'campaign_id' handling ( #4699 , #4715 )
...
- add ways to directly specify a 'campaign_id'
- 'campaign-id' config option
- 'c' or 'campaign_id' URL query parameter
- more descriptive error messages
- show 'campaign_id' value in debug log
11 months ago
31dbbffc0b
[twitter] cache 'user_by_…' results ( #4719 )
11 months ago
c0714d5585
[4archive] add 'thread' and 'board' extractors
11 months ago
b68aad3dab
[reddit] implement Reddit Mobile share links
11 months ago
7958ab1946
[newgrounds] support 'imageData' files ( #4642 )
11 months ago
b52fd91ac6
[sankaku] support '/posts/' URLs ( #4688 )
11 months ago
b8674776e9
[4chanarchives] disable Referer headers by default ( #4686 )
11 months ago
78493f0870
[bunkr] fix '/d/' file URLs ( #4685 )
11 months ago
b2c3db3e24
[bunkr] add extractor for media URLs ( #4684 )
11 months ago
0d52b775cb
[kemonoparty] add 'revisions' option ( #4498 , #4597 )
11 months ago
6e830ffc9e
[kemonoparty] support post searches ( #3385 , #4057 )
11 months ago
aaf539009b
[kemonoparty] initial support for post revisions ( #4498 , #4597 )
...
- single revision
https://kemono.party/SERVICE/user/12345/post/12345/revision/12345
- all revisions
https://kemono.party/SERVICE/user/12345/post/12345/revisions
11 months ago
174191cb79
[kemonoparty] restore discord pagination ( #4676 )
11 months ago
c9a976d8a6
[kemonoparty] various updates and fixes ( #4676 , #4681 )
...
- fix pagination
- fix 'date' metadata
- fix discord channel API endpoint
11 months ago
dc1c2139b1
fix line too long
11 months ago
6b22af9720
[kemonoparty] update API endpoint ( #4676 )
11 months ago
bfdc07632a
[deviantart] expand nested comment replies ( #4653 )
11 months ago
390d14dbcc
[chevereto] support 'img.kiwi' and 'deltaporno.com' ( #4664 , #1381 )
11 months ago
727c8eec6c
merge #4667 : [redgifs] fix 'niches' extraction ( #4666 )
11 months ago
2911ed1240
[chevereto] add generic extractors ( #4664 )
...
- support jpgfish
- support pixl.li / pixl.is (#3179 , #4357 )
11 months ago
db3363ac0b
[redgifs] fix 'niches' extraction
11 months ago
bug-assassin