5ca9d55595
merge #3870 : [blogger] update 'sub' regex to get the highest resolution url
1 year ago
fd7ce4c081
merge #3868 : [shopify] fix 'collection' extractor
1 year ago
135ac9c302
merge #3854 : [twitter] fix: graphql_timeline_v2_bookmark_timeline cannot be null
1 year ago
bbb1e34c34
[blogger] update sub regex
1 year ago
96e3dd2128
[shopify] fix 'collection' extractor
1 year ago
ac97aca99c
[realbooru] fix extraction
...
get file URLs from HTML pages
1 year ago
75666cf9c3
[danbooru] reduce API requests for fetching extended 'metadata'
...
Instead of using one additional API request per post object (N+1),
this requires only one request per 200-post batch.
1 year ago
bebbff6578
fix: graphql_timeline_v2_bookmark_timeline cannot be null
...
twitter: 400 Bad Request (The following features cannot be null: graphql_timeline_v2_bookmark_timeline)
1 year ago
71b26adb9b
[urlshortener] add tinyurl.com as an example
2 years ago
421db26aff
[bunkr] update domain to 'bunkr.la'
2 years ago
9e2a945013
[urlshortener] add support for bit.ly & t.co
2 years ago
9b5e7ce8b9
[hiperdex] fix extraction
2 years ago
89a67c45e0
[nitter] support nitter.it ( #3819 )
2 years ago
88f29a751d
[nitter] skip broadcasts
...
instead of downloading an "Unsupported feature" HTML page
2 years ago
1e013eba5a
[nitter] fix extraction for instances without user banners
2 years ago
d94aa1ee02
[gelbooru] fix --range for favorites ( #3704 )
2 years ago
1f82b00b8f
[gelbooru] fix and improve --range for pools
2 years ago
197882cf12
[twitter] add 'hashtag' extractor ( #3783 )
2 years ago
9789ebac52
[naverwebtoon] fix extraction ( #3729 )
2 years ago
72f1f16eb2
[weibo] support 'mix_media_info' entries ( #3793 )
2 years ago
2bb937014f
[twitter] fall back to legacy /media endpoint when not logged in
2 years ago
b68094d326
[twitter] support 'note_tweet's
2 years ago
3dcabc97ed
[twitter] update API endpoints and parameters
2 years ago
dcb8af659a
[gelbooru] extract favorites without needing cookies ( #3704 )
...
TODO: fix --range
2 years ago
b756dc13aa
[gelbooru] warn about missing cookies for favorites ( #3704 )
...
and add docstring so it shows up in --list-extractors
2 years ago
17bd053d94
[hiperdex] fix extraction ( #3768 )
2 years ago
817fc0fbd1
[nitter] remove nitter.pussthecat.org
...
"Shutdown"
2 years ago
67ec91cdbd
[downloader:http] change '_http_retry' to accept a Python function
...
and rename '_http_retry_codes' to '_http_retry'
(#3569 )
2 years ago
175822e065
merge #3738 : [generic] add tests
2 years ago
4883420e67
[generic] revert pattern change
2 years ago
9037128315
[twitter] fix some 'original' retweets not downloading ( #3744 )
2 years ago
ea3d95e7e8
merge #3740 : [deviantart] add support for fxdeviantart.com URLs
2 years ago
9abcb2b6e5
update headers and ciphers for '"browser": "chrome"'
2 years ago
c489aecb3e
[deviantart] add support for fxdeviantart.com URLs
...
fxdeviantart.com is a service that fixes embeds on Discord, similar to
fxtwitter.com
2 years ago
34a7fab0e2
[generic] add support for IDNs
...
(internationalized domain name)
2 years ago
c9a7345228
[newgrounds] prevent archive ID overlap ( #3681 )
...
add an 'i' and 'a' prefix to image and audio files
(/art/view/, /audio/listen/)
since their numeric ID may conflict with movies and other media
2 years ago
da9840a39d
[reddit] update 'videos' option ( #3712 )
...
- add 'dash' to directly extract DASH manifest URLs
(was default behavior since a7c79531a7c79531
2 years ago
baf41d7437
[misskey] update ( #3717 )
...
- add module docstring
- add options to docs/gallery-dl.conf
2 years ago
6762d99515
merge #3717 : [misskey] add misskey extractors
2 years ago
b8a702929d
[oauth] import extractor modules on demand
2 years ago
dd88740ec7
replace remaining instances of base64 with binascii
2 years ago
e1867cf5eb
[misskey] add 'renotes' and 'replies' options
2 years ago
a95b5e0d8e
[misskey] add misskey extractors
2 years ago
0d142e403c
[szurubooru] add 'tag' and 'post' extractors ( #3583 , #3713 )
2 years ago
b14f8d5817
[gelbooru] add 'favorite' extractor ( #3704 )
...
requires logged in cookies to work
2 years ago
a70a3e5da6
[mangasee] extract 'author' and 'genre' metadata ( #3703 )
...
Both are lists/arrays. Use {author!S} or {genre:J, } to format them.
2 years ago
6b03506655
[deviantart] allow searching when not logged in
2 years ago
511a051705
[fanbox] fix crash with missing images ( #3673 )
2 years ago
3fa456d989
[deviantart] remove mature scraps warning ( #3691 )
...
warn about private deviations
when paginating over eclipse results
2 years ago
51301e0c31
replace remaining time.sleep() calls
...
with Extractor.sleep() or request_interval
2 years ago
6ed4309aba
[deviantart] add 'gallery-search' extractor ( #1695 )
2 years ago
3d8777fbc1
move user agent string to util.py
2 years ago
e1df7f73b1
[deviantart] add 'search' extractor
...
(#538 , #1264 , #2954 , #2970 , #3577 )
Requires login to fetch any results, since the API endpoint raises an
error for not logged in requests.
TODO: parse HTML search results
2 years ago
4f029ab38b
[pornpics] support '/pornstar' and '/channels' listings
...
- fix docstring (#3671 )
- simplify code
2 years ago
cbe4769246
[danbooru] use gallery-dl UA ( #3665 )
...
this removes the ability to set a custom UA via 'user-agent' option
for extractor requests
2 years ago
253ac08203
pre-define and use 'gallery-dö/<version>' UA string
2 years ago
b4899c266f
merge #3656 : [deviantart] fix crash when handling deleted deviations in status updates
2 years ago
bb11c2a576
merge #3662 : [redgifs] add 'collection' extractors
2 years ago
884f1848d6
[redgifs] fix syntax for older Python versions
...
and update docs/supportedsites
2 years ago
725baedad3
[deviantart] use '/collections/all' endpoint for favorites
...
(#3666 ,#3668)
2 years ago
2bd8f2f4bd
[pornpics] add 'search' and 'tag' extractors
...
(#263 , #3544 , #3654 )
2 years ago
79bc82884c
[pornpics] add 'gallery' extractor ( #263 , #3544 , #3654 )
2 years ago
7bdc1d6d3d
[manganelo] update and fix metadata extraction
2 years ago
363bb76dff
[manganelo] simplify URL pattern
2 years ago
b28bd9789e
[redgifs] add 'collection' extractors
2 years ago
f4e211356d
[deviantart] slight refactor
2 years ago
bd5d08abbc
[catbox] add 'file' extractor ( #3570 )
2 years ago
8e1e8a5bea
[soundgasm] rewrite ( #3578 )
...
use a more standard extractor structure to make -A work as expected
2 years ago
0b93420a81
[pinterest] unescape search terms ( #3621 )
2 years ago
ad96e70546
[bunkr] fix extraction ( #3636 , #3655 )
2 years ago
9335d55bbc
[manganelo] support mobile-only chapters
2 years ago
a74114ef7a
[deviantart] fix crash when handling deleted deviations
...
in status updates
2 years ago
75570ad3f1
[oauth] remove stray 'exit()' ( #3628 )
...
- bug from 70ce45d9
2 years ago
8fb043e8ff
[tumblr] raise more detailed errors for dashboard-only blogs
...
(#3628 )
2 years ago
ce996dd21b
[poipiku] warn about incorrect passwords ( #3646 )
2 years ago
70ce45d965
[oauth] use default name for browsers without 'name' attribute
...
(#3645 )
Seem to only be an issue for MacOSXOSAScript before Python 3.11.
d12bec6993
2 years ago
2a53e6445c
[bunkr] update domain ( #3636 )
2 years ago
5503ac4d5e
replace json.dumps with direct calls to JSONEncoder.encode
2 years ago
dd884b02ee
replace json.loads with direct calls to JSONDecoder.decode
2 years ago
8805bd38ab
merge #3622 : [imagetwist] add phun.imagetwist.com and imagehaha.com support
2 years ago
706ec70e89
[imagetwist] simplify pattern and add tests
2 years ago
f2e91732ae
[instagram] add 'user' metadata field ( #3107 )
...
at the moment only for URLs that need to translate user name to ID
2 years ago
29f0830b53
[imagetwist] add phun.imagetwist.com and imagehaha.com alias to imagetwist extractor
2 years ago
bbf0911a46
[e621] implement 'notes' and 'pools' metadata extraction
...
(#3425 )
2 years ago
925b467496
split e621 from danbooru module ( #3425 )
2 years ago
1ae48a54f8
[twitter] add 'transform' option
2 years ago
489c51cecc
[telegraph] fix extraction when images not in <figure> ( #3590 )
2 years ago
0f7e6c422a
merge #3596 : [shopify] support ohpolly.com
2 years ago
fcf7030b85
[shopify] support ohpolly.com
2 years ago
a6a631f992
merge #3589 : [redgifs] support v3 URLs
2 years ago
137a395ae0
[imagefap] fix infinite pagination loop ( #3594 )
2 years ago
3c708ade8f
[imagefap] fix metadata extraction
2 years ago
17e24eacf0
[imagefap] update 'gallery' URLs ( #3595 )
2 years ago
c2bc70593e
implement ability to load external extractor classes
...
- -X/--extractors
- extractor.module-sources
2 years ago
a18f627bfc
[redgifs] support v3 URLs
2 years ago
13a90969c7
merge #3575 : [nudecollect] add 'image' and 'album' extractors
2 years ago
aacd27e4ef
merge #3581 : [hotleak] fix video URLs
2 years ago
abc3619feb
[lexica] add 'search' extractor ( #3567 )
2 years ago
7c9b1ec830
[hotleak] optimize decoding video URLs
...
- use binascii module
- combine slice and reverse step
2 years ago
f14dbfe079
Make decode_video_url static (used in both post and creator extractor).
2 years ago
Mike Fährmann