enduser420
e1867cf5eb
[misskey] add 'renotes' and 'replies' options
2 years ago
enduser420
a95b5e0d8e
[misskey] add misskey extractors
2 years ago
Mike Fährmann
0d142e403c
[szurubooru] add 'tag' and 'post' extractors ( #3583 , #3713 )
2 years ago
Mike Fährmann
b14f8d5817
[gelbooru] add 'favorite' extractor ( #3704 )
...
requires logged in cookies to work
2 years ago
Mike Fährmann
a70a3e5da6
[mangasee] extract 'author' and 'genre' metadata ( #3703 )
...
Both are lists/arrays. Use {author!S} or {genre:J, } to format them.
2 years ago
Mike Fährmann
6b03506655
[deviantart] allow searching when not logged in
2 years ago
Mike Fährmann
511a051705
[fanbox] fix crash with missing images ( #3673 )
2 years ago
Mike Fährmann
3fa456d989
[deviantart] remove mature scraps warning ( #3691 )
...
warn about private deviations
when paginating over eclipse results
2 years ago
Mike Fährmann
51301e0c31
replace remaining time.sleep() calls
...
with Extractor.sleep() or request_interval
2 years ago
Mike Fährmann
6ed4309aba
[deviantart] add 'gallery-search' extractor ( #1695 )
2 years ago
Mike Fährmann
3d8777fbc1
move user agent string to util.py
2 years ago
Mike Fährmann
e1df7f73b1
[deviantart] add 'search' extractor
...
(#538 , #1264 , #2954 , #2970 , #3577 )
Requires login to fetch any results, since the API endpoint raises an
error for not logged in requests.
TODO: parse HTML search results
2 years ago
Mike Fährmann
4f029ab38b
[pornpics] support '/pornstar' and '/channels' listings
...
- fix docstring (#3671 )
- simplify code
2 years ago
Mike Fährmann
cbe4769246
[danbooru] use gallery-dl UA ( #3665 )
...
this removes the ability to set a custom UA via 'user-agent' option
for extractor requests
2 years ago
Mike Fährmann
253ac08203
pre-define and use 'gallery-dö/<version>' UA string
2 years ago
Mike Fährmann
b4899c266f
merge #3656 : [deviantart] fix crash when handling deleted deviations in status updates
2 years ago
Mike Fährmann
bb11c2a576
merge #3662 : [redgifs] add 'collection' extractors
2 years ago
Mike Fährmann
884f1848d6
[redgifs] fix syntax for older Python versions
...
and update docs/supportedsites
2 years ago
Mike Fährmann
725baedad3
[deviantart] use '/collections/all' endpoint for favorites
...
(#3666 ,#3668)
2 years ago
Mike Fährmann
2bd8f2f4bd
[pornpics] add 'search' and 'tag' extractors
...
(#263 , #3544 , #3654 )
2 years ago
Mike Fährmann
79bc82884c
[pornpics] add 'gallery' extractor ( #263 , #3544 , #3654 )
2 years ago
Mike Fährmann
7bdc1d6d3d
[manganelo] update and fix metadata extraction
2 years ago
Mike Fährmann
363bb76dff
[manganelo] simplify URL pattern
2 years ago
enduser420
b28bd9789e
[redgifs] add 'collection' extractors
2 years ago
ClosedPort22
f4e211356d
[deviantart] slight refactor
2 years ago
Mike Fährmann
bd5d08abbc
[catbox] add 'file' extractor ( #3570 )
2 years ago
Mike Fährmann
8e1e8a5bea
[soundgasm] rewrite ( #3578 )
...
use a more standard extractor structure to make -A work as expected
2 years ago
Mike Fährmann
0b93420a81
[pinterest] unescape search terms ( #3621 )
2 years ago
Mike Fährmann
ad96e70546
[bunkr] fix extraction ( #3636 , #3655 )
2 years ago
Mike Fährmann
9335d55bbc
[manganelo] support mobile-only chapters
2 years ago
ClosedPort22
a74114ef7a
[deviantart] fix crash when handling deleted deviations
...
in status updates
2 years ago
Mike Fährmann
75570ad3f1
[oauth] remove stray 'exit()' ( #3628 )
...
- bug from 70ce45d9
- broke oauth:tumblr, oauth:flickr, and oauth:smugmug
2 years ago
Mike Fährmann
8fb043e8ff
[tumblr] raise more detailed errors for dashboard-only blogs
...
(#3628 )
2 years ago
Mike Fährmann
ce996dd21b
[poipiku] warn about incorrect passwords ( #3646 )
2 years ago
Mike Fährmann
70ce45d965
[oauth] use default name for browsers without 'name' attribute
...
(#3645 )
Seem to only be an issue for MacOSXOSAScript before Python 3.11.
d12bec6993
2 years ago
Mike Fährmann
2a53e6445c
[bunkr] update domain ( #3636 )
2 years ago
Mike Fährmann
5503ac4d5e
replace json.dumps with direct calls to JSONEncoder.encode
2 years ago
Mike Fährmann
dd884b02ee
replace json.loads with direct calls to JSONDecoder.decode
2 years ago
Mike Fährmann
8805bd38ab
merge #3622 : [imagetwist] add phun.imagetwist.com and imagehaha.com support
2 years ago
Mike Fährmann
706ec70e89
[imagetwist] simplify pattern and add tests
2 years ago
Mike Fährmann
f2e91732ae
[instagram] add 'user' metadata field ( #3107 )
...
at the moment only for URLs that need to translate user name to ID
2 years ago
Prinz23
29f0830b53
[imagetwist] add phun.imagetwist.com and imagehaha.com alias to imagetwist extractor
2 years ago
Mike Fährmann
bbf0911a46
[e621] implement 'notes' and 'pools' metadata extraction
...
(#3425 )
2 years ago
Mike Fährmann
925b467496
split e621 from danbooru module ( #3425 )
2 years ago
Mike Fährmann
1ae48a54f8
[twitter] add 'transform' option
2 years ago
Mike Fährmann
489c51cecc
[telegraph] fix extraction when images not in <figure> ( #3590 )
2 years ago
Mike Fährmann
0f7e6c422a
merge #3596 : [shopify] support ohpolly.com
2 years ago
enduser420
fcf7030b85
[shopify] support ohpolly.com
2 years ago
Mike Fährmann
a6a631f992
merge #3589 : [redgifs] support v3 URLs
2 years ago
Mike Fährmann
137a395ae0
[imagefap] fix infinite pagination loop ( #3594 )
2 years ago
Mike Fährmann
3c708ade8f
[imagefap] fix metadata extraction
2 years ago
Mike Fährmann
17e24eacf0
[imagefap] update 'gallery' URLs ( #3595 )
2 years ago
Mike Fährmann
c2bc70593e
implement ability to load external extractor classes
...
- -X/--extractors
- extractor.module-sources
2 years ago
enduser420
a18f627bfc
[redgifs] support v3 URLs
2 years ago
Mike Fährmann
13a90969c7
merge #3575 : [nudecollect] add 'image' and 'album' extractors
2 years ago
Mike Fährmann
aacd27e4ef
merge #3581 : [hotleak] fix video URLs
2 years ago
Mike Fährmann
abc3619feb
[lexica] add 'search' extractor ( #3567 )
2 years ago
Mike Fährmann
7c9b1ec830
[hotleak] optimize decoding video URLs
...
- use binascii module
- combine slice and reverse step
2 years ago
nifnat
f14dbfe079
Make decode_video_url static (used in both post and creator extractor).
2 years ago
nifnat
bd23a701f3
Tidy up code.
2 years ago
nifnat
7f34f99a26
Reverse engineered obfuscated JS function and reimplemented in python.
2 years ago
Mike Fährmann
0d818d3540
[fantia] send 'X-CSRF-Token' headers ( #3576 )
2 years ago
enduser420
2a5903dc16
[nudecollect] add 'image' and 'album' extractors
2 years ago
Mike Fährmann
c8fdd5096e
merge #3571 : [bunkr] Fix extracting mkv and ts files
2 years ago
Mike Fährmann
58c008e30a
[hiperdex] update domain ( #3572 )
2 years ago
Luc Ritchie
842064e597
[bunkr] Fix extracting ts files
2 years ago
Luc Ritchie
99ca0437e4
[bunkr] Fix extracting mkv files
2 years ago
Mike Fährmann
76b01b64cf
[kemonoparty] remove MD5 hash extraction ( #3531 )
...
This partially reverts commit 20d6194ffa
.
2 years ago
Mike Fährmann
09fb212414
[philomena] match URLs with www subdomain
2 years ago
Mike Fährmann
7e2fd2e573
merge #3560 : [deviantart] add support for /deviation/ and fav.me URLs
2 years ago
Mike Fährmann
caae8fefe1
merge #3541 : [deviantart] add extractor for status updates
2 years ago
ClosedPort22
c90b4ea8d9
[deviantart] add support for fav.me URLs
2 years ago
Mike Fährmann
d63af4f3d3
merge #3555 : [generic] fix regex for non-src image URLs
2 years ago
Mike Fährmann
8993b10751
[mastodon] add 'num' and 'count' metadata fields ( #3517 )
2 years ago
Mike Fährmann
d817d23ccb
[instagram] update csrf token handling
...
- update internal value according to cookie
- do not send a second 'csrftoken' cookie
2 years ago
Mike Fährmann
00b94946b3
[instagram] show -o cursor=… after every error ( #3440 )
2 years ago
ClosedPort22
674c719646
[deviantart] refactor base36 conversion
2 years ago
ClosedPort22
293abb8921
[deviantart] add support for /deviation/ URLs
2 years ago
thatfuckingbird
8cfeed78b1
[generic] fix regex for non-src image URLs
2 years ago
Mike Fährmann
fc6ea8ee5c
[instagram] update API domain and headers
2 years ago
ClosedPort22
597b89245e
[deviantart] misc improvements to status extractor
...
- relax regex pattern
- handle invalid 'items' field
- add a test for shared sta.sh item
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
Mike Fährmann
137de090dd
merge #3549 : [twitter] fix search ( #3536 )
2 years ago
Mike Fährmann
02e314c1b6
merge #3537 : [wikifeet/wikifeetx] add 'gallery' extractor
2 years ago
Mike Fährmann
568112dfbb
[oauth] improve output
...
- show which api key / client id gets used (#3518 )
- show in which browser authorization URLs gets opened in
2 years ago
ClosedPort22
ab58c375b4
[twitter] fix search ( #3536 )
...
- partially revert 18fe4b334d
- properly search for cursor when processing 'replaceEntry'
2 years ago
Mike Fährmann
df91ebb945
[oauth] simplify OAuth 1.0a init
2 years ago
ClosedPort22
013733c9e9
[deviantart] fix index fields for embedded/shared images
2 years ago
ClosedPort22
c4aeca7a5a
[deviantart] improve handling of statuses
...
- recursively yield statuses
- ignore items with missing or unexpected field(s)
2 years ago
ClosedPort22
3b32671fbd
[deviantart] add extractor for status updates
...
extract user status updates using the '/user/statuses/' endpoint
2 years ago
Mike Fährmann
107c60c973
[sankaku] update URL pattern ( #3523 )
...
match tag searches with language codes without a trailing slash
2 years ago
enduser420
5cb263fdd2
[wikifeet/wikifeetx] add 'gallery' extractor
2 years ago
Mike Fährmann
35a30498bc
merge #3531 : [kemonoparty] improve hash extraction
...
- extract md5 hashes if available
- extract discord file hashes
2 years ago
Mike Fährmann
9683d79bb7
[twitter] "fix" search pagination ( #3536 , #3534 )
...
- properly process instructions
- do not expect a predetermined instruction order
2 years ago
Mike Fährmann
4fec848858
[twitter] use "browser": "firefox" by default ( #3522 )
...
and reenable TLS 1.2 ciphers
2 years ago
Mike Fährmann
78937564fd
[twitter] fix login after 32b03433
2 years ago
ClosedPort22
20d6194ffa
[kemonoparty] improve hash extraction
...
- extract MD5 hash from URLs
- extract MD5 and SHA256 hash from Discord URLs (kemono.party only)
- minor optimization (do not call 'hashes.add' when 'duplicates' is
true)
- update tests accordingly
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
Mike Fährmann
80a2ff2d38
support setting 'write-pages' to "ALL"
...
to show authentication header, cookies, etc
2 years ago
Mike Fährmann
c881548a27
add 'extractor.retry-codes' option ( #3313 )
...
do not retry 429 and 430 by default
2 years ago
Mike Fährmann
e30e8aeef7
[mastodon] rename '_check_move' -> '_check_moved'
2 years ago
Mike Fährmann
32b0343334
[twitter] refresh guest tokens ( #3445 , #3458 )
2 years ago
Mike Fährmann
512abeb4ae
[booru] add 'url' option
2 years ago
Mike Fährmann
c87bd1a752
[danbooru] extend 'metadata' option
...
make it possible to specify a custom list of metadata includes
2 years ago
Mike Fährmann
26c3292538
[twitter] disable TLS 1.2 ciphers by default ( #3522 )
2 years ago
Mike Fährmann
18fe4b334d
[twitter] remove 'tweet_search_mode' from search parameters ( #3522 )
...
and update API root and general query parameters
2 years ago
Mike Fährmann
85bd1cbc89
[kemonoparty] fix regression from 473bd380
( #3519 )
...
- do not access 'response.content' unless necessary
- only validate responses if filename extensions differ
2 years ago
Mike Fährmann
473bd380c8
[kemonoparty] reject invalid/empty files ( #3510 )
2 years ago
Mike Fährmann
4833ec323e
[imagefap] add 'folder' extractor ( #3504 )
2 years ago
Mike Fährmann
362cd6991b
[pixiv] implement 'metadata-bookmark' option ( #3417 )
2 years ago
Mike Fährmann
2142b9c7ae
merge #3503 : [myhentaigallery] handle whitespace before title tag
2 years ago
Mike Fährmann
3a0450adbf
[behance] use default delay between requests ( #2507 )
2 years ago
Mike Fährmann
2cae4567ba
[telegraph] fix file URLs ( #3506 )
2 years ago
Mike Fährmann
cbaeee9533
[imagefap] warn about redirects to '/human-verification' ( #1140 )
2 years ago
Mike Fährmann
435de1329a
[imagefap] use default delay between requests ( #1140 )
2 years ago
Erik Rimskog
a8a982359e
[myhentaigallery] handle whitespace before the title tag
2 years ago
Mike Fährmann
d1dd52349a
merge #3189 : [tcbscans] add 'chapter' and 'manga' extractors
2 years ago
Mike Fährmann
2f31d21509
merge #3455 : [twitter] apply tweet type checks before uniqueness check
2 years ago
enduser420
e8541a131d
[tcbscans] add 'chapter' and 'manga' extractors
2 years ago
Mike Fährmann
9695c4e88d
emit debug logging message when loading cookies from file
...
attempt nr. 2
no idea how I managed to remove 6514828d
in a918ce29
2 years ago
Mike Fährmann
30a31836e7
merge #3449 : [twitter] force HTTPS for TwitPic URLs
2 years ago
Mike Fährmann
e18482e9ae
[twitter] improve 'http' -> 'https' replacement
2 years ago
Mike Fährmann
4fd6da474f
merge #3473 : [twitter] fix crash when using 'expand' and 'syndication'
2 years ago
Mike Fährmann
a918ce29b5
run tests on ubuntu-20.04
...
and remove Python 3.4, since that's no longer available
on this test runner
2 years ago
Mike Fährmann
6514828d4e
emit debug logging message when loading cookies from file
2 years ago
Mike Fährmann
3a238fd490
[poipiku] warn about login requirements
2 years ago
Mike Fährmann
f29ba089ff
merge #3474 : [fanleaks] add 'post' and 'model' extractors
2 years ago
Mike Fährmann
6933727b45
merge #3483 : [twitter] implement 'syndication=extended'
2 years ago
Mike Fährmann
07ed3a1fbf
merge #3460 : [poipiku] fix extraction for a different warning button style
...
(#3493 , #3492 )
2 years ago
Mike Fährmann
9116398c1c
[pinterest] add 'domain' option ( #3484 )
...
use input URL domain by default
2 years ago
blankie
2f985bcddb
[poipiku] fix extraction for a different warning button style
2 years ago
Mike Fährmann
294108c90a
[pinterest] support 'All Pins' boards ( #2855 , #3484 )
2 years ago
Mike Fährmann
77df8d3116
[deviantart] implement username&password login for scraps ( #1029 )
...
re-login when getting prematurely logged out by dA
is missing at the moment
2 years ago
Mike Fährmann
ed2d715019
fix 'keywords' in extractor tests ( #3491 )
2 years ago
ClosedPort22
6853b14be3
[twitter] apply suggestions from code review
...
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
Mike Fährmann
4611237f8c
merge #3457 : [danbooru] extract uploader metadata (if option is set)
2 years ago
Mike Fährmann
e7522482bb
merge #3463 : [lynxchan] support 'bbw-chan.nl'
2 years ago
Mike Fährmann
7d6c846176
[fanbox] return 'imageMap' files in order ( #2718 )
2 years ago
Mike Fährmann
dc8e7ff54e
[bunkr] fix URLs returned by API ( #3481 )
2 years ago
enduser420
5fedef3a1a
[fanleaks] update 'model' URL pattern
2 years ago
enduser420
5a740ef78b
[fanleaks] add 'post' and 'model' extractors
2 years ago
ClosedPort22
7c8eab8d52
[twitter] implement 'syndication=extended'
...
to be able to fetch extended user metadata
2 years ago
ClosedPort22
be3286206a
[twitter] assume 'conversation_id' when using syndication
...
not possible to expand replies at the momemt
2 years ago
ClosedPort22
ce8dbb1ccc
[twitter] fix crash when using 'expand' and 'syndication'
...
caused by KeyError: 'conversation_id_str'
2 years ago
ClosedPort22
38786a9593
[twitter] refactor extraction of TwitPic URLs
...
flattening
2 years ago
enduser420
527bb2c4ab
[lynxchan/bbw-chan] add 'thread' and 'board' extractors
2 years ago
blankie
f82ee93676
[danbooru] extract uploader metadata (if metadata is set)
2 years ago
ClosedPort22
250d35107c
[twitter] prioritize tweet type checks ( #3439 )
...
Do not consider a tweet seen before applying 'retweet', 'quote' and
'reply' checks. Otherwise the original tweets will also be skipped if
the "derivative" tweets and the original tweets are from the same user.
2 years ago
ClosedPort22
3eb352fcb0
[twitter] force HTTPS for TwitPic URLs
2 years ago
lx30011
895b41f1ac
[jschan] add generic jschan extractor
2 years ago
Mike Fährmann
bee354c264
Merge pull request #3415 from enduser420/extractor/fapello
...
[fapello] add 'post', 'user' and 'path' extractors
2 years ago
Mike Fährmann
8d7585534e
Merge pull request #3367 from the-blank-x/deviantart-view
...
[deviantart] add /view URL support
2 years ago
blankie
6614d94b08
[deviantart] add /view URL support
2 years ago
Mike Fährmann
dd6eeb4336
Merge pull request #3366 from ClosedPort22/da-extra-stash
...
[deviantart] extract sta.sh URLs from `text_content`
2 years ago
Mike Fährmann
f36cbb3911
Merge pull request #3413 from ClosedPort22/e621-manual-pagination
...
[e621] implement manual pagination
2 years ago
ClosedPort22
dd4a4a3fa6
[e621] softcode the pagination threshold
2 years ago
ClosedPort22
9faa4ed738
[e621] refactor pagination control
...
as suggested by @mikf
2 years ago
Mike Fährmann
7851a2c520
[seiga] raise error when redirected to login page ( #3401 )
2 years ago
Mike Fährmann
68ce5f965d
[instagram] remove unused code
2 years ago
Mike Fährmann
4063563cd7
[zerochan] update for layout v3
...
- remove cookie disabling v3
- fix and improve metadata extraction
2 years ago
Mike Fährmann
1e6407ca98
Merge pull request #3414 from pubak42/master
...
[sex.com] Download videos from cdn (#3408 )
2 years ago
ClosedPort22
bf1649dadb
[imgur] add support for imgur.io URLs
2 years ago
enduser420
7e08e2d982
[fapello] set 'filename_fmt'
2 years ago
enduser420
e5076ba056
[fapello] add 'post', 'user' and 'path' extractors
2 years ago
pubak42
e7326cdf1d
[sex.com] Download videos from cdn ( #3408 )
...
The format of video sources was changed recently to be a full URL with https:// in the beginning.
The original extractor code appended the video source URL to root url of the website, thus yielding
invalid url in format ...sex.comhttps... that failed to resolve.
2 years ago
ClosedPort22
d0ad6d0e67
[e621] implement manual pagination mode
2 years ago
Mike Fährmann
6f0735568c
[2chen] fix file URLs
2 years ago
enduser420
a2be06d873
[2chen] add '.club' support ( #3406 )
2 years ago
Mike Fährmann
a6d4733e11
[pixiv] extract 'date_url' metadata ( #3405 )
...
i.e. the datetime encoded in each file URL.
https://i.pximg.net/img-master/img/2022/12/01/13/44/55/12345678_p0.jpg
->
2022-12-01 13:44:55 +09:00
->
2022-12-01 04:44:55
2 years ago
Mike Fährmann
1317625ec4
[webmshare] add 'video' extractor ( #2410 )
2 years ago
Mike Fährmann
90a9c0790f
[twitter] update 'search' pagination ( #544 )
...
Only stop when list of all returned Tweets is empty
instead of when no valid Tweet was found.
2 years ago
Mike Fährmann
1cbc234819
[mangafox] extract more metadata ( #3167 )
2 years ago
Mike Fährmann
3082544fff
misc fixes
...
- fix typo (#3399 )
- remove double assignment
- [bunkr] update things I forgot in 6b6f886d
- [soundgasm] adjust 'archive_fmt' (#3388 )
2 years ago
enduser420
41bf236d36
[lynxchan] add generic extractors for lynxchan imageboards ( #3394 )
...
* [lynxchan] add generic extractors for lynxchan imageboards
includes kohlchan.net, endchan.org:wq
* [lynxchan] set pop default to empty tuple
* Apply suggestions from code review
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
Mike Fährmann
3c75c3bbc4
[soundgasm] add 'user' extractor ( #3384 )
...
based on code from PR #3388 by @enduser420
2 years ago
Mike Fährmann
2952add4a8
[reddit] increase 'id-max' default value ( #3397 )
...
to float("inf")
2 years ago
Mike Fährmann
a001c9c06f
[instagram] prevent post 'date' overwriting file 'date' ( #3392 )
2 years ago
Mike Fährmann
6b6f886dcf
[bunkr] update domain ( #3391 )
...
and improve bunkr/app.bunkr handling
2 years ago
ClosedPort22
bf3fd5951a
Merge branch 'master' into da-extra-stash
2 years ago
Mike Fährmann
eb94568e1f
[soundgasm] add 'audio' extractor ( #3384 )
2 years ago
Mike Fährmann
cd931e1139
update extractor test results
2 years ago
Mike Fährmann
989ec9fc79
[khinsider] fix metadata extraction
2 years ago
Mike Fährmann
1c25cc7a3e
[warosu] fix and update
2 years ago
Mike Fährmann
79e52f3539
[imgth] rewrite
...
- inherit from GalleryExtractor
- fix image URLs
- better metadata
2 years ago
Mike Fährmann
202c1210d5
[exhentai] fix pagination
2 years ago
Mike Fährmann
4a3a1f4c87
[komikcast] update domain and fix extraction
2 years ago
ClosedPort22
13d825731e
[deviantart] fix test for sta.sh URL extraction
...
Without the 'count' assertion, the test would be essentially useless.
2 years ago
ClosedPort22
6356c9be96
[deviantart] extract sta.sh URLs from 'text_content'
2 years ago
Mike Fährmann
5f57a27ba6
[imagetwist] fix extraction
2 years ago
Mike Fährmann
a42ba25ca1
[foolslide] remove 'kireicake'
...
site redirects to (unclaimed) mangadex group
2 years ago
Mike Fährmann
86f0597c95
[kissgoddess] remove module
...
site does not host albums anymore
2 years ago
Mike Fährmann
20e12b5d7c
[nitter] support '/i/user/' URLs ( #3310 )
...
as well as using 'id:<userid>' as username
not all nitter instances seem to support '/i/user/' ...
2 years ago
Mike Fährmann
fceaee3c4f
[lolisafe] remove zz.ht
2 years ago
Mike Fährmann
4554c43d5f
[bunkr] use 'media-files' servers for more file types
2 years ago
enduser420
4bc756dfe0
[2chen] fix extraction ( #3356 )
...
update 'archive_fmt'
update tests
update 'board' regex
2 years ago
enduser420
54844944ab
[pixhost] add 'gallery' support ( #3353 )
2 years ago
enduser420
213676c785
[fapachi] add 'post' and 'user' extractors ( #3347 )
...
* [fapachi] add 'post' and 'user' extractors
* [fapachi] add 'keyword' to test
* [fapachi] remove whitespaces
2 years ago
Mike Fährmann
a18511e346
[nitter] retry downloads on 404 ( #3313 )
2 years ago
Mike Fährmann
88610c3478
[patreon] update API query parameters
2 years ago
Mike Fährmann
c19b1f03b9
[patreon] fix '403 Forbidden' errors
...
send 'Content-Type' headers for API requests
2 years ago
Mike Fährmann
fc34f76cc5
[bunkr] fix video downloads ( #3326 )
...
by sending 'https://stream.bunkr.is/ ' as Referer header
2 years ago
Mike Fährmann
86a396e086
[bcy] fix JSONDecodeError ( #3321 )
2 years ago
Mike Fährmann
5b9a22af7f
[patreon] improve 'campaign_id' extraction ( #3235 )
2 years ago
Mike Fährmann
1bdd0e4338
[nitter] support '/i/web/' Tweet URLs ( #3310 )
2 years ago
Mike Fährmann
7e277d0f7d
[weibo] add 'count' metadata field ( #3305 )
...
or '{status[count]}', as most metadata for weibo is inside 'status'
2 years ago
0x1f595
19ea6ee84f
Fix 8muses album URL, add permalink path
2 years ago
0x1f595
8cbc05786a
Add 8muses album permalink parts to album data
...
This allows customizing the directory without breaking changes.
2 years ago
Mike Fährmann
4287a93202
[nitter] handle base64-encoded filenames
2 years ago
sudo
a6305d031c
[hitomi] apply format check for every image ( #3030 ) ( #3280 )
2 years ago
Steven Docherty
a7c7953107
[reddit] use 'dash_url' for videos ( #3258 ) ( #3306 )
...
* use fallback_url for reddit_video to fix issue 3258
* changed to dash_url to include audio
* update
- use [] instead of .get
- catch TypeErrors in case one of the elements is not a dict
Co-authored-by: InterruptSpeed <steven@docherty.ca>
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
Mike Fährmann
0e75358af8
[twitter] fix using user IDs for suspended accounts
2 years ago
Mike Fährmann
c25905641e
[weibo] fix bug with empty 'playback_list' ( #3301 )
2 years ago
Mike Fährmann
6cb12f513b
[nitter] support quoted Tweets
...
- distinguish between regular and quoted Tweets and media
- add 'quoted' option and metadata field
2 years ago
Mike Fährmann
aabfa7cf34
[nitter] fix direct Tweet links
2 years ago
Mike Fährmann
a41d093bb1
[nitter] add 'retweets' option ( #3278 )
2 years ago
Mike Fährmann
3d6489a4c0
[nitter] update 'user' and 'author'
2 years ago
Mike Fährmann
e99ce99284
[danbooru] remove stray 'print()'
2 years ago
Mike Fährmann
ed49e63d95
[nitter] set 'hlsPlayback' cookie
2 years ago
Mike Fährmann
e081b1fac4
[nitter] sanitize filenames ( #3294 )
2 years ago
Mike Fährmann
e31d12139c
[nitter] add 'videos' option ( #3279 )
...
with the same semantics as for twitter
2 years ago
enduser420
8c4e21b110
[itaku] remove 'Extreme' rating ( #3287 )
2 years ago
Mike Fährmann
72c5d26e85
[hotleak] fix UnboundLocalError ( #3288 , #3293 )
2 years ago
Mike Fährmann
501d9bccfe
[artstation] add 'max-posts' option ( #3270 )
2 years ago
Mike Fährmann
b1ad6f2289
[artstation] add 'pro-first' option ( #3273 )
2 years ago
Mike Fährmann
5a17e15b76
[pixiv] preserve 'tags' order ( #3266 )
...
for '"tags": "translated"'
As it turns out, set() does *not* preserve insertion order.
2 years ago
Mike Fährmann
1392b44bfe
[inkbunny] provide additional metadata ( #3274 )
...
- 'pool_id' for pools
- 'favs_user_id' for favorites
- 'search[...]' for searches
2 years ago
Mike Fährmann
a24dcbe802
[twitter] fix login ( #3220 )
...
Using an email as 'username' seems to no longer be possible,
as Twitter will always additionally ask for username or phone number
when providing an email address as 'username'.
2 years ago
Mike Fährmann
53a5d95b7d
[instagram] skip private check for avatars ( #3255 )
2 years ago
Mike Fährmann
08fd1ff835
[twitter] add 'avatar' and 'background' extractors ( #349 , #3023 )
2 years ago
Mike Fährmann
6379157543
[instagram] use REST API by default
...
regardless of logged in status
2 years ago
enduser420
7897f68225
[wallhaven] update 'user' extractor ( #3226 )
...
* [wallhaven] update 'user' extractor
* [wallhaven] update 'configuration.rst'
add 'extractor.wallhaven.include' entry
* [wallhaven] add 'wallhaven.include' in gallery-dl.conf
2 years ago
enduser420
5a68b5cb3c
[wallhaven] add 'user' extractor ( #3213 )
2 years ago
enduser420
442b03f7c3
[khinsider] fix song extraction ( #3219 )
2 years ago
Mike Fährmann
eaae4d9b65
[pixiv] stop with error for invalid search/ranking parameters
...
instead of falling back to defaults
2 years ago
Mike Fährmann
368f156378
[pixiv] rankings: add support for the new daily AI and daily AI R18
...
(#3214 , #3221 )
In remembrance of @thatfuckingbird
2 years ago
Mike Fährmann
6c153750fa
[nitter] add extractors for Nitter instances ( #2696 )
2 years ago
Mike Fährmann
9f06e79868
implement '"user-agent": "browser"' ( #2636 )
2 years ago
enduser420
ade9789b3e
[mangaread] update regex
2 years ago
enduser420
039d06c8f6
[mangaread] add 'chapter' and 'manga' extractors
2 years ago
Mike Fährmann
70c7fbe89a
[instagram] add 'guide' extractor ( #3192 )
2 years ago
enduser420
93ea8ca8e3
[imxto] extract additional metadata ( #3175 )
2 years ago
Mike Fährmann
e3abab8629
[weibo] send 'Referer' headers ( #3188 )
2 years ago
Mike Fährmann
6423f990de
[realbooru] fix 'tags' extraction ( #2530 )
2 years ago
Mike Fährmann
ecad02cf3f
[realbooru] fix download URLs ( #2530 )
2 years ago
Mike Fährmann
15cd114c9c
[twitter] update bookmarks pagination ( #3172 )
...
Do not stop when there aren't any tweets in a batch,
but only when the same cursor value appears twice in a row.
2 years ago
Mike Fährmann
20fbba9d7c
[exhentai] add metadata to search results ( #3181 )
...
'gallery_id' and 'gallery_token'
2 years ago
Mike Fährmann
6a0c5e34f4
[exhentai] fix pagination ( #3181 )
2 years ago
Mike Fährmann
171262c1b6
[instagram] remove login support
...
broken feature that I cannot get to work anymore
2 years ago
Mike Fährmann
93e6bd6847
[uploadir] use utf-8 filenames ( #3162 )
2 years ago
Mike Fährmann
b7a83ac726
[uploadir] update ( #3162 )
...
- prevent extra HTTP request from redirects
- add 'id' metadata field
- set 'filename_fmt' and 'archive_fmt'
2 years ago
Mike Fährmann
ccb80f1b8b
[uploadir] add support for 'uploadir.com' ( #3162 )
2 years ago
Mike Fährmann
b0cb4a1b9c
replace 'text.extract()' with 'text.extr()' where possible
2 years ago
Mike Fährmann
4fd3c893fa
[booru] adjust/match '_tags' and '_notes' code
2 years ago
Mike Fährmann
88954aa2e4
[gelbooru_v02] implement 'notes' extraction
...
same code as for 'moebooru' works here as well
2 years ago
ClosedPort22
4e80d3210e
[tumblr] Fallback to `gifv` when possible ( #3095 ) ( #3159 )
2 years ago
thatfuckingbird
9d3f86dbcd
[twitter] update URL for syndication API ( #3160 )
...
Twitter changed the URL format to access tweet data through their syndication API.
2 years ago
enduser420
c01cad599a
[lolisafe] add support for xbunkr ( #3156 )
2 years ago
Allen
9fc142d27b
[mastodon] add "remote_instance" field ( #3119 )
...
Example Usage:
If the url is "mastodon:https://mastodon.example.org/@VoteChess@botsin.space the "remote_instance" will be "botsin.space"
...
"directory": ["mastodon", "{remote_instance|instance}", "{account[username]!l}"]
...
2 years ago
Mike Fährmann
2a1cb403ee
Revert "[Deviantart] [ #1776 ] Remove the "you need session cookies to download mature scraps" warning ( #1777 )"
...
This reverts commit 1f02878351
.
Mature scraps do yet again require cookies.
2 years ago
Mike Fährmann
86790da2d5
update Cloudflare IUAM detection
...
again
2 years ago
Mike Fährmann
775895f44b
[booru] refactor 'tags' and 'notes' extraction
...
- move HTML request for post pages into its own function
- move gelbooru_v02.py notes extraction to gelbooru.py
since it only works there
- clean up some code
2 years ago
Luc Ritchie
0f9dfb7e62
[instagram] Fix AttributeError on user stories extraction ( #3123 )
2 years ago
Mike Fährmann
f81dd5297a
[skeb] fix extraction ( #3112 )
...
'completed_at' is no longer included in API responses
2 years ago
enduser420
fb2dbb04e2
[moebooru] extract 'notes' ( #3094 )
2 years ago
Mike Fährmann
4e26bf98f5
[aibooru] support 'safe' subdomain ( #3110 )
2 years ago
Mike Fährmann
5c31791b3c
[mastodon] support '/web/' URLs ( #3109 )
2 years ago
Mike Fährmann
9a2cfd4421
[mastodon] support cross-instance user references ( #3109 )
2 years ago
Mike Fährmann
58d97188b4
[mastodon] add 'bookmark' extractor ( #3109 )
2 years ago
Mike Fährmann
46b64251eb
[bcy] fix extraction ( #3103 )
...
- fix regex for non-watermarked images
- fetch data from '/item/detail' pages for all other posts,
since '/apiv3/user/selfPosts' only has incomplete data
2 years ago
Mike Fährmann
77173694d5
[kemonoparty] fix 'dms' extraction ( #3106 )
2 years ago
Mike Fährmann
f168ec9572
[instagram] extract 'coauthors' metadata ( #3107 )
2 years ago
Mike Fährmann
7c6af27eb8
[tumblr] add 'fallback-*' options ( #2957 )
...
specifically 'fallback-delay' and 'fallback-retries'
and change default number of retries to 2 (down from 3)
2 years ago
Mike Fährmann
4aa56d500b
[hentaihere] fix test results
2 years ago
Mike Fährmann
75d707fd92
[hentaihere] update
...
- support minor versions in chapter URLs
- fix manga metadata extraction
- update tests
2 years ago
Mike Fährmann
d2fc73f20b
[hentai2read] fix manga metadata extraction
...
and update tests
2 years ago
Mike Fährmann
f4d06e5180
[manganelo] update domain to 'chapmanganato.com' ( #3097 )
2 years ago
Mike Fährmann
769e6754dc
[pixiv] use 'exact_match_for_tags' as default search mode ( #3092 )
2 years ago
Mike Fährmann
a90e5cb354
[instagram] support 'instagram.com/s/' highlight URLs ( #3076 )
2 years ago
enduser420
fd19c4b228
[hentai2read] recognize '.' in chapter ( #3089 )
2 years ago
enduser420
2ff1897421
[vichan] recognize board url w/o trailing slash ( #3087 )
2 years ago
enduser420
ac6111e693
[mangasee] add support for 'mangalife' ( #3086 )
2 years ago
KJ16609
300bc03deb
[gelbooru] allow alternate parameter order in post URLs ( #2821 )
2 years ago
Mike Fährmann
a7d23f1484
[vichan] add generic extractors for vichan imageboards
...
includes 8kun.top, smuglo.li, and wikieat.club
2 years ago
Mike Fährmann
04d3ebdfb4
[redgifs] fix 'token' extraction ( #3080 , #3081 )
2 years ago
thatfuckingbird
062ef238a6
add support for aibooru (using danbooru extractor) ( #3075 )
2 years ago
enduser420
0163ca86f7
[smugloli] add smugloli extractors ( #3060 )
2 years ago
Mike Fährmann
cf86f68864
[instagram] add 'avatar' extractor ( #929 , #1097 , #2992 )
2 years ago
Mike Fährmann
ea8113ff36
[reactor] match 'best', 'new', 'all' URLs ( #3073 )
2 years ago
Mike Fährmann
618c81afdf
[ngomik] remove module
...
"Access denied"
2 years ago
Mike Fährmann
94a2dfe205
[kemonoparty] update pagination offset
2 years ago
Mike Fährmann
52d1eb928d
[pixiv] extend 'metadata' option ( #3057 )
...
make it usable for all 'pixiv' extractors
2 years ago
Mike Fährmann
0714274f1f
[instagram] remove 'channel' extractor
2 years ago
Mike Fährmann
d0d4ce1a13
[danbooru] fix ugoira metadata extraction ( #3056 )
2 years ago
Mike Fährmann
096b8f2cfc
[instagram] prevent request for private '/tagged' feeds ( #3045 )
2 years ago
Mike Fährmann
3b369ce3d1
[nijie] add 'followed' extractor ( #3048 )
2 years ago
Mike Fährmann
c4a62a48ae
[nijie] add 'feed' extractor ( #3048 )
2 years ago
Mike Fährmann
d1314df6e6
[nozomi] fix extraction ( #3051 )
2 years ago
Mike Fährmann
277be410a7
[2chen] update 'archive_fmt'
2 years ago
Mike Fährmann
ed55bd3a5c
[redgifs] extract Bearer token ( #3037 )
2 years ago
Mike Fährmann
e974c75083
[redgifs] fix extraction ( #3037 )
...
send public Bearer token as 'authorization' header
2 years ago
Mike Fährmann
68466a7d61
[tumblr] support ' https://www.tumblr.com/BLOGNAME ' URLs ( #3034 )
2 years ago
Mike Fährmann
b6a68f5a4b
[fanbox] extend 'content' test result ( #3020 )
2 years ago
Mike Fährmann
f1f89b2436
[tumblr] add 'offset' option
2 years ago
Mike Fährmann
827ab0a62d
[instagram] fix login
...
- use mobile user agent header
- update general headers
- skip /data/shared_data/ step
2 years ago
Mike Fährmann
1ca6be8619
[fanbox] add 'content' metadata field ( #3020 )
2 years ago
Mike Fährmann
e5d229c524
[tumblr] sleep between fallback retries ( #2957 )
2 years ago
Mike Fährmann
b2b0b1c455
[hitomi] fall back to webp when format not available ( #3030 )
2 years ago
Mike Fährmann
1696f68a68
[8chan] add 'thread' and 'board' extractors ( #2938 )
2 years ago
Mike Fährmann
560f7b41d8
[vk] add 'tagged' extractor ( #2997 )
2 years ago
Mike Fährmann
122e1a467a
[vk] unescape error messages
2 years ago
Mike Fährmann
bc9d291c13
[imagefap] fix and improve folder extraction ( #3013 )
2 years ago
Mike Fährmann
55fca5fe4b
[imagefap] fix and improve gallery pagination ( #3013 )
2 years ago
Mike Fährmann
8b1fe0bcf1
emit debug logging messages before calling time.sleep() ( #2982 )
2 years ago
Mike Fährmann
14717f3fc9
[deviantart] add 'group' option ( #3018 )
...
disabling this option allows to better download from deleted accounts
2 years ago
Mike Fährmann
220a04a74a
[artstation] skip missing projects ( #3016 )
2 years ago
Mike Fährmann
a12ce2bb41
[deviantart] fix 'deviation' extraction ( #2981 )
2 years ago
Mike Fährmann
36afb519b3
[instagram] prevent crash on empty user profile
2 years ago
enduser420
f0321f423d
[2chen] Add 2chen.moe extractor ( #2707 )
...
* [2chen] Add 2chen.moe extractor
* change "==" to is
* fix for "test_unique_pattern_matches"
* fix regex pattern and group matching
* fix regex again
* [2chen] add 'reply_no' and 'hash' metadata and change 'filename_fmt'
also made an entry in supportedsites.md
* [2chen] unescape 'title'
* [2chen] partition() -> rpartition()
* [2chen] extract 'date' and 'name' metadata
* [2chen] remove 'offset' argument
* [2chen] do some changes
* [2chen] do some more changes
* [2chen] unescape 'name' and 'filename'
2 years ago
enduser420
f7ba19a1c0
[nana] add 'nana' extractors ( #2967 )
2 years ago
Mike Fährmann
fce6642699
[instagram] restore warnings for private profiles ( #3004 )
2 years ago
Mike Fährmann
3e65645cfa
[instagram] restore 'cursor' functionality ( #2991 )
2 years ago
Mike Fährmann
b8d268f57e
allow '/' and '?' in URL queries
2 years ago
Mike Fährmann
7b5dad075d
[fappic] fix extraction
2 years ago
Mike Fährmann
78694a61bb
[kemonoparty] restore 'favorites' API endpoints ( #2994 )
2 years ago
Mike Fährmann
5fd4374036
[sankaku] improve 429 and tag limit handling
2 years ago
Mike Fährmann
b84982b2f9
[kemonoparty] send Referer headers ( #2989 , #2990 )
2 years ago
blankie
98f67ae333
[instagram] add 'count' metadata field ( #2979 )
2 years ago
Mike Fährmann
4089bceddd
[sankaku] implement 'refresh' option ( #2958 )
2 years ago
Mike Fährmann
779e75c6f8
[kemonoparty] fix attachment IDs overwriting post IDs ( #2984 )
...
regression from 09a5cc61
2 years ago
Mike Fährmann
e1d714943b
[tumblr] catch exception when updating image token ( #2957 )
2 years ago
Mike Fährmann
e3a03f335c
[instagram] fix GraphQL bugs
2 years ago
Mike Fährmann
6c76b5f90c
[deviantart] fix extraction ( #2981 , #2983 )
...
send a 'csrf_token' with every Eclipse API request
2 years ago
Mike Fährmann
f728b5ca06
[tumblr] add fallback for failed higher-resolution images ( #2957 )
2 years ago
Mike Fährmann
6992d01e19
[artstation] support search filters ( #2970 )
2 years ago
Mike Fährmann
194803f3a7
[plurk] fix extraction ( #2977 )
2 years ago
Mike Fährmann
63e0924927
[pixiv] add 'series' extractor ( #2964 )
2 years ago
Mike Fährmann
aafea0c4f8
[artstation] fix searches ( #2970 )
2 years ago
Mike Fährmann
2c67bee5c4
[instagram] update
...
- reorder some functions and extractors
- add missing GraphQL endpoints
- fix some GraphQL bugs
2 years ago
Mike Fährmann
aa49bf13d2
[instagram] add 'api' option
2 years ago
Mike Fährmann
6f77193a24
[instagram] move API related code into separate classes
...
may contain bugs and is probably incomplete for the GraphQL variant
2 years ago
Mike Fährmann
ac45ed2764
[skeb] implement 'filters' option ( #2945 )
2 years ago
Mike Fährmann
32c30754d1
[tumblr] warn when unable to fetch higher-resolution images ( #2957 )
...
and download the smaller version
instead of failing with a 404 error
2 years ago
Mike Fährmann
ff532d6c3c
[newgrounds] extract 'type' metadata
2 years ago
Mike Fährmann
0393e59535
[newgrounds] add 'games' extractor ( #2955 )
2 years ago
Mike Fährmann
68f11e02a9
[skeb] add 'search_tags' metadata to search results ( #2945 )
2 years ago
Mike Fährmann
1378cbb8dd
[myportfolio] use fallback when no images are found ( #2959 )
2 years ago
Mike Fährmann
850608551c
[sankaku] detect expired links ( #2958 )
2 years ago
Mike Fährmann
09a5cc6103
[kemonoparty] add 'count' metadata field ( #2952 )
2 years ago
Mike Fährmann
89610a49dc
[instagram] use REST API endpoint for user feeds ( #2666 )
...
With this change, everything is using the newer REST API endpoints
providing higher-quality photos except the now obsolete '/channel' feed.
2 years ago
Mike Fährmann
6737499dbd
[instagram] use REST API endpoint for saved posts ( #2911 )
...
provides 'username' and 'fullname'
as well as higher-quality images
2 years ago
Mike Fährmann
50e3179c56
[instagram] update _user_by_screen_name()
...
use REST API
2 years ago
Mike Fährmann
3dacfb3c56
[instagram] update API headers
2 years ago
Mike Fährmann
4b2a006871
[skeb] add 'search' extractor ( #2945 )
2 years ago
Mike Fährmann
94b34f460e
[exhentai] add slash to the end of gallery URLs ( #2947 )
2 years ago
Mike Fährmann
2787c8511a
[mastodon] warn about moved accounts ( #2939 )
2 years ago
Mike Fährmann
d699310fdf
[blogger] add 'label' or 'query' metadata fields ( #2930 )
...
for '/search/label/…' or '/search?q=…' URLs
2 years ago
Mike Fährmann
eef50c1f28
[blogger] split 'search' extractor ( #2930 )
2 years ago
Mike Fährmann
d29fb94098
[bunkr] use 'media-files' servers for m4v and mov files ( #2925 )
2 years ago
enduser420
bd846abba0
[hotleak] add hotleak extractor ( #2909 ) ( #2890 )
2 years ago
Mike Fährmann
e99a9b2aff
[twitter] improve 'cards-blacklist' ( #2875 )
...
allow blacklisting domains and 'name:domain',
where 'domain' depends on a card's 'vanity_url' value
2 years ago
Mike Fährmann
aaf6992bae
[twitter] fix new-style '/card_img/' URLs
2 years ago
Mike Fährmann
40baa77630
[twitter] provide proper 'date' for syndication results ( #2920 )
2 years ago
Mike Fährmann
46fe469c53
[tumblr] implement 'ratelimit' option ( #2919 )
2 years ago
Mike Fährmann
d0b73fec14
[flickr] add support for secure.flickr.com ( #2910 )
2 years ago
Mike Fährmann
35eddaa94e
[reddit] prevent exception with empty submission URLs ( #2913 )
2 years ago
Mike Fährmann
464ea90d14
[exhentai] guess extension for original files ( #2842 )
...
makes it possible to sometimes, when guessed correctly ('.jpg'),
skip an original file download without costing image limit points
2 years ago
Mike Fährmann
551fdf7ad7
[exhentai] move 509 check into its own function
2 years ago
Mike Fährmann
7a799df17f
[tumblr] pre-compile regular expressions
2 years ago
Mike Fährmann
73a52a95b0
update Cloudflare IUAM detection
2 years ago
Mike Fährmann
673b6f1218
[poipiku] use 'img-org.poipiku.com' as image domain ( #2796 )
2 years ago
Mike Fährmann
4ca1a6e5f3
[bunkr] fix extraction ( #2903 )
2 years ago
Mike Fährmann
8b76149521
[exhentai] improve 509.gif detection ( #2901 )
2 years ago
Mike Fährmann
2ed58029f9
{paheal[ add proper support for videos ( #2892 )
2 years ago
Mike Fährmann
444dfb4aa6
[instagram] add 'highlight_title' and 'date' metadata
...
to highlight posts (#2879 )
2 years ago
Mike Fährmann
7f764ebee6
[redgifs] "fix" download URLs ( #2884 )
2 years ago
Mike Fährmann
3cb8327c60
[zerochan] add 'metadata' option ( #2861 )
2 years ago
blankie
9745b48830
[tumblr] attempt to fetch high-quality inline images ( #2877 )
...
* [tumblr] attempt to fetch high-quality images (again)
Fixes #1846 , and fixes #1344
* slight refactor
* update configuration.rst entry
2 years ago
Mike Fährmann
daef91c925
[smugmug] update default API credentials ( #2881 )
...
The old key lacked v2 access and I'm unable to accept
the new terms of service since my old account got deleted
2 years ago
Mike Fährmann
4d78ca89db
[twitter] add 'cards-blacklist' option ( #2875 )
2 years ago
Mike Fährmann
4d7cb0bf56
[twitter] general support for unified cards ( #2875 )
...
just removing the 'type' check seems to work
2 years ago
Mike Fährmann
7ddfff957c
[twitter] support "image_website" unified cards ( #2875 )
2 years ago
Mike Fährmann
2eb0ddd083
[hitomi] fix error when number of tag results is multiple of 25
...
(#2870 )
2 years ago
Mike Fährmann
3cebf787c4
[slideshare] fix metadata extraction
2 years ago
Mike Fährmann
da11fb32d0
update extractor test results
2 years ago
Mike Fährmann
636d03df95
[nijie] reduce cache maxage to 90 days
2 years ago
Mike Fährmann
f375ec0ffa
[vsco] fix 'collection' extraction
2 years ago
Mike Fährmann
8672f8a2b9
[skeb] fix archive_ids for thumbnails and article images
...
8cf5981ded (commitcomment-82316040)
2 years ago
Mike Fährmann
69995d789b
Revert "[twitter] use '{author[name]' in default directory names"
...
This reverts commit 9ad3cdc5d8
.
2 years ago
Mike Fährmann
946643c23c
[hitomi] use maxage for gg.js cache ( #2863 )
...
cached values become invalid after 1-2 hours
2 years ago
Mike Fährmann
d508b2c049
[gelbooru] implement 'pool' pagination ( #2853 )
2 years ago
Mike Fährmann
67a2efb885
[rule34] implement 'pool' pagination ( #2853 )
2 years ago
Mike Fährmann
70dc4ce911
[skeb] ignore article images with empty URL
...
8cf5981ded (commitcomment-81980633)
2 years ago
Mike Fährmann
f362d4a3c7
[e621] fix 'popular' extraction
2 years ago
Mike Fährmann
7e385ed63e
[foolfuuka] update domains
...
- remove nyafuu
- add rozenarcana (https://archive.alice.al/ )
- add tokyochronos (https://www.tokyochronos.net )
2 years ago
Mike Fährmann
6ba72b6bc6
[twitter] ignore invalid user entries ( #2850 )
2 years ago
enduser420
3d87cedc58
[jpgchurch] rework the image extractor
...
now the image extractor can recognize if an image if from an album
also removed some unnecessary methods
2 years ago
blankie
e4cff67aaa
[tumblr] add count metadata field ( #2804 )
...
Fixes #2778
2 years ago
enduser420
574e38a287
[kemonoparty] add 'favorites' option ( #2826 ) ( #2831 )
...
* [kemonoparty] add 'favorites' option (#2826 )
* [kemonoparty] add regex for the url parameter and fallback on the config
option
* [kemonoparty] simplify
2 years ago
Mike Fährmann
a799fae2df
[catbox] add 'album' extractor ( #2410 )
...
adapted from https://github.com/mikf/gallery-dl/pull/2805
- rewrite using GalleryExtractor
- extract more metadata
- match lolisafe names
- add test
2 years ago
Mike Fährmann
264f1336ad
[twitter] unescape '+' in search queries ( #2226 )
...
... and do not raise exception if searched user does not exist
2 years ago
Mike Fährmann
21ff77fea0
[zerochan] extract more metadata for single posts
...
Neither HTML pages nor RSS feed entries have *all* metadata.
It might be necessary to do 1-2 extra HTTP requests to grab everything.
2 years ago
Mike Fährmann
391aecf219
[instagram] provide 'date' for directories ( #2830 )
2 years ago