Mike Fährmann
e480a93337
add 'output.stdout', '.stdin', and '.stderr' options
...
(#1621 , #2152 , #2529 )
Allow setting custom input/output encodings and options
without having to rely on Python's defaults.
2 years ago
Mike Fährmann
6ed4309aba
[deviantart] add 'gallery-search' extractor ( #1695 )
2 years ago
Mike Fährmann
e1df7f73b1
[deviantart] add 'search' extractor
...
(#538 , #1264 , #2954 , #2970 , #3577 )
Requires login to fetch any results, since the API endpoint raises an
error for not logged in requests.
TODO: parse HTML search results
2 years ago
Mike Fährmann
bb11c2a576
merge #3662 : [redgifs] add 'collection' extractors
2 years ago
Mike Fährmann
884f1848d6
[redgifs] fix syntax for older Python versions
...
and update docs/supportedsites
2 years ago
Mike Fährmann
2bd8f2f4bd
[pornpics] add 'search' and 'tag' extractors
...
(#263 , #3544 , #3654 )
2 years ago
Mike Fährmann
79bc82884c
[pornpics] add 'gallery' extractor ( #263 , #3544 , #3654 )
2 years ago
Mike Fährmann
bd5d08abbc
[catbox] add 'file' extractor ( #3570 )
2 years ago
Mike Fährmann
2a53e6445c
[bunkr] update domain ( #3636 )
2 years ago
Mike Fährmann
b7337d810e
[postprocessor:metadata] add 'sort' and 'separators' options
2 years ago
Mike Fährmann
762a68996b
implement 'archive-pragma' option
2 years ago
Mike Fährmann
bbf0911a46
[e621] implement 'notes' and 'pools' metadata extraction
...
(#3425 )
2 years ago
Mike Fährmann
925b467496
split e621 from danbooru module ( #3425 )
2 years ago
Mike Fährmann
1ae48a54f8
[twitter] add 'transform' option
2 years ago
Mike Fährmann
78d3960a31
[postprocessor:exec] implement archive options ( #3584 )
2 years ago
Mike Fährmann
0f7e6c422a
merge #3596 : [shopify] support ohpolly.com
2 years ago
enduser420
fcf7030b85
[shopify] support ohpolly.com
2 years ago
Mike Fährmann
4572c122d8
document 'extractor.module-sources'
2 years ago
Mike Fährmann
c2bc70593e
implement ability to load external extractor classes
...
- -X/--extractors
- extractor.module-sources
2 years ago
Mike Fährmann
13a90969c7
merge #3575 : [nudecollect] add 'image' and 'album' extractors
2 years ago
Mike Fährmann
abc3619feb
[lexica] add 'search' extractor ( #3567 )
2 years ago
Mike Fährmann
f58215705a
add '-O/--postprocessor-option' command-line option ( #3565 )
2 years ago
enduser420
2a5903dc16
[nudecollect] add 'image' and 'album' extractors
2 years ago
Mike Fährmann
58c008e30a
[hiperdex] update domain ( #3572 )
2 years ago
Mike Fährmann
caae8fefe1
merge #3541 : [deviantart] add extractor for status updates
2 years ago
ClosedPort22
3b32671fbd
[deviantart] add extractor for status updates
...
extract user status updates using the '/user/statuses/' endpoint
2 years ago
enduser420
5cb263fdd2
[wikifeet/wikifeetx] add 'gallery' extractor
2 years ago
Mike Fährmann
4e86aaa4db
document 'extractor.headers' and 'extractor.ciphers'
2 years ago
Mike Fährmann
c881548a27
add 'extractor.retry-codes' option ( #3313 )
...
do not retry 429 and 430 by default
2 years ago
Mike Fährmann
512abeb4ae
[booru] add 'url' option
2 years ago
Mike Fährmann
c87bd1a752
[danbooru] extend 'metadata' option
...
make it possible to specify a custom list of metadata includes
2 years ago
Mike Fährmann
c0d7d2be35
[downloader:http] add 'validate' option
2 years ago
Mike Fährmann
4833ec323e
[imagefap] add 'folder' extractor ( #3504 )
2 years ago
Mike Fährmann
cb9d100bac
update Authentication support in docs
...
https://github.com/mikf/gallery-dl/pull/3075#issuecomment-1374723412
2 years ago
Mike Fährmann
362cd6991b
[pixiv] implement 'metadata-bookmark' option ( #3417 )
2 years ago
Mike Fährmann
0895e6afee
merge #3462 : [docs] Update links and fix field typo
2 years ago
Mike Fährmann
4b9bb632f2
[docs] update links
2 years ago
Mike Fährmann
d1dd52349a
merge #3189 : [tcbscans] add 'chapter' and 'manga' extractors
2 years ago
Mike Fährmann
ac78712618
merge #3496 : [docs] update OAuth guide in README.rst
2 years ago
Mike Fährmann
2f31d21509
merge #3455 : [twitter] apply tweet type checks before uniqueness check
2 years ago
Mike Fährmann
64435cc37e
[twitter] update docs
2 years ago
Mike Fährmann
b125fcafc5
update docs/configuration.rst
...
- document metadata.indent
- add concrete mappings to object types
- fix lists for code examples
- fix typos
2 years ago
enduser420
e8541a131d
[tcbscans] add 'chapter' and 'manga' extractors
2 years ago
Mike Fährmann
3c03928d75
simplify 'docs/options.md' generation
...
use the plain text output from '--help'
instead of trying to generate 'fancy' HTML with Markdown restrictions
2 years ago
ClosedPort22
c94692f39b
[docs] document 'access-token' for Mastodon instances
2 years ago
Mike Fährmann
4fd6da474f
merge #3473 : [twitter] fix crash when using 'expand' and 'syndication'
2 years ago
Mike Fährmann
818f6fa71f
merge #3453 : [twitter] document #1396 and #2712
2 years ago
ClosedPort22
455481ad76
[twitter] improve documentation
...
This includes:
- The automatic expansion of conversations (#2712 )
- How to retrieve age-restricted content from search results (#1396 )
2 years ago
Mike Fährmann
f29ba089ff
merge #3474 : [fanleaks] add 'post' and 'model' extractors
2 years ago
Mike Fährmann
6933727b45
merge #3483 : [twitter] implement 'syndication=extended'
2 years ago
Mike Fährmann
9116398c1c
[pinterest] add 'domain' option ( #3484 )
...
use input URL domain by default
2 years ago
Mike Fährmann
f1a715dbcd
improve docs/options.md
...
- use non-breaking hyphens
- don't replace '…' in examples
- don't force width for long option column
2 years ago
Mike Fährmann
294108c90a
[pinterest] support 'All Pins' boards ( #2855 , #3484 )
2 years ago
Mike Fährmann
13c16490bd
handle subcategory names ending with 'y'
2 years ago
Mike Fährmann
50d89d4acb
docs/options.md: use a separate table for each option group
2 years ago
Mike Fährmann
7de6b7f489
add list of command-line options as docs/options.md ( #3464 )
2 years ago
ClosedPort22
6853b14be3
[twitter] apply suggestions from code review
...
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
Mike Fährmann
4611237f8c
merge #3457 : [danbooru] extract uploader metadata (if option is set)
2 years ago
Mike Fährmann
e7522482bb
merge #3463 : [lynxchan] support 'bbw-chan.nl'
2 years ago
Mike Fährmann
0ed5986062
document custom 'output.mode' ( #2529 )
2 years ago
enduser420
5a740ef78b
[fanleaks] add 'post' and 'model' extractors
2 years ago
ClosedPort22
7c8eab8d52
[twitter] implement 'syndication=extended'
...
to be able to fetch extended user metadata
2 years ago
ClosedPort22
be3286206a
[twitter] assume 'conversation_id' when using syndication
...
not possible to expand replies at the momemt
2 years ago
ClosedPort22
ce8dbb1ccc
[twitter] fix crash when using 'expand' and 'syndication'
...
caused by KeyError: 'conversation_id_str'
2 years ago
Mike Fährmann
d651d45239
implement specifying ranges in slice notation ( #918 , #2865 )
...
e.g.
- '1:101' or ':101' or ':101:' for files 1 to 100
- '1::2' or '::2' for every second file
- '1:101:5' or ':101:5' for files 1, 6, 11, ..., 91, 96
(the second argument specifies the first index NOT included)
2 years ago
enduser420
527bb2c4ab
[lynxchan/bbw-chan] add 'thread' and 'board' extractors
2 years ago
pi_allen
64902f518e
[docs] Update links and fix field typo
2 years ago
ClosedPort22
5d5ea58887
[twitter] update docs for 'quoted' & 'unique'
2 years ago
blankie
f82ee93676
[danbooru] extract uploader metadata (if metadata is set)
2 years ago
Mike Fährmann
1800bd7d14
allow '*-filter' options to be a list of expressions
2 years ago
lx30011
895b41f1ac
[jschan] add generic jschan extractor
2 years ago
Mike Fährmann
73ab5d84c0
update docs/configuration.rst
2 years ago
Mike Fährmann
bee354c264
Merge pull request #3415 from enduser420/extractor/fapello
...
[fapello] add 'post', 'user' and 'path' extractors
2 years ago
ClosedPort22
dd4a4a3fa6
[e621] softcode the pagination threshold
2 years ago
enduser420
e5076ba056
[fapello] add 'post', 'user' and 'path' extractors
2 years ago
ClosedPort22
d0ad6d0e67
[e621] implement manual pagination mode
2 years ago
Mike Fährmann
1317625ec4
[webmshare] add 'video' extractor ( #2410 )
2 years ago
enduser420
41bf236d36
[lynxchan] add generic extractors for lynxchan imageboards ( #3394 )
...
* [lynxchan] add generic extractors for lynxchan imageboards
includes kohlchan.net, endchan.org:wq
* [lynxchan] set pop default to empty tuple
* Apply suggestions from code review
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
Mike Fährmann
3c75c3bbc4
[soundgasm] add 'user' extractor ( #3384 )
...
based on code from PR #3388 by @enduser420
2 years ago
Mike Fährmann
2952add4a8
[reddit] increase 'id-max' default value ( #3397 )
...
to float("inf")
2 years ago
Mike Fährmann
6b6f886dcf
[bunkr] update domain ( #3391 )
...
and improve bunkr/app.bunkr handling
2 years ago
Mike Fährmann
eb94568e1f
[soundgasm] add 'audio' extractor ( #3384 )
2 years ago
Mike Fährmann
dfe7b23579
support Firefox containers for --cookies-from-browser ( #3346 )
2 years ago
Mike Fährmann
4a3a1f4c87
[komikcast] update domain and fix extraction
2 years ago
Mike Fährmann
a42ba25ca1
[foolslide] remove 'kireicake'
...
site redirects to (unclaimed) mangadex group
2 years ago
Mike Fährmann
86f0597c95
[kissgoddess] remove module
...
site does not host albums anymore
2 years ago
Mike Fährmann
fceaee3c4f
[lolisafe] remove zz.ht
2 years ago
enduser420
54844944ab
[pixhost] add 'gallery' support ( #3353 )
2 years ago
enduser420
213676c785
[fapachi] add 'post' and 'user' extractors ( #3347 )
...
* [fapachi] add 'post' and 'user' extractors
* [fapachi] add 'keyword' to test
* [fapachi] remove whitespaces
2 years ago
Mike Fährmann
80102fa367
[downloader:http] add 'retry-codes' option ( #3313 )
2 years ago
ClosedPort22
b14b33f19e
Implement `version-metadata` option ( #3201 )
2 years ago
Mike Fährmann
6cb12f513b
[nitter] support quoted Tweets
...
- distinguish between regular and quoted Tweets and media
- add 'quoted' option and metadata field
2 years ago
Mike Fährmann
a41d093bb1
[nitter] add 'retweets' option ( #3278 )
2 years ago
Mike Fährmann
e31d12139c
[nitter] add 'videos' option ( #3279 )
...
with the same semantics as for twitter
2 years ago
Mike Fährmann
501d9bccfe
[artstation] add 'max-posts' option ( #3270 )
2 years ago
Mike Fährmann
b1ad6f2289
[artstation] add 'pro-first' option ( #3273 )
2 years ago
Mike Fährmann
42481aed59
[formatter] implement 'S' format specifier ( #3266 )
...
to Sort lists
2 years ago
Mike Fährmann
6b97dcf2e0
[postprocessor:metadata] add 'private' option
2 years ago
Mike Fährmann
08fd1ff835
[twitter] add 'avatar' and 'background' extractors ( #349 , #3023 )
2 years ago
Mike Fährmann
6379157543
[instagram] use REST API by default
...
regardless of logged in status
2 years ago
enduser420
7897f68225
[wallhaven] update 'user' extractor ( #3226 )
...
* [wallhaven] update 'user' extractor
* [wallhaven] update 'configuration.rst'
add 'extractor.wallhaven.include' entry
* [wallhaven] add 'wallhaven.include' in gallery-dl.conf
2 years ago
enduser420
5a68b5cb3c
[wallhaven] add 'user' extractor ( #3213 )
2 years ago
Mike Fährmann
6c153750fa
[nitter] add extractors for Nitter instances ( #2696 )
2 years ago
Mike Fährmann
9f06e79868
implement '"user-agent": "browser"' ( #2636 )
2 years ago
enduser420
039d06c8f6
[mangaread] add 'chapter' and 'manga' extractors
2 years ago
Mike Fährmann
70c7fbe89a
[instagram] add 'guide' extractor ( #3192 )
2 years ago
Mike Fährmann
8d543b040d
update docs
...
- document '"mode": "jsonl"' for metadata pp
- remove 'username' and 'password' from 'instagram'
- add 'ytdl' example (#3180 )
2 years ago
Mike Fährmann
171262c1b6
[instagram] remove login support
...
broken feature that I cannot get to work anymore
2 years ago
Mike Fährmann
895f36e53b
[postprocessor:metadata] add 'open' and 'encoding' options
2 years ago
Mike Fährmann
870e6a48a0
implement 'http-metadata' option
...
or at least attempt to.
2 years ago
Mike Fährmann
ccb80f1b8b
[uploadir] add support for 'uploadir.com' ( #3162 )
2 years ago
enduser420
c01cad599a
[lolisafe] add support for xbunkr ( #3156 )
2 years ago
Mike Fährmann
bca9f965e5
[downloader:http] add 'chunk-size' option ( #3143 )
...
and double the previous default from 16384 (2**14) to 32768 (2**15)
2 years ago
Mike Fährmann
c12a97bcde
[postprocessor] add 'post-after' event ( #3117 )
2 years ago
Mike Fährmann
58d97188b4
[mastodon] add 'bookmark' extractor ( #3109 )
2 years ago
Mike Fährmann
7c6af27eb8
[tumblr] add 'fallback-*' options ( #2957 )
...
specifically 'fallback-delay' and 'fallback-retries'
and change default number of retries to 2 (down from 3)
2 years ago
enduser420
ac6111e693
[mangasee] add support for 'mangalife' ( #3086 )
2 years ago
Mike Fährmann
a7d23f1484
[vichan] add generic extractors for vichan imageboards
...
includes 8kun.top, smuglo.li, and wikieat.club
2 years ago
thatfuckingbird
062ef238a6
add support for aibooru (using danbooru extractor) ( #3075 )
2 years ago
enduser420
0163ca86f7
[smugloli] add smugloli extractors ( #3060 )
2 years ago
Mike Fährmann
cf86f68864
[instagram] add 'avatar' extractor ( #929 , #1097 , #2992 )
2 years ago
Mike Fährmann
618c81afdf
[ngomik] remove module
...
"Access denied"
2 years ago
Mike Fährmann
52d1eb928d
[pixiv] extend 'metadata' option ( #3057 )
...
make it usable for all 'pixiv' extractors
2 years ago
Mike Fährmann
0714274f1f
[instagram] remove 'channel' extractor
2 years ago
Mike Fährmann
3b369ce3d1
[nijie] add 'followed' extractor ( #3048 )
2 years ago
Mike Fährmann
c4a62a48ae
[nijie] add 'feed' extractor ( #3048 )
2 years ago
Mike Fährmann
f1f89b2436
[tumblr] add 'offset' option
2 years ago
Mike Fährmann
15a4d7696f
[docs] update 'adjust-extensions' explanation ( #3031 )
2 years ago
Mike Fährmann
1696f68a68
[8chan] add 'thread' and 'board' extractors ( #2938 )
2 years ago
Mike Fährmann
560f7b41d8
[vk] add 'tagged' extractor ( #2997 )
2 years ago
Mike Fährmann
7f30a0d7a7
add 'path-extended' option ( #3021 )
2 years ago
Mike Fährmann
14717f3fc9
[deviantart] add 'group' option ( #3018 )
...
disabling this option allows to better download from deleted accounts
2 years ago
enduser420
f0321f423d
[2chen] Add 2chen.moe extractor ( #2707 )
...
* [2chen] Add 2chen.moe extractor
* change "==" to is
* fix for "test_unique_pattern_matches"
* fix regex pattern and group matching
* fix regex again
* [2chen] add 'reply_no' and 'hash' metadata and change 'filename_fmt'
also made an entry in supportedsites.md
* [2chen] unescape 'title'
* [2chen] partition() -> rpartition()
* [2chen] extract 'date' and 'name' metadata
* [2chen] remove 'offset' argument
* [2chen] do some changes
* [2chen] do some more changes
* [2chen] unescape 'name' and 'filename'
2 years ago
enduser420
f7ba19a1c0
[nana] add 'nana' extractors ( #2967 )
2 years ago
Mike Fährmann
4089bceddd
[sankaku] implement 'refresh' option ( #2958 )
2 years ago
Mike Fährmann
63e0924927
[pixiv] add 'series' extractor ( #2964 )
2 years ago
Mike Fährmann
aa49bf13d2
[instagram] add 'api' option
2 years ago
Mike Fährmann
ac45ed2764
[skeb] implement 'filters' option ( #2945 )
2 years ago
Mike Fährmann
08b0c4c0b1
[docs] add kemono text content example ( #2963 )
2 years ago
Mike Fährmann
0393e59535
[newgrounds] add 'games' extractor ( #2955 )
2 years ago
Mike Fährmann
4b2a006871
[skeb] add 'search' extractor ( #2945 )
2 years ago
Mike Fährmann
eef50c1f28
[blogger] split 'search' extractor ( #2930 )
2 years ago
enduser420
bd846abba0
[hotleak] add hotleak extractor ( #2909 ) ( #2890 )
2 years ago
Mike Fährmann
e99a9b2aff
[twitter] improve 'cards-blacklist' ( #2875 )
...
allow blacklisting domains and 'name:domain',
where 'domain' depends on a card's 'vanity_url' value
2 years ago
Mike Fährmann
46fe469c53
[tumblr] implement 'ratelimit' option ( #2919 )
2 years ago
Mike Fährmann
b36125333f
[postprocessor:zip] implement 'files' option ( #2872 )
2 years ago
Mike Fährmann
3cb8327c60
[zerochan] add 'metadata' option ( #2861 )
2 years ago
blankie
9745b48830
[tumblr] attempt to fetch high-quality inline images ( #2877 )
...
* [tumblr] attempt to fetch high-quality images (again)
Fixes #1846 , and fixes #1344
* slight refactor
* update configuration.rst entry
2 years ago
Mike Fährmann
4d78ca89db
[twitter] add 'cards-blacklist' option ( #2875 )
2 years ago
Mike Fährmann
8839b0d2ee
add section about global replacement fields to formatting.md
...
(#2862 )
2 years ago
Mike Fährmann
f16fbe9f93
document 'extractor.twitter.expand' ( #2848 )
2 years ago
Mike Fährmann
7e385ed63e
[foolfuuka] update domains
...
- remove nyafuu
- add rozenarcana (https://archive.alice.al/ )
- add tokyochronos (https://www.tokyochronos.net )
2 years ago
Mike Fährmann
67bad04dda
[formatter] add 'g' conversion to sluGify a string ( #2410 )
2 years ago
enduser420
3d87cedc58
[jpgchurch] rework the image extractor
...
now the image extractor can recognize if an image if from an album
also removed some unnecessary methods
2 years ago
enduser420
574e38a287
[kemonoparty] add 'favorites' option ( #2826 ) ( #2831 )
...
* [kemonoparty] add 'favorites' option (#2826 )
* [kemonoparty] add regex for the url parameter and fallback on the config
option
* [kemonoparty] simplify
2 years ago
Mike Fährmann
a799fae2df
[catbox] add 'album' extractor ( #2410 )
...
adapted from https://github.com/mikf/gallery-dl/pull/2805
- rewrite using GalleryExtractor
- extract more metadata
- match lolisafe names
- add test
2 years ago
Martin Juhasz
9ea521bec5
[oauth] add host config option ( #2806 )
2 years ago
Mike Fährmann
df1c643dda
[tumblr] attempt to extract full-resolution photos
...
- for photos with apparent width == 2048 or height == 3072
- can be disabled with 'original' option
2 years ago
Mike Fährmann
a69f198ac9
[wallhaven] implement 'metadata' option ( #2803 )
2 years ago
Mike Fährmann
7d1a95ada6
implement 'path-metadata' option ( #2734 )
2 years ago
Mike Fährmann
8cf5981ded
[skeb] add option to download 'article' images ( #1031 )
2 years ago
Mike Fährmann
98af5a0409
[zerochan] implement login with username & password ( #1434 )
2 years ago
Mike Fährmann
3a8addfe45
[zerochan] add 'tag' and 'image' extractors ( #1434 )
2 years ago
Chew Shee Yang
977d53b640
[Instagram] Add support for user's saved collection ( #2769 )
...
* [Instagram] Add support for user's saved collection
* [Instagram] Run formatter
* [Instagram] Simplify collection_id retrieval and add metadata
* [Instagram] Fix bug when params is not passed to _pagination_api
2 years ago
Mike Fährmann
0c73914848
[postprocessor:metadata] implement 'mode: modify' ( #2640 )
2 years ago
Mike Fährmann
f3de6b7a87
[postprocessor:metadata] implement 'mode: delete' ( #2640 )
2 years ago
Mike Fährmann
eb68d45544
add global 'warnings' option ( #2762 )
2 years ago
Mike Fährmann
f225247670
[gelbooru] add support for `api_key` and `user_id` ( #2767 )
2 years ago
Mike Fährmann
46f11a3118
[bunkr] fix extraction ( #2732 )
...
move bunkr.is code to its own module
2 years ago
Mike Fährmann
fe2b3d57d4
[komikcast] update domain
2 years ago
Mike Fährmann
2dc57637cf
[foolfuuka] remove archive.wakarimasen.moe
2 years ago
Mike Fährmann
74865adae5
implement 'format-separator' option ( #2737 )
...
a global option, that servers as a workaround for shortcomings due to
lack of a proper format string parser
2 years ago
bradenhilton
117eeefda0
[postprocessor:mtime] add 'value' option ( #2739 )
2 years ago
Mike Fährmann
90ae48c40c
[formatter] implement 'O' format specifier ( #2736 )
...
to apply a UTC offset to 'date' values and other datetime objects
2 years ago
Mike Fährmann
1d14928bd9
[twitter] ignore previously seen Tweets ( #2712 )
...
occurs primarily for /with_replies results when logged in
2 years ago
Mike Fährmann
4b2a0a0eda
[twitter] implement 'strategy' option ( #2712 )
...
to be able to better control what Tweets get used an returned
for twitter.com/USER URLs.
2 years ago
Mike Fährmann
124b5c6410
[postprocessor:ugoira] enable 'mtime' by default ( #2714 )
2 years ago
Mike Fährmann
3a5d5c3a91
update default User-Agent header to Firefox 102 ESR
...
snd update headers and ciphers for "browser": "firefox"
2 years ago
Mike Fährmann
54525d2e21
[formatter] implement slice operator as format specifier
...
this allows using a slice operator alongside other (special) format
specifiers like J, to first join list elements to a string and then
trimming that with a slice.
{tags:J, /[:50]}
2 years ago
Mike Fährmann
241e82e18d
[horne] add support for horne.red ( #2700 )
2 years ago
Mike Fährmann
9d8e99af80
[itaku] support videos ( #1842 )
2 years ago
Mike Fährmann
27e8078fb7
[poipiku] add 'user' and 'post' extractors ( #1602 )
2 years ago
Mike Fährmann
fa902cd54d
[itaku] add 'gallery' and 'image' extractors ( #1842 )
2 years ago
Mike Fährmann
9c8d895d19
[twitter] implement 'csrf' option ( #2676 )
2 years ago
Mike Fährmann
6db77d4656
[weibo] support '?tabtype=video' listings ( #2601 )
2 years ago
Mike Fährmann
4b78bd423f
[paheal] add 'metadata' option ( #2641 )
2 years ago
Mike Fährmann
57508d3bb7
[weibo] support all different 'tabtype' listings ( #686 , #2601 )
2 years ago
HRXN
6d0aa8c0f3
[docs/configuration.rst] Fix small typo ( #2649 )
2 years ago
Mike Fährmann
e0ac358aa5
[gofile] fix 401 Unauthorized errors ( #2632 )
2 years ago
Mike Fährmann
8a42d859bf
[bunkr] change domain to 'app.bunkr.is' ( #2634 )
2 years ago
Mike Fährmann
4bf5bc2403
[weibo] support 'livephoto' entries ( #2146 )
2 years ago
Mike Fährmann
5b43faffed
[postprocessor:metadata] write to stdout by setting filename to "-"
...
(#2624 )
2 years ago
Mike Fährmann
603af48265
implement 'output.ansi' option ( #2628 )
2 years ago
Mike Fährmann
ff5e10a86d
[hypnohub] move to gelbooru_v02 instances ( #2631 )
2 years ago
Mike Fährmann
d6e744bf0f
[gfycat] add 'collection' extractor ( #2629 )
2 years ago
Mike Fährmann
d4e9d51760
[reddit] add 'home' extractor ( #2614 )
2 years ago
Mike Fährmann
915dba8345
[twitter] improve results for regular user URLs
...
- continuation of 3346f58a
- use media timeline results (or tweet timeline if retweets are enabled)
plus search results starting from the last tweet id of the first
timeline, similar to how Twitter Media Downloader operates
- the old behavior can be forced by appending '/tweets' to a user URL,
like with '/media' (https://twitter.com/USER/tweets )
although there should be no need to ever do that
2 years ago
Mike Fährmann
9df4e0f65b
[twitter] disable 'cards' by default
2 years ago
Mike Fährmann
79dce8ae68
[weasyl] implement 'metadata' option ( #2610 )
2 years ago
Mike Fährmann
feb470d19a
[shopifx] natively support a few more sites ( closes #2089 )
...
- chelseacrew.com
- michaels.com.au
- modcloth.com
- pinupgirlclothing.com
- raidlondon.com (loveraid.com)
- unique-vintage.com
2 years ago
Mike Fährmann
60f4d59b1e
[gelbooru_v01] remove 'tlb.booru.org' from supported domains
...
403 Forbidden
nginx
it is also no longer listed on https://booru.org/top
2 years ago
Mike Fährmann
6b6eb0b8f6
[lolisafe] implement 'domain' option ( #2575 )
2 years ago
Mike Fährmann
6742f3bc1e
implement --cookies-from-browser ( #1606 )
...
most of the code is adapted from yt-dlp's implementation
and *should* work the same.
2 years ago
Mike Fährmann
61887c895b
implement 'output.colors' options ( #2532 )
2 years ago
Mike Fährmann
52b47c3cf9
[gelbooru_v01] add 'favorite' extractor ( #2546 )
2 years ago
Mike Fährmann
84756982e9
[pixiv] implement 'include' option
...
- split 'user' extractor and its 'avatar' and 'background' options into
separate extractors ('artworks', 'avatar', 'background')
- avatars can now be downloaded with
https://www.pixiv.net/en/users/ID/avatar
as URL and will use a proper archive key; similar for backgrounds
- options for the 'user' subcategory must be moved to 'artworks' to have
the same effect as before
2 years ago
Mike Fährmann
d11e2191ae
[nijie] support /history_nuita.php listings ( closes #2541 )
2 years ago
Mike Fährmann
61afa973b7
use extended format strings for archive keys
2 years ago
Federico Ravasio
0381752575
[photovogue] switch to .com, update api endpoint ( #2494 )
2 years ago
Mike Fährmann
9e6ff42a9d
[pixiv] implement 'background' option ( #623 , #1124 , #2495 )
2 years ago
Mike Fährmann
97e8a15295
[deviantart] implement 'pagination' option ( #2488 )
2 years ago
Mike Fährmann
12bd9ba33a
[readcomiconline] add 'quality' option ( #2467 )
2 years ago
Mike Fährmann
24ea1398bc
[postprocessor:ugoira] do not auto-select image2 demuxer ( #2492 )
2 years ago
Mike Fährmann
ffa1dd1d91
add 'coomer.party' to supported sites ( closes #2462 )
2 years ago
Mike Fährmann
9c5d2d7af3
[pinterest] add extractor for created pins ( #2452 )
3 years ago
Mike Fährmann
1171911dc3
[twitter] add 'syndication' option ( #2354 )
...
to fetch age-restricted content using Twitter's syndication API
3 years ago
loragja
7e545a3ae9
[gofile] add gofile.io extractor ( #2364 )
...
* Add gofile extractor
* add gofile extractor to module list
* add support for tiny monitors and ancient python versions
* seriously, f-strings are not *that* new...
* i love flake8 :)
* add 'api-token' and 'recursive' options
* add tests
3 years ago
Layerex
625f4d4cc4
[telegraph] Add telegra.ph extractor ( #2312 )
3 years ago
Mike Fährmann
5c2d953766
[postprocessor:ugoira] auto-select demuxer by default
3 years ago
Mike Fährmann
37d584a9b2
[hitomi] update metadata extraction ( fixes #2444 )
...
remove 'hitomi.metadata' option, as it is no longer necessary
to make additional HTTP requests to fetch all metadata.
3 years ago
Mike Fährmann
e718dd7b32
[postprocessor:ugoira] support setting timecodes with mkvmerge
...
by selecting the "mkvmerge" demuxer
(#1550 )
3 years ago
Mike Fährmann
71bba774da
respect 'output.private' in '-K/--list-keywords' output
3 years ago
Mike Fährmann
ba69fb669d
[kemonoparty] add 'duplicates' option ( closes #2440 )
3 years ago
Mike Fährmann
40ce505420
[postprocessor:ugoira] add 'mtime' option ( #2307 )
3 years ago
Mike Fährmann
9bd27b1b8d
[postprocessor:metadata] implement archive options ( #2421 )
...
'archive', 'archive-format', and 'archive-prefix'
3 years ago
Mike Fährmann
f8230dde43
[instagram] add 'previews' option ( #2135 )
3 years ago
Mike Fährmann
cf44aba333
[formatter] allow evaluating f-string literals
...
by starting a format string with '\fF'.
This was technically already possible with '\fE',
but this makes it a bit more convenient.
3 years ago
Mike Fährmann
58e0b17211
add note about using '\f' in shells to docs/formatting.md
...
(closes #2398 , closes #2406 )
3 years ago
lingegrino
b3d8b40690
Rename meme.museum ( #2393 )
3 years ago
Mike Fährmann
fc277fa45f
[seiga] require authentication with 'user_session' cookie ( #2372 )
...
Login with username & password would now require entering a 2FA token.
see also 7b009cc893
3 years ago
Mike Fährmann
47cf05c4ab
refactor proxy handling code ( #2357 )
...
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
- add 'downloader.proxy' option
- '-o extractor.proxy=–PROXY_URL -o downloader.proxy=null'
now has the same effect as youtube-dl's '--geo-verification-proxy'
3 years ago
Mike Fährmann
acbb0f5a57
clarify 'metadata.mtime' usage and shortcomings ( #2307 )
3 years ago
Mike Fährmann
bfa5e61900
[patreon] add explicit 'image_large' file type ( #2257 )
...
to allow more control over when and if to download 'large_url' images
4fee3a0e52
forced them to be downloaded
instead of regular images, even though 'large_url' images are most likely
an upscaled version of the original.
3 years ago
Alice
f1cab23724
[skeb] add 'sent-requests' option ( #2322 ) ( #2330 )
...
* Update skeb.py
* Update configuration.rst
* flake8
3 years ago
Mike Fährmann
4385a34e05
[twitter] fix handling of 429 responses ( fixes #2339 )
...
Twitter doesn't return a valid JSON response for 429 errors anymore.
3 years ago
Mike Fährmann
5a50569360
[toyhouse] support 'art' listings ( #1546 , #2331 )
3 years ago
Mike Fährmann
59749555b1
[postprocessor:mtime] add 'event' option ( #2307 )
3 years ago
Mike Fährmann
7958995398
[postprocessor:metadata] implement 'mtime' option ( #2307 )
3 years ago
Mike Fährmann
fdfdc1b614
[kissgoddess] add 'gallery' and 'model' extractors
...
(closes #1052 , #2304 )
3 years ago
Mike Fährmann
79a461a2c1
[mememuseum] add 'tag' and 'post' extractors ( closes #2264 )
3 years ago
Mike Fährmann
254a5b26e0
[twibooru] add extractors for searches, galleries, and posts
...
(#2219 )
3 years ago
Mike Fährmann
f5b2a9fcc6
add 'signals-ignore' option ( #2296 )
3 years ago
David Hoppenbrouwers
b17e2dcf93
[wallpapercave] add extractor for images ( #2205 )
3 years ago
Thomas Jost
a7de819aca
[lightroom] add Lightroom gallery extractor ( #2263 )
3 years ago
Mike Fährmann
563bd0ecf4
[danbooru] inherit from BaseExtractor
...
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283 )
- remove support for old e621 tag search URLs
3 years ago
Mike Fährmann
f1c853c6ef
[furaffinity] add 'layout' option ( #2277 )
...
to be able to force gallery-dl to parse according to a specific layout
in case its auto-detect fails
3 years ago
Mike Fährmann
4e4cb40663
do not sort BaseExtractor instances by name
3 years ago
Mike Fährmann
f351746483
release version 1.20.4
3 years ago
HRXN
f4a7c6eca2
Add two examples for custom instances of gelbooru ( #2262 )
...
* Add two examples for custom instances of gelbooru
- First example uses some custom options at the "gelbooru_v01" basecategory
These apply to all instances used herein
- Second example adds all options to the specific "gelbooru_v02" basecategory of TBIB
These settings are only used by TBIB
* move tbib settings into their own block
3 years ago
Mike Fährmann
cc7dce5755
[sexcom] add 'pins' extractor ( closes #2265 )
3 years ago
Mike Fährmann
02e18f56be
[e621] add 'favorite' extractor ( closes #2250 )
3 years ago
Mike Fährmann
86fa412b47
[hitomi] add 'format' option ( #2260 )
...
default is 'webp' since downloading original files is no longer allowed
3 years ago
Mike Fährmann
492436f936
[twitter] add 'warnings' option ( #2258 )
...
disable reporting any non-fatal errors by default
3 years ago
Mike Fährmann
9a221494c3
[twitter] add 'event' extractor ( closes #2109 )
3 years ago
Mike Fährmann
de754590e0
add --source-address command-line option ( closes #2206 )
3 years ago
Mike Fährmann
f2e8aedd74
[twitter] changes to 'cards' option
...
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
when 'cards' is set to "ytdl"
"cards": true --> only download card images
"cards": "ytdl" --> download card images and
use youtube_dl on otherwise unsupported cards
3 years ago
ImportTaste
c559c49781
formatting.md: add environment variable syntax ( #2065 )
3 years ago
Mike Fährmann
3c79c9b271
document extended blacklist/whitelist syntax ( #2025 )
...
and not just in the commit message of 010d65dc
3 years ago
Mike Fährmann
37beb1298e
[newgrounds] add 'search' extractor ( closes #2161 )
3 years ago
Mike Fährmann
fa7d92f7a9
add docs for 'extractor.generic.enabled'
3 years ago
Mike Fährmann
89bebe1bef
[500px] add 'favorite' extractor ( closes #1927 )
3 years ago
Mike Fährmann
882c614281
add album extractor for lolisafe/chibisafe instances
...
- support bunkr.is (closes #2038 )
- support zz.ht (closes #2105 )
3 years ago
Mike Fährmann
64cf26eaf4
allow specifying sleep-* options as string
...
either as single value or as range: "3.5", "2.1 - 5.0"
3 years ago
Mike Fährmann
62692c6842
[exhentai] add 'source' option
...
setting it to "hitomi" downloads the corresponding gallery from
hitomi.la; might be extended to other sources in the future
3 years ago
Mike Fährmann
099ed72de7
[hitomi] disable extra 'metadata' by default
...
safes one HTTP request that not needed with default filename settings
3 years ago
Mike Fährmann
f587458a3c
[twitter] include '4096x4096' as a default image fallback
...
(closes #2107 , closes #1881 )
3 years ago
Mike Fährmann
299bd2f1f5
[rule34us] add 'tag' and 'post' extractors ( #1527 )
3 years ago
Mike Fährmann
3cf1075d86
[inkbunny] add 'search' extractor ( closes #2094 )
3 years ago
Mike Fährmann
f4e3cee6ac
use yt-dlp by default ( #1850 , #2028 )
3 years ago
Mike Fährmann
f1b142e993
{kemonoparty[ change default 'files' order to attachments,file,inline
...
(#1991 )
3 years ago
Mike Fährmann
addb72e1bb
[reactor] support thatpervert.com ( closes #2029 )
3 years ago
Mike Fährmann
d8d9502e1e
[reactor] inherit from BaseExtractor
3 years ago
Mike Fährmann
f4ea216c95
[shopify] support loungeunderwear.com ( closes #2053 )
3 years ago
Alice
612850438e
[skeb] add 'thumbnails' option ( #2047 ) ( #2051 )
3 years ago
Mike Fährmann
11a3d96d13
[mangadex] load additional metadata using includes[] directives
...
- always provide 'artist', 'author', and 'group' metadata fields (#2049 )
- remove 'metadata' option
3 years ago
Mike Fährmann
c67756e187
[kemonoparty] add 'dms' option ( #2008 )
3 years ago
Mike Fährmann
3842cdcd8f
[formatter] implement 'D' format specifier
...
To be able to parse any string into a 'datetime' object
and format it as necessary.
Example:
{created_at:D%Y-%m-%dT%H:%M:%S%z}
->
"2010-01-01 00:00:00"
{created_at:D%Y-%m-%dT%H:%M:%S%z/%b %d %Y %I:%M %p}
->
"Jan 01 2010 12:00 AM"
with 'created_at' == "2010-01-01T01:00:00+0100"
3 years ago
Mike Fährmann
3a7a19c7b9
[dynastyscans] add 'manga' extractor ( closes #2035 )
3 years ago
Mike Fährmann
9bc83af3a6
[kemonoparty] 'postfile' -> 'file' ( #1991 )
...
to stay consistent with the existing file types for kemono
3 years ago
Mike Fährmann
d433735750
[kemonoparty] skip duplicate files ( #2032 , #1991 , #1899 )
...
Extract the SHA-256 file hash from URLs
and skip files with the same hash in the same post.
- provide a 'hash' metadata field (empty string if not available)
- remove 'patreon-skip-file' option
3 years ago
Mike Fährmann
d4ec245554
[kemonoparty] implement a 'files' option ( #1991 )
...
similar to 8d676151
3 years ago
Mike Fährmann
cfa4876848
[philomena] support furbooru.org ( closes #1995 )
3 years ago
Mike Fährmann
7c50f473e4
[ytdl] document 'cmdline-args' and 'config-file' options ( #1680 )
3 years ago
Mike Fährmann
37c9dedee1
[seisoparty] remove module
3 years ago
Mike Fährmann
b315a0ecef
[redgifs] update to API v2 ( #1984 )
3 years ago
Mike Fährmann
f0fc3b0ba1
[kemonoparty] add 'comments' option ( #1980 )
3 years ago
Mike Fährmann
4c49174579
[mangakakalot] update domain and fix extraction
3 years ago
ImportTaste
69cd573809
add special type format strings to docs ( #1987 )
3 years ago
Mike Fährmann
9156e90f1f
[twitter] add 'pinned' option
3 years ago
Mike Fährmann
06b414c9a3
[redgifs] 'gfyId' -> 'id' ( #1984 )
3 years ago
Alice
bfd7401b1e
[skeb] add 'user' and 'post' extractors ( #1031 ) ( #1971 )
...
* Create skeb.py
* Update __init__.py
* Update supportedsites.py
* Update supportedsites.md
* Update supportedsites.py
* Update skeb.py
3 years ago
Mike Fährmann
02a247f4e5
[deviantart] full resolution for non-downloadable images ( #293 )
...
Many thanks to @Ironchest337 for discovering this method
and providing a well-documented implementation.
3 years ago
Mike Fährmann
c19e762fdf
[vk] add 'album' extractor ( #474 , fixes #1952 )
...
todo: better metadata for albums
3 years ago
Mike Fährmann
8bb442f20d
[redgifs][gfycat] provide fallback URLs ( fixes #1962 )
...
and extend the 'format' option
3 years ago
Mike Fährmann
8d676151b7
[patreon] implement 'files' option ( #1935 )
3 years ago
Mike Fährmann
80314e981b
small corrections to gallery-dl-example.conf
...
use "'_reddit' in locals()"
instead of ""locals().get('_reddit')"
since the reddit metadata dict could be empty. The latter expression
would evaluate to False in that case, even though it has a reddit parent.
3 years ago
Mike Fährmann
3ecf4a9729
update gallery-dl-example.conf
...
- add comments
- remove/replace some older options
- display more post processor settings
3 years ago
Mike Fährmann
50098762e3
[nhentai] add 'tag' extractor ( closes #1950 )
3 years ago
Mike Fährmann
fe6ce5495a
[kemonoparty] add 'discord' extractor ( #1827 , #1940 )
3 years ago
Mike Fährmann
918fc9974d
[picarto] add 'gallery' extractor ( closes #1931 )
3 years ago