da9840a39d
[reddit] update 'videos' option ( #3712 )
...
- add 'dash' to directly extract DASH manifest URLs
(was default behavior since a7c79531a7c79531
2 years ago
baf41d7437
[misskey] update ( #3717 )
...
- add module docstring
- add options to docs/gallery-dl.conf
2 years ago
e1867cf5eb
[misskey] add 'renotes' and 'replies' options
2 years ago
a95b5e0d8e
[misskey] add misskey extractors
2 years ago
0d142e403c
[szurubooru] add 'tag' and 'post' extractors ( #3583 , #3713 )
2 years ago
075c965512
add '--config-create' command-line option
...
(#2333 )
2 years ago
632d5d7745
allow loading config files in TOML format with --config-toml
2 years ago
d788e6c60c
implement 'globals' option
2 years ago
b14f8d5817
[gelbooru] add 'favorite' extractor ( #3704 )
...
requires logged in cookies to work
2 years ago
e480a93337
add 'output.stdout', '.stdin', and '.stderr' options
...
(#1621 , #2152 , #2529 )
Allow setting custom input/output encodings and options
without having to rely on Python's defaults.
2 years ago
6ed4309aba
[deviantart] add 'gallery-search' extractor ( #1695 )
2 years ago
e1df7f73b1
[deviantart] add 'search' extractor
...
(#538 , #1264 , #2954 , #2970 , #3577 )
Requires login to fetch any results, since the API endpoint raises an
error for not logged in requests.
TODO: parse HTML search results
2 years ago
bb11c2a576
merge #3662 : [redgifs] add 'collection' extractors
2 years ago
884f1848d6
[redgifs] fix syntax for older Python versions
...
and update docs/supportedsites
2 years ago
2bd8f2f4bd
[pornpics] add 'search' and 'tag' extractors
...
(#263 , #3544 , #3654 )
2 years ago
79bc82884c
[pornpics] add 'gallery' extractor ( #263 , #3544 , #3654 )
2 years ago
bd5d08abbc
[catbox] add 'file' extractor ( #3570 )
2 years ago
2a53e6445c
[bunkr] update domain ( #3636 )
2 years ago
b7337d810e
[postprocessor:metadata] add 'sort' and 'separators' options
2 years ago
762a68996b
implement 'archive-pragma' option
2 years ago
bbf0911a46
[e621] implement 'notes' and 'pools' metadata extraction
...
(#3425 )
2 years ago
925b467496
split e621 from danbooru module ( #3425 )
2 years ago
1ae48a54f8
[twitter] add 'transform' option
2 years ago
78d3960a31
[postprocessor:exec] implement archive options ( #3584 )
2 years ago
0f7e6c422a
merge #3596 : [shopify] support ohpolly.com
2 years ago
fcf7030b85
[shopify] support ohpolly.com
2 years ago
4572c122d8
document 'extractor.module-sources'
2 years ago
c2bc70593e
implement ability to load external extractor classes
...
- -X/--extractors
- extractor.module-sources
2 years ago
13a90969c7
merge #3575 : [nudecollect] add 'image' and 'album' extractors
2 years ago
abc3619feb
[lexica] add 'search' extractor ( #3567 )
2 years ago
f58215705a
add '-O/--postprocessor-option' command-line option ( #3565 )
2 years ago
2a5903dc16
[nudecollect] add 'image' and 'album' extractors
2 years ago
58c008e30a
[hiperdex] update domain ( #3572 )
2 years ago
caae8fefe1
merge #3541 : [deviantart] add extractor for status updates
2 years ago
3b32671fbd
[deviantart] add extractor for status updates
...
extract user status updates using the '/user/statuses/' endpoint
2 years ago
5cb263fdd2
[wikifeet/wikifeetx] add 'gallery' extractor
2 years ago
4e86aaa4db
document 'extractor.headers' and 'extractor.ciphers'
2 years ago
c881548a27
add 'extractor.retry-codes' option ( #3313 )
...
do not retry 429 and 430 by default
2 years ago
512abeb4ae
[booru] add 'url' option
2 years ago
c87bd1a752
[danbooru] extend 'metadata' option
...
make it possible to specify a custom list of metadata includes
2 years ago
c0d7d2be35
[downloader:http] add 'validate' option
2 years ago
4833ec323e
[imagefap] add 'folder' extractor ( #3504 )
2 years ago
cb9d100bac
update Authentication support in docs
...
https://github.com/mikf/gallery-dl/pull/3075#issuecomment-1374723412
2 years ago
362cd6991b
[pixiv] implement 'metadata-bookmark' option ( #3417 )
2 years ago
0895e6afee
merge #3462 : [docs] Update links and fix field typo
2 years ago
4b9bb632f2
[docs] update links
2 years ago
d1dd52349a
merge #3189 : [tcbscans] add 'chapter' and 'manga' extractors
2 years ago
ac78712618
merge #3496 : [docs] update OAuth guide in README.rst
2 years ago
2f31d21509
merge #3455 : [twitter] apply tweet type checks before uniqueness check
2 years ago
64435cc37e
[twitter] update docs
2 years ago
b125fcafc5
update docs/configuration.rst
...
- document metadata.indent
- add concrete mappings to object types
- fix lists for code examples
- fix typos
2 years ago
e8541a131d
[tcbscans] add 'chapter' and 'manga' extractors
2 years ago
3c03928d75
simplify 'docs/options.md' generation
...
use the plain text output from '--help'
instead of trying to generate 'fancy' HTML with Markdown restrictions
2 years ago
c94692f39b
[docs] document 'access-token' for Mastodon instances
2 years ago
4fd6da474f
merge #3473 : [twitter] fix crash when using 'expand' and 'syndication'
2 years ago
818f6fa71f
merge #3453 : [twitter] document #1396 and #2712
2 years ago
455481ad76
[twitter] improve documentation
...
This includes:
- The automatic expansion of conversations (#2712 )
- How to retrieve age-restricted content from search results (#1396 )
2 years ago
f29ba089ff
merge #3474 : [fanleaks] add 'post' and 'model' extractors
2 years ago
6933727b45
merge #3483 : [twitter] implement 'syndication=extended'
2 years ago
9116398c1c
[pinterest] add 'domain' option ( #3484 )
...
use input URL domain by default
2 years ago
f1a715dbcd
improve docs/options.md
...
- use non-breaking hyphens
- don't replace '…' in examples
- don't force width for long option column
2 years ago
294108c90a
[pinterest] support 'All Pins' boards ( #2855 , #3484 )
2 years ago
13c16490bd
handle subcategory names ending with 'y'
2 years ago
50d89d4acb
docs/options.md: use a separate table for each option group
2 years ago
7de6b7f489
add list of command-line options as docs/options.md ( #3464 )
2 years ago
6853b14be3
[twitter] apply suggestions from code review
...
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
4611237f8c
merge #3457 : [danbooru] extract uploader metadata (if option is set)
2 years ago
e7522482bb
merge #3463 : [lynxchan] support 'bbw-chan.nl'
2 years ago
0ed5986062
document custom 'output.mode' ( #2529 )
2 years ago
5a740ef78b
[fanleaks] add 'post' and 'model' extractors
2 years ago
7c8eab8d52
[twitter] implement 'syndication=extended'
...
to be able to fetch extended user metadata
2 years ago
be3286206a
[twitter] assume 'conversation_id' when using syndication
...
not possible to expand replies at the momemt
2 years ago
ce8dbb1ccc
[twitter] fix crash when using 'expand' and 'syndication'
...
caused by KeyError: 'conversation_id_str'
2 years ago
d651d45239
implement specifying ranges in slice notation ( #918 , #2865 )
...
e.g.
- '1:101' or ':101' or ':101:' for files 1 to 100
- '1::2' or '::2' for every second file
- '1:101:5' or ':101:5' for files 1, 6, 11, ..., 91, 96
(the second argument specifies the first index NOT included)
2 years ago
527bb2c4ab
[lynxchan/bbw-chan] add 'thread' and 'board' extractors
2 years ago
64902f518e
[docs] Update links and fix field typo
2 years ago
5d5ea58887
[twitter] update docs for 'quoted' & 'unique'
2 years ago
f82ee93676
[danbooru] extract uploader metadata (if metadata is set)
2 years ago
1800bd7d14
allow '*-filter' options to be a list of expressions
2 years ago
895b41f1ac
[jschan] add generic jschan extractor
2 years ago
73ab5d84c0
update docs/configuration.rst
2 years ago
bee354c264
Merge pull request #3415 from enduser420/extractor/fapello
...
[fapello] add 'post', 'user' and 'path' extractors
2 years ago
dd4a4a3fa6
[e621] softcode the pagination threshold
2 years ago
e5076ba056
[fapello] add 'post', 'user' and 'path' extractors
2 years ago
d0ad6d0e67
[e621] implement manual pagination mode
2 years ago
1317625ec4
[webmshare] add 'video' extractor ( #2410 )
2 years ago
41bf236d36
[lynxchan] add generic extractors for lynxchan imageboards ( #3394 )
...
* [lynxchan] add generic extractors for lynxchan imageboards
includes kohlchan.net, endchan.org:wq
* [lynxchan] set pop default to empty tuple
* Apply suggestions from code review
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
3c75c3bbc4
[soundgasm] add 'user' extractor ( #3384 )
...
based on code from PR #3388 by @enduser420
2 years ago
2952add4a8
[reddit] increase 'id-max' default value ( #3397 )
...
to float("inf")
2 years ago
6b6f886dcf
[bunkr] update domain ( #3391 )
...
and improve bunkr/app.bunkr handling
2 years ago
eb94568e1f
[soundgasm] add 'audio' extractor ( #3384 )
2 years ago
dfe7b23579
support Firefox containers for --cookies-from-browser ( #3346 )
2 years ago
4a3a1f4c87
[komikcast] update domain and fix extraction
2 years ago
a42ba25ca1
[foolslide] remove 'kireicake'
...
site redirects to (unclaimed) mangadex group
2 years ago
86f0597c95
[kissgoddess] remove module
...
site does not host albums anymore
2 years ago
fceaee3c4f
[lolisafe] remove zz.ht
2 years ago
54844944ab
[pixhost] add 'gallery' support ( #3353 )
2 years ago
213676c785
[fapachi] add 'post' and 'user' extractors ( #3347 )
...
* [fapachi] add 'post' and 'user' extractors
* [fapachi] add 'keyword' to test
* [fapachi] remove whitespaces
2 years ago
80102fa367
[downloader:http] add 'retry-codes' option ( #3313 )
2 years ago
b14b33f19e
Implement `version-metadata` option ( #3201 )
2 years ago
6cb12f513b
[nitter] support quoted Tweets
...
- distinguish between regular and quoted Tweets and media
- add 'quoted' option and metadata field
2 years ago
a41d093bb1
[nitter] add 'retweets' option ( #3278 )
2 years ago
e31d12139c
[nitter] add 'videos' option ( #3279 )
...
with the same semantics as for twitter
2 years ago
501d9bccfe
[artstation] add 'max-posts' option ( #3270 )
2 years ago
b1ad6f2289
[artstation] add 'pro-first' option ( #3273 )
2 years ago
42481aed59
[formatter] implement 'S' format specifier ( #3266 )
...
to Sort lists
2 years ago
6b97dcf2e0
[postprocessor:metadata] add 'private' option
2 years ago
08fd1ff835
[twitter] add 'avatar' and 'background' extractors ( #349 , #3023 )
2 years ago
6379157543
[instagram] use REST API by default
...
regardless of logged in status
2 years ago
7897f68225
[wallhaven] update 'user' extractor ( #3226 )
...
* [wallhaven] update 'user' extractor
* [wallhaven] update 'configuration.rst'
add 'extractor.wallhaven.include' entry
* [wallhaven] add 'wallhaven.include' in gallery-dl.conf
2 years ago
5a68b5cb3c
[wallhaven] add 'user' extractor ( #3213 )
2 years ago
6c153750fa
[nitter] add extractors for Nitter instances ( #2696 )
2 years ago
9f06e79868
implement '"user-agent": "browser"' ( #2636 )
2 years ago
039d06c8f6
[mangaread] add 'chapter' and 'manga' extractors
2 years ago
70c7fbe89a
[instagram] add 'guide' extractor ( #3192 )
2 years ago
8d543b040d
update docs
...
- document '"mode": "jsonl"' for metadata pp
- remove 'username' and 'password' from 'instagram'
- add 'ytdl' example (#3180 )
2 years ago
171262c1b6
[instagram] remove login support
...
broken feature that I cannot get to work anymore
2 years ago
895f36e53b
[postprocessor:metadata] add 'open' and 'encoding' options
2 years ago
870e6a48a0
implement 'http-metadata' option
...
or at least attempt to.
2 years ago
ccb80f1b8b
[uploadir] add support for 'uploadir.com' ( #3162 )
2 years ago
c01cad599a
[lolisafe] add support for xbunkr ( #3156 )
2 years ago
bca9f965e5
[downloader:http] add 'chunk-size' option ( #3143 )
...
and double the previous default from 16384 (2**14) to 32768 (2**15)
2 years ago
c12a97bcde
[postprocessor] add 'post-after' event ( #3117 )
2 years ago
58d97188b4
[mastodon] add 'bookmark' extractor ( #3109 )
2 years ago
7c6af27eb8
[tumblr] add 'fallback-*' options ( #2957 )
...
specifically 'fallback-delay' and 'fallback-retries'
and change default number of retries to 2 (down from 3)
2 years ago
ac6111e693
[mangasee] add support for 'mangalife' ( #3086 )
2 years ago
a7d23f1484
[vichan] add generic extractors for vichan imageboards
...
includes 8kun.top, smuglo.li, and wikieat.club
2 years ago
062ef238a6
add support for aibooru (using danbooru extractor) ( #3075 )
2 years ago
0163ca86f7
[smugloli] add smugloli extractors ( #3060 )
2 years ago
cf86f68864
[instagram] add 'avatar' extractor ( #929 , #1097 , #2992 )
2 years ago
618c81afdf
[ngomik] remove module
...
"Access denied"
2 years ago
52d1eb928d
[pixiv] extend 'metadata' option ( #3057 )
...
make it usable for all 'pixiv' extractors
2 years ago
0714274f1f
[instagram] remove 'channel' extractor
2 years ago
3b369ce3d1
[nijie] add 'followed' extractor ( #3048 )
2 years ago
c4a62a48ae
[nijie] add 'feed' extractor ( #3048 )
2 years ago
f1f89b2436
[tumblr] add 'offset' option
2 years ago
15a4d7696f
[docs] update 'adjust-extensions' explanation ( #3031 )
2 years ago
1696f68a68
[8chan] add 'thread' and 'board' extractors ( #2938 )
2 years ago
560f7b41d8
[vk] add 'tagged' extractor ( #2997 )
2 years ago
7f30a0d7a7
add 'path-extended' option ( #3021 )
2 years ago
14717f3fc9
[deviantart] add 'group' option ( #3018 )
...
disabling this option allows to better download from deleted accounts
2 years ago
f0321f423d
[2chen] Add 2chen.moe extractor ( #2707 )
...
* [2chen] Add 2chen.moe extractor
* change "==" to is
* fix for "test_unique_pattern_matches"
* fix regex pattern and group matching
* fix regex again
* [2chen] add 'reply_no' and 'hash' metadata and change 'filename_fmt'
also made an entry in supportedsites.md
* [2chen] unescape 'title'
* [2chen] partition() -> rpartition()
* [2chen] extract 'date' and 'name' metadata
* [2chen] remove 'offset' argument
* [2chen] do some changes
* [2chen] do some more changes
* [2chen] unescape 'name' and 'filename'
2 years ago
f7ba19a1c0
[nana] add 'nana' extractors ( #2967 )
2 years ago
4089bceddd
[sankaku] implement 'refresh' option ( #2958 )
2 years ago
63e0924927
[pixiv] add 'series' extractor ( #2964 )
2 years ago
aa49bf13d2
[instagram] add 'api' option
2 years ago
ac45ed2764
[skeb] implement 'filters' option ( #2945 )
2 years ago
08b0c4c0b1
[docs] add kemono text content example ( #2963 )
2 years ago
0393e59535
[newgrounds] add 'games' extractor ( #2955 )
2 years ago
4b2a006871
[skeb] add 'search' extractor ( #2945 )
2 years ago
eef50c1f28
[blogger] split 'search' extractor ( #2930 )
2 years ago
bd846abba0
[hotleak] add hotleak extractor ( #2909 ) ( #2890 )
2 years ago
e99a9b2aff
[twitter] improve 'cards-blacklist' ( #2875 )
...
allow blacklisting domains and 'name:domain',
where 'domain' depends on a card's 'vanity_url' value
2 years ago
46fe469c53
[tumblr] implement 'ratelimit' option ( #2919 )
2 years ago
b36125333f
[postprocessor:zip] implement 'files' option ( #2872 )
2 years ago
3cb8327c60
[zerochan] add 'metadata' option ( #2861 )
2 years ago
9745b48830
[tumblr] attempt to fetch high-quality inline images ( #2877 )
...
* [tumblr] attempt to fetch high-quality images (again)
Fixes #1846 , and fixes #1344
* slight refactor
* update configuration.rst entry
2 years ago
4d78ca89db
[twitter] add 'cards-blacklist' option ( #2875 )
2 years ago
8839b0d2ee
add section about global replacement fields to formatting.md
...
(#2862 )
2 years ago
f16fbe9f93
document 'extractor.twitter.expand' ( #2848 )
2 years ago
7e385ed63e
[foolfuuka] update domains
...
- remove nyafuu
- add rozenarcana (https://archive.alice.al/ )
- add tokyochronos (https://www.tokyochronos.net )
2 years ago
67bad04dda
[formatter] add 'g' conversion to sluGify a string ( #2410 )
2 years ago
3d87cedc58
[jpgchurch] rework the image extractor
...
now the image extractor can recognize if an image if from an album
also removed some unnecessary methods
2 years ago
574e38a287
[kemonoparty] add 'favorites' option ( #2826 ) ( #2831 )
...
* [kemonoparty] add 'favorites' option (#2826 )
* [kemonoparty] add regex for the url parameter and fallback on the config
option
* [kemonoparty] simplify
2 years ago
a799fae2df
[catbox] add 'album' extractor ( #2410 )
...
adapted from https://github.com/mikf/gallery-dl/pull/2805
- rewrite using GalleryExtractor
- extract more metadata
- match lolisafe names
- add test
2 years ago
9ea521bec5
[oauth] add host config option ( #2806 )
2 years ago
df1c643dda
[tumblr] attempt to extract full-resolution photos
...
- for photos with apparent width == 2048 or height == 3072
- can be disabled with 'original' option
2 years ago
a69f198ac9
[wallhaven] implement 'metadata' option ( #2803 )
2 years ago
7d1a95ada6
implement 'path-metadata' option ( #2734 )
2 years ago
8cf5981ded
[skeb] add option to download 'article' images ( #1031 )
2 years ago
98af5a0409
[zerochan] implement login with username & password ( #1434 )
2 years ago
3a8addfe45
[zerochan] add 'tag' and 'image' extractors ( #1434 )
2 years ago
977d53b640
[Instagram] Add support for user's saved collection ( #2769 )
...
* [Instagram] Add support for user's saved collection
* [Instagram] Run formatter
* [Instagram] Simplify collection_id retrieval and add metadata
* [Instagram] Fix bug when params is not passed to _pagination_api
2 years ago
0c73914848
[postprocessor:metadata] implement 'mode: modify' ( #2640 )
2 years ago
f3de6b7a87
[postprocessor:metadata] implement 'mode: delete' ( #2640 )
2 years ago
eb68d45544
add global 'warnings' option ( #2762 )
2 years ago
f225247670
[gelbooru] add support for `api_key` and `user_id` ( #2767 )
2 years ago
46f11a3118
[bunkr] fix extraction ( #2732 )
...
move bunkr.is code to its own module
2 years ago
fe2b3d57d4
[komikcast] update domain
2 years ago
2dc57637cf
[foolfuuka] remove archive.wakarimasen.moe
2 years ago
74865adae5
implement 'format-separator' option ( #2737 )
...
a global option, that servers as a workaround for shortcomings due to
lack of a proper format string parser
2 years ago
117eeefda0
[postprocessor:mtime] add 'value' option ( #2739 )
2 years ago
90ae48c40c
[formatter] implement 'O' format specifier ( #2736 )
...
to apply a UTC offset to 'date' values and other datetime objects
2 years ago
1d14928bd9
[twitter] ignore previously seen Tweets ( #2712 )
...
occurs primarily for /with_replies results when logged in
2 years ago
4b2a0a0eda
[twitter] implement 'strategy' option ( #2712 )
...
to be able to better control what Tweets get used an returned
for twitter.com/USER URLs.
2 years ago
124b5c6410
[postprocessor:ugoira] enable 'mtime' by default ( #2714 )
2 years ago
3a5d5c3a91
update default User-Agent header to Firefox 102 ESR
...
snd update headers and ciphers for "browser": "firefox"
2 years ago
54525d2e21
[formatter] implement slice operator as format specifier
...
this allows using a slice operator alongside other (special) format
specifiers like J, to first join list elements to a string and then
trimming that with a slice.
{tags:J, /[:50]}
2 years ago
241e82e18d
[horne] add support for horne.red ( #2700 )
2 years ago
9d8e99af80
[itaku] support videos ( #1842 )
2 years ago
27e8078fb7
[poipiku] add 'user' and 'post' extractors ( #1602 )
2 years ago
fa902cd54d
[itaku] add 'gallery' and 'image' extractors ( #1842 )
2 years ago
9c8d895d19
[twitter] implement 'csrf' option ( #2676 )
2 years ago
6db77d4656
[weibo] support '?tabtype=video' listings ( #2601 )
2 years ago
4b78bd423f
[paheal] add 'metadata' option ( #2641 )
2 years ago
57508d3bb7
[weibo] support all different 'tabtype' listings ( #686 , #2601 )
2 years ago
6d0aa8c0f3
[docs/configuration.rst] Fix small typo ( #2649 )
2 years ago
e0ac358aa5
[gofile] fix 401 Unauthorized errors ( #2632 )
2 years ago
8a42d859bf
[bunkr] change domain to 'app.bunkr.is' ( #2634 )
2 years ago
4bf5bc2403
[weibo] support 'livephoto' entries ( #2146 )
2 years ago
5b43faffed
[postprocessor:metadata] write to stdout by setting filename to "-"
...
(#2624 )
2 years ago
603af48265
implement 'output.ansi' option ( #2628 )
2 years ago
ff5e10a86d
[hypnohub] move to gelbooru_v02 instances ( #2631 )
2 years ago
d6e744bf0f
[gfycat] add 'collection' extractor ( #2629 )
2 years ago
d4e9d51760
[reddit] add 'home' extractor ( #2614 )
2 years ago
915dba8345
[twitter] improve results for regular user URLs
...
- continuation of 3346f58ahttps://twitter.com/USER/tweets )
although there should be no need to ever do that
2 years ago
9df4e0f65b
[twitter] disable 'cards' by default
2 years ago
79dce8ae68
[weasyl] implement 'metadata' option ( #2610 )
2 years ago
feb470d19a
[shopifx] natively support a few more sites ( closes #2089 )
...
- chelseacrew.com
- michaels.com.au
- modcloth.com
- pinupgirlclothing.com
- raidlondon.com (loveraid.com)
- unique-vintage.com
2 years ago
60f4d59b1e
[gelbooru_v01] remove 'tlb.booru.org' from supported domains
...
403 Forbidden
nginx
it is also no longer listed on https://booru.org/top
2 years ago
6b6eb0b8f6
[lolisafe] implement 'domain' option ( #2575 )
2 years ago
6742f3bc1e
implement --cookies-from-browser ( #1606 )
...
most of the code is adapted from yt-dlp's implementation
and *should* work the same.
2 years ago
61887c895b
implement 'output.colors' options ( #2532 )
2 years ago
52b47c3cf9
[gelbooru_v01] add 'favorite' extractor ( #2546 )
2 years ago
84756982e9
[pixiv] implement 'include' option
...
- split 'user' extractor and its 'avatar' and 'background' options into
separate extractors ('artworks', 'avatar', 'background')
- avatars can now be downloaded with
https://www.pixiv.net/en/users/ID/avatar
as URL and will use a proper archive key; similar for backgrounds
- options for the 'user' subcategory must be moved to 'artworks' to have
the same effect as before
2 years ago
d11e2191ae
[nijie] support /history_nuita.php listings ( closes #2541 )
2 years ago
61afa973b7
use extended format strings for archive keys
2 years ago
0381752575
[photovogue] switch to .com, update api endpoint ( #2494 )
2 years ago
9e6ff42a9d
[pixiv] implement 'background' option ( #623 , #1124 , #2495 )
2 years ago
97e8a15295
[deviantart] implement 'pagination' option ( #2488 )
2 years ago
12bd9ba33a
[readcomiconline] add 'quality' option ( #2467 )
2 years ago
24ea1398bc
[postprocessor:ugoira] do not auto-select image2 demuxer ( #2492 )
2 years ago
ffa1dd1d91
add 'coomer.party' to supported sites ( closes #2462 )
2 years ago
9c5d2d7af3
[pinterest] add extractor for created pins ( #2452 )
3 years ago
1171911dc3
[twitter] add 'syndication' option ( #2354 )
...
to fetch age-restricted content using Twitter's syndication API
3 years ago
7e545a3ae9
[gofile] add gofile.io extractor ( #2364 )
...
* Add gofile extractor
* add gofile extractor to module list
* add support for tiny monitors and ancient python versions
* seriously, f-strings are not *that* new...
* i love flake8 :)
* add 'api-token' and 'recursive' options
* add tests
3 years ago
625f4d4cc4
[telegraph] Add telegra.ph extractor ( #2312 )
3 years ago
5c2d953766
[postprocessor:ugoira] auto-select demuxer by default
3 years ago
37d584a9b2
[hitomi] update metadata extraction ( fixes #2444 )
...
remove 'hitomi.metadata' option, as it is no longer necessary
to make additional HTTP requests to fetch all metadata.
3 years ago
e718dd7b32
[postprocessor:ugoira] support setting timecodes with mkvmerge
...
by selecting the "mkvmerge" demuxer
(#1550 )
3 years ago
71bba774da
respect 'output.private' in '-K/--list-keywords' output
3 years ago
ba69fb669d
[kemonoparty] add 'duplicates' option ( closes #2440 )
3 years ago
40ce505420
[postprocessor:ugoira] add 'mtime' option ( #2307 )
3 years ago
9bd27b1b8d
[postprocessor:metadata] implement archive options ( #2421 )
...
'archive', 'archive-format', and 'archive-prefix'
3 years ago
f8230dde43
[instagram] add 'previews' option ( #2135 )
3 years ago
cf44aba333
[formatter] allow evaluating f-string literals
...
by starting a format string with '\fF'.
This was technically already possible with '\fE',
but this makes it a bit more convenient.
3 years ago
58e0b17211
add note about using '\f' in shells to docs/formatting.md
...
(closes #2398 , closes #2406 )
3 years ago
b3d8b40690
Rename meme.museum ( #2393 )
3 years ago
fc277fa45f
[seiga] require authentication with 'user_session' cookie ( #2372 )
...
Login with username & password would now require entering a 2FA token.
see also 7b009cc893
3 years ago
47cf05c4ab
refactor proxy handling code ( #2357 )
...
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
- add 'downloader.proxy' option
- '-o extractor.proxy=–PROXY_URL -o downloader.proxy=null'
now has the same effect as youtube-dl's '--geo-verification-proxy'
3 years ago
acbb0f5a57
clarify 'metadata.mtime' usage and shortcomings ( #2307 )
3 years ago
bfa5e61900
[patreon] add explicit 'image_large' file type ( #2257 )
...
to allow more control over when and if to download 'large_url' images
4fee3a0e52
3 years ago
f1cab23724
[skeb] add 'sent-requests' option ( #2322 ) ( #2330 )
...
* Update skeb.py
* Update configuration.rst
* flake8
3 years ago
4385a34e05
[twitter] fix handling of 429 responses ( fixes #2339 )
...
Twitter doesn't return a valid JSON response for 429 errors anymore.
3 years ago
5a50569360
[toyhouse] support 'art' listings ( #1546 , #2331 )
3 years ago
59749555b1
[postprocessor:mtime] add 'event' option ( #2307 )
3 years ago
7958995398
[postprocessor:metadata] implement 'mtime' option ( #2307 )
3 years ago
fdfdc1b614
[kissgoddess] add 'gallery' and 'model' extractors
...
(closes #1052 , #2304 )
3 years ago
79a461a2c1
[mememuseum] add 'tag' and 'post' extractors ( closes #2264 )
3 years ago
254a5b26e0
[twibooru] add extractors for searches, galleries, and posts
...
(#2219 )
3 years ago
f5b2a9fcc6
add 'signals-ignore' option ( #2296 )
3 years ago
b17e2dcf93
[wallpapercave] add extractor for images ( #2205 )
3 years ago
a7de819aca
[lightroom] add Lightroom gallery extractor ( #2263 )
3 years ago
563bd0ecf4
[danbooru] inherit from BaseExtractor
...
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283 )
- remove support for old e621 tag search URLs
3 years ago
f1c853c6ef
[furaffinity] add 'layout' option ( #2277 )
...
to be able to force gallery-dl to parse according to a specific layout
in case its auto-detect fails
3 years ago
4e4cb40663
do not sort BaseExtractor instances by name
3 years ago
f351746483
release version 1.20.4
3 years ago
f4a7c6eca2
Add two examples for custom instances of gelbooru ( #2262 )
...
* Add two examples for custom instances of gelbooru
- First example uses some custom options at the "gelbooru_v01" basecategory
These apply to all instances used herein
- Second example adds all options to the specific "gelbooru_v02" basecategory of TBIB
These settings are only used by TBIB
* move tbib settings into their own block
3 years ago
cc7dce5755
[sexcom] add 'pins' extractor ( closes #2265 )
3 years ago
02e18f56be
[e621] add 'favorite' extractor ( closes #2250 )
3 years ago
86fa412b47
[hitomi] add 'format' option ( #2260 )
...
default is 'webp' since downloading original files is no longer allowed
3 years ago
492436f936
[twitter] add 'warnings' option ( #2258 )
...
disable reporting any non-fatal errors by default
3 years ago
9a221494c3
[twitter] add 'event' extractor ( closes #2109 )
3 years ago
de754590e0
add --source-address command-line option ( closes #2206 )
3 years ago
f2e8aedd74
[twitter] changes to 'cards' option
...
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
when 'cards' is set to "ytdl"
"cards": true --> only download card images
"cards": "ytdl" --> download card images and
use youtube_dl on otherwise unsupported cards
3 years ago
c559c49781
formatting.md: add environment variable syntax ( #2065 )
3 years ago
3c79c9b271
document extended blacklist/whitelist syntax ( #2025 )
...
and not just in the commit message of 010d65dc
3 years ago
37beb1298e
[newgrounds] add 'search' extractor ( closes #2161 )
3 years ago
fa7d92f7a9
add docs for 'extractor.generic.enabled'
3 years ago
89bebe1bef
[500px] add 'favorite' extractor ( closes #1927 )
3 years ago
882c614281
add album extractor for lolisafe/chibisafe instances
...
- support bunkr.is (closes #2038 )
- support zz.ht (closes #2105 )
3 years ago
64cf26eaf4
allow specifying sleep-* options as string
...
either as single value or as range: "3.5", "2.1 - 5.0"
3 years ago
62692c6842
[exhentai] add 'source' option
...
setting it to "hitomi" downloads the corresponding gallery from
hitomi.la; might be extended to other sources in the future
3 years ago
099ed72de7
[hitomi] disable extra 'metadata' by default
...
safes one HTTP request that not needed with default filename settings
3 years ago
f587458a3c
[twitter] include '4096x4096' as a default image fallback
...
(closes #2107 , closes #1881 )
3 years ago
299bd2f1f5
[rule34us] add 'tag' and 'post' extractors ( #1527 )
3 years ago
3cf1075d86
[inkbunny] add 'search' extractor ( closes #2094 )
3 years ago
f4e3cee6ac
use yt-dlp by default ( #1850 , #2028 )
3 years ago
f1b142e993
{kemonoparty[ change default 'files' order to attachments,file,inline
...
(#1991 )
3 years ago
addb72e1bb
[reactor] support thatpervert.com ( closes #2029 )
3 years ago
d8d9502e1e
[reactor] inherit from BaseExtractor
3 years ago
f4ea216c95
[shopify] support loungeunderwear.com ( closes #2053 )
3 years ago
612850438e
[skeb] add 'thumbnails' option ( #2047 ) ( #2051 )
3 years ago
11a3d96d13
[mangadex] load additional metadata using includes[] directives
...
- always provide 'artist', 'author', and 'group' metadata fields (#2049 )
- remove 'metadata' option
3 years ago
c67756e187
[kemonoparty] add 'dms' option ( #2008 )
3 years ago
3842cdcd8f
[formatter] implement 'D' format specifier
...
To be able to parse any string into a 'datetime' object
and format it as necessary.
Example:
{created_at:D%Y-%m-%dT%H:%M:%S%z}
->
"2010-01-01 00:00:00"
{created_at:D%Y-%m-%dT%H:%M:%S%z/%b %d %Y %I:%M %p}
->
"Jan 01 2010 12:00 AM"
with 'created_at' == "2010-01-01T01:00:00+0100"
3 years ago
3a7a19c7b9
[dynastyscans] add 'manga' extractor ( closes #2035 )
3 years ago
9bc83af3a6
[kemonoparty] 'postfile' -> 'file' ( #1991 )
...
to stay consistent with the existing file types for kemono
3 years ago
d433735750
[kemonoparty] skip duplicate files ( #2032 , #1991 , #1899 )
...
Extract the SHA-256 file hash from URLs
and skip files with the same hash in the same post.
- provide a 'hash' metadata field (empty string if not available)
- remove 'patreon-skip-file' option
3 years ago
d4ec245554
[kemonoparty] implement a 'files' option ( #1991 )
...
similar to 8d676151
3 years ago
cfa4876848
[philomena] support furbooru.org ( closes #1995 )
3 years ago
7c50f473e4
[ytdl] document 'cmdline-args' and 'config-file' options ( #1680 )
3 years ago
37c9dedee1
[seisoparty] remove module
3 years ago
b315a0ecef
[redgifs] update to API v2 ( #1984 )
3 years ago
f0fc3b0ba1
[kemonoparty] add 'comments' option ( #1980 )
3 years ago
4c49174579
[mangakakalot] update domain and fix extraction
3 years ago
69cd573809
add special type format strings to docs ( #1987 )
3 years ago
9156e90f1f
[twitter] add 'pinned' option
3 years ago
06b414c9a3
[redgifs] 'gfyId' -> 'id' ( #1984 )
3 years ago
bfd7401b1e
[skeb] add 'user' and 'post' extractors ( #1031 ) ( #1971 )
...
* Create skeb.py
* Update __init__.py
* Update supportedsites.py
* Update supportedsites.md
* Update supportedsites.py
* Update skeb.py
3 years ago
02a247f4e5
[deviantart] full resolution for non-downloadable images ( #293 )
...
Many thanks to @Ironchest337 for discovering this method
and providing a well-documented implementation.
3 years ago
c19e762fdf
[vk] add 'album' extractor ( #474 , fixes #1952 )
...
todo: better metadata for albums
3 years ago
8bb442f20d
[redgifs][gfycat] provide fallback URLs ( fixes #1962 )
...
and extend the 'format' option
3 years ago
8d676151b7
[patreon] implement 'files' option ( #1935 )
3 years ago
80314e981b
small corrections to gallery-dl-example.conf
...
use "'_reddit' in locals()"
instead of ""locals().get('_reddit')"
since the reddit metadata dict could be empty. The latter expression
would evaluate to False in that case, even though it has a reddit parent.
3 years ago
3ecf4a9729
update gallery-dl-example.conf
...
- add comments
- remove/replace some older options
- display more post processor settings
3 years ago
50098762e3
[nhentai] add 'tag' extractor ( closes #1950 )
3 years ago
fe6ce5495a
[kemonoparty] add 'discord' extractor ( #1827 , #1940 )
3 years ago
918fc9974d
[picarto] add 'gallery' extractor ( closes #1931 )
3 years ago
e33125ad39
[pixiv] add 'sketch' extractor ( #1497 )
3 years ago
b93915c113
[inkbunny] add 'pool' extractor ( #1937 )
3 years ago
373d3e1c57
[seisoparty] implement login with username & password ( #1906 )
3 years ago
7c5f62b453
[seisoparty] add 'favorite' extractor ( #1906 )
3 years ago
d93b5474c3
[mangadex] update parameter handling for API requests
...
- move common parameters into '_pagination()'
- add 'ratings' (#1908 ) and 'api-parameters' options
3 years ago
cd66c3c415
[twitter] add 'size' option ( #1881 )
3 years ago
df8050b81d
[postprocessor:compare] add 'equal' option ( #1592 )
...
Move functionality from cdd72e14
3 years ago
cdd72e1413
[postprocessor:compare] extend 'action' option ( #1592 )
...
allow setting it to "abort", "terminate", or "exit" as with 'skip'
3 years ago
e0bdacd932
[fappic] add 'image' extractor ( closes #1898 )
3 years ago
8c29a6e491
add example for 'url-metadata' option ( closes #1884 )
3 years ago
d0761454b1
implement a download progress indicator ( #1519 )
3 years ago
140a7bfdcd
update --clear-cache command in deviantart client-id instructions
3 years ago
e69ee41f25
implement 'page-reverse' option ( #1854 )
3 years ago
e4684c5cb9
[desktopography] simplify ( #1740 )
3 years ago
7b05600056
[pinterest] only mention cookies for authentication ( #1856 )
...
regular login is broken due to CAPTCHA
3 years ago
266ed9b62e
[aryion] add 'tag' extractor ( closes #1849 )
3 years ago
c9e6693530
allow specifying a minimum/maximum for 'sleep-*' options ( #1835 )
...
for example '"sleep-request": [5.0, 10.0]' to wait between 5 and 10
seconds between each HTTP request
3 years ago
bd845303ad
implement a way to shorten filenames with east-asian characters
...
(#1377 )
Setting 'output.shorten' to "eaw" (East-Asian Width) uses a slower
algorithm that also considers characters with a width > 1.
3 years ago
7bbb1f92d7
[gelbooru_v02] add 'favorite' extractor ( closes #1834 )
3 years ago
4ec11af6a4
[kemonoparty] implement login with username & password ( #1824 )
3 years ago
83bbb628d8
[kemonoparty] add 'favorite' extractor ( #1824 )
3 years ago
44f572c27f
[deviantart] implement a 'auto-unwatch' option ( #1466 , #1757 )
3 years ago
268c9ca034
update default gallery-dl.conf
3 years ago
3e36543c98
[nhentai] add 'favorite' extractor ( #1814 )
3 years ago
96fec14ef7
[deviantart] rename 'watch' option to 'auto-watch'
...
(#1466 , #1757 )
Similar reason as in e05a96db
3 years ago
e75f2de9da
[deviantart] add 'comments' option ( #1800 )
3 years ago
6ce16c6d31
[deviantart] add 'tag' extractor ( closes #1803 )
3 years ago
4e9f8fe395
[shopify] support windsorstore.com ( #1793 )
3 years ago
292fffc83c
add 'j' format string conversion
...
to convert to a JSON formatted string
3 years ago
ecc8da4704
[deviantart] implement a 'watch' option ( #1466 , #1757 )
3 years ago
d3eab417ed
implement a 'path-strip' option
3 years ago
c04f7ab139
[foolfuuka] add 'gallery' extractor ( #1785 )
3 years ago
20ee091289
[429chan] add 'thread' and 'board' extractors ( closes #1773 )
3 years ago
51eb50749f
[foolslide] remove entry for kobato.hologfx.com
3 years ago
d320ee6251
implement a 'fallback' option ( closes #1770 )
3 years ago
c866fcba48
[twitter] fix 'logout' ( #1719 )
...
delete 'auth_token' cookie and cookies.txt path
3 years ago
ed4b3c48cb
fix flake8 and other tests
3 years ago
fa6af46756
Added ability to download GIFs instead of mp4 from Luscious and Reactor ( #1701 )
3 years ago
e5a93e113f
[twitter] extend 'replies' option ( #1254 )
...
Allow setting 'replies to '"self"' to only download from self-replies.
3 years ago
cadfad4eea
[danbooru] add 'external' option ( closes #1747 )
3 years ago
5b1c62bfa9
[furaffinity] add 'external' option ( closes #1492 )
3 years ago
Mike Fährmann