f9fb276e81
[postprocessor] add 'prepare-after' event ( #4083 )
1 year ago
0ef1fcab20
[postprocessor] update 'finalize' events
...
Add 'finalize-error' and 'finalize-success' events that trigger
depending on whether error(s) did or did not happen.
'finalize' itself now always triggers regardless of error status.
(was supposed to have the same behavior as the new 'finalize-success')
1 year ago
9d67655397
add "ascii+" as a special 'path-restrict' value ( #4371 )
1 year ago
1ece3b92ff
[mangadex] allow multiple values for 'lang' ( #4093 )
...
This was already possible by setting 'lang' to a list of strings,
but now it can also be done as a more command-line friendly string.
-o lang=fr,it
1 year ago
c6d70c7483
[docs] fix typo ( #4350 )
1 year ago
7cd5130e88
[docs] small fixes/updates
1 year ago
54d974deb0
add 'python' post processor
...
similar to 'exec' but calls a Python function
1 year ago
ceebacc9e1
remove 'pyopenssl' option
1 year ago
90231f2d5a
[twitter] add 'tweet-endpoint' option ( #4307 )
...
use the newer TweetResultByRestId only for guests by default
1 year ago
20ed647f6f
[twitter] add 'user' extractor and 'include' option ( #4275 )
1 year ago
86be197d11
[twitter] remove '/search/adaptive.json'
1 year ago
fc43c74694
[docs] update reddit client-id instructions ( #4292 )
...
add clearing cached access tokens and getting a new refresh token
1 year ago
7444fc125b
[gfycat] implement login support ( #3770 , #4271 )
...
For the record: '/webtoken' and '/weblogin' are not the same ...
1 year ago
5b59a0d143
update default User-Agent header to Firefox 115 ESR
1 year ago
1bf9f52c99
[twitter] add 'ratelimit' option ( #4251 )
1 year ago
f86fdf64a6
[twitter] use GraphQL search by default ( #4264 )
1 year ago
6ae3101fd0
[mangapark] add 'source' option ( #3969 )
1 year ago
c45a913bfd
[flickr] add 'exif' option
1 year ago
ccbc1a1d55
[flickr] add 'metadata' option ( #4227 )
1 year ago
c1cce4a80b
[twitter] extend 'conversations' option ( #4211 )
1 year ago
f50b81d3e9
[docs] add novel-related keys to 'pixiv.include' ( #4111 )
1 year ago
deff3b434d
[vipergirls] implement login support ( #4166 )
1 year ago
54cf1fa3e7
[twitter] use GraphQL search endpoint ( #3942 )
...
for guest users; selectable with 'search-endpoint' option.
adapted from 9c7b888ffa
1 year ago
0cf7282fa0
[pixiv] add 'full-series' option for novels ( #4111 )
1 year ago
28f6487c64
[instagram] add 'metadata' option ( #3107 )
1 year ago
f3cca50b9e
[mangadex] update links to API docs
1 year ago
65a9f4b124
merge #3950 : [misskey] add 'favorite' extractor
1 year ago
c76f0f3a1b
[misskey] update
...
- rename to 'MisskeyFavoriteExtractor'
- add 'access-token' option to docs
- add test URLs for other instances
- simplify 'pattern'
1 year ago
3fca455b82
[pixiv] add 'embeds' option ( #1241 )
1 year ago
e6f55d1555
[imagechest] add API support and 'access-token' option ( #4065 )
1 year ago
77abcf5ab3
[gofile] automatically fetch 'website-token' by default
...
the hardcoded token changed yet again
1 year ago
a83983c651
[instagram] add 'order-posts' option ( #4017 , #3993 )
1 year ago
d680623db3
[instagram] add 'order-files' option ( #4017 , #3993 )
1 year ago
5037013e2b
[gofile] update 'website-token' ( #4056 )
1 year ago
a14b63d941
support selecting a domain for '--cookies-from-browser'
...
for example 'gallery-dl --cookies-from-browser firefox/twitter.com'
1 year ago
3ca5dac8b6
extend 'cookies-update' functionality
...
Allow writing cookies to a different file than a given cookies.txt,
making it possible to export cookies imported with --cookies-from-browser
To convert browser cookies to cookies.txt format:
gallery-dl --cookies-fr chromium \
-o cookies-update=cookies.txt \
--no-download \
http://example.org/file.jpg
1 year ago
79f47f98dd
[nana] remove module
...
permanently gone since 2023-03-13
1 year ago
c182094ebf
merge #3748 : [downloader:http] add 'consume-content' option
1 year ago
b81ce381f0
fix backticks for 'extractor.redgifs.format' docs
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1518331927
1 year ago
aaf58a1259
[imgur] document 'client-id' option ( #3937 )
1 year ago
0a7eee3ee0
[deviantart] add 'public' option
1 year ago
75666cf9c3
[danbooru] reduce API requests for fetching extended 'metadata'
...
Instead of using one additional API request per post object (N+1),
this requires only one request per 200-post batch.
1 year ago
00f0233b28
[postprocessor:metadata] add 'skip' option ( #3786 )
2 years ago
a1ca2404f9
add 'globals' instead of overwriting the default ( #3773 )
2 years ago
fcaeaf539c
[downloader:http] handle exceptions while consuming content
2 years ago
df77271438
[downloader:http] add 'consume-content' option
...
* fix connection not being released when the response is neither
successful nor retried
* add the ability to consume the HTTP response body instead of closing
the connection
reference:
https://docs.python-requests.org/en/latest/user/advanced/#body-content-workflow
2 years ago
da9840a39d
[reddit] update 'videos' option ( #3712 )
...
- add 'dash' to directly extract DASH manifest URLs
(was default behavior since a7c79531a7c79531
2 years ago
e1867cf5eb
[misskey] add 'renotes' and 'replies' options
2 years ago
0d142e403c
[szurubooru] add 'tag' and 'post' extractors ( #3583 , #3713 )
2 years ago
d788e6c60c
implement 'globals' option
2 years ago
e480a93337
add 'output.stdout', '.stdin', and '.stderr' options
...
(#1621 , #2152 , #2529 )
Allow setting custom input/output encodings and options
without having to rely on Python's defaults.
2 years ago
b7337d810e
[postprocessor:metadata] add 'sort' and 'separators' options
2 years ago
762a68996b
implement 'archive-pragma' option
2 years ago
bbf0911a46
[e621] implement 'notes' and 'pools' metadata extraction
...
(#3425 )
2 years ago
925b467496
split e621 from danbooru module ( #3425 )
2 years ago
1ae48a54f8
[twitter] add 'transform' option
2 years ago
78d3960a31
[postprocessor:exec] implement archive options ( #3584 )
2 years ago
4572c122d8
document 'extractor.module-sources'
2 years ago
f58215705a
add '-O/--postprocessor-option' command-line option ( #3565 )
2 years ago
3b32671fbd
[deviantart] add extractor for status updates
...
extract user status updates using the '/user/statuses/' endpoint
2 years ago
4e86aaa4db
document 'extractor.headers' and 'extractor.ciphers'
2 years ago
c881548a27
add 'extractor.retry-codes' option ( #3313 )
...
do not retry 429 and 430 by default
2 years ago
512abeb4ae
[booru] add 'url' option
2 years ago
c87bd1a752
[danbooru] extend 'metadata' option
...
make it possible to specify a custom list of metadata includes
2 years ago
c0d7d2be35
[downloader:http] add 'validate' option
2 years ago
cb9d100bac
update Authentication support in docs
...
https://github.com/mikf/gallery-dl/pull/3075#issuecomment-1374723412
2 years ago
362cd6991b
[pixiv] implement 'metadata-bookmark' option ( #3417 )
2 years ago
0895e6afee
merge #3462 : [docs] Update links and fix field typo
2 years ago
4b9bb632f2
[docs] update links
2 years ago
ac78712618
merge #3496 : [docs] update OAuth guide in README.rst
2 years ago
2f31d21509
merge #3455 : [twitter] apply tweet type checks before uniqueness check
2 years ago
64435cc37e
[twitter] update docs
2 years ago
b125fcafc5
update docs/configuration.rst
...
- document metadata.indent
- add concrete mappings to object types
- fix lists for code examples
- fix typos
2 years ago
c94692f39b
[docs] document 'access-token' for Mastodon instances
2 years ago
4fd6da474f
merge #3473 : [twitter] fix crash when using 'expand' and 'syndication'
2 years ago
818f6fa71f
merge #3453 : [twitter] document #1396 and #2712
2 years ago
455481ad76
[twitter] improve documentation
...
This includes:
- The automatic expansion of conversations (#2712 )
- How to retrieve age-restricted content from search results (#1396 )
2 years ago
6933727b45
merge #3483 : [twitter] implement 'syndication=extended'
2 years ago
9116398c1c
[pinterest] add 'domain' option ( #3484 )
...
use input URL domain by default
2 years ago
6853b14be3
[twitter] apply suggestions from code review
...
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
4611237f8c
merge #3457 : [danbooru] extract uploader metadata (if option is set)
2 years ago
0ed5986062
document custom 'output.mode' ( #2529 )
2 years ago
7c8eab8d52
[twitter] implement 'syndication=extended'
...
to be able to fetch extended user metadata
2 years ago
be3286206a
[twitter] assume 'conversation_id' when using syndication
...
not possible to expand replies at the momemt
2 years ago
ce8dbb1ccc
[twitter] fix crash when using 'expand' and 'syndication'
...
caused by KeyError: 'conversation_id_str'
2 years ago
d651d45239
implement specifying ranges in slice notation ( #918 , #2865 )
...
e.g.
- '1:101' or ':101' or ':101:' for files 1 to 100
- '1::2' or '::2' for every second file
- '1:101:5' or ':101:5' for files 1, 6, 11, ..., 91, 96
(the second argument specifies the first index NOT included)
2 years ago
64902f518e
[docs] Update links and fix field typo
2 years ago
5d5ea58887
[twitter] update docs for 'quoted' & 'unique'
2 years ago
f82ee93676
[danbooru] extract uploader metadata (if metadata is set)
2 years ago
1800bd7d14
allow '*-filter' options to be a list of expressions
2 years ago
73ab5d84c0
update docs/configuration.rst
2 years ago
dd4a4a3fa6
[e621] softcode the pagination threshold
2 years ago
d0ad6d0e67
[e621] implement manual pagination mode
2 years ago
6b6f886dcf
[bunkr] update domain ( #3391 )
...
and improve bunkr/app.bunkr handling
2 years ago
dfe7b23579
support Firefox containers for --cookies-from-browser ( #3346 )
2 years ago
80102fa367
[downloader:http] add 'retry-codes' option ( #3313 )
2 years ago
b14b33f19e
Implement `version-metadata` option ( #3201 )
2 years ago
6cb12f513b
[nitter] support quoted Tweets
...
- distinguish between regular and quoted Tweets and media
- add 'quoted' option and metadata field
2 years ago
a41d093bb1
[nitter] add 'retweets' option ( #3278 )
2 years ago
e31d12139c
[nitter] add 'videos' option ( #3279 )
...
with the same semantics as for twitter
2 years ago
501d9bccfe
[artstation] add 'max-posts' option ( #3270 )
2 years ago
b1ad6f2289
[artstation] add 'pro-first' option ( #3273 )
2 years ago
6b97dcf2e0
[postprocessor:metadata] add 'private' option
2 years ago
6379157543
[instagram] use REST API by default
...
regardless of logged in status
2 years ago
7897f68225
[wallhaven] update 'user' extractor ( #3226 )
...
* [wallhaven] update 'user' extractor
* [wallhaven] update 'configuration.rst'
add 'extractor.wallhaven.include' entry
* [wallhaven] add 'wallhaven.include' in gallery-dl.conf
2 years ago
9f06e79868
implement '"user-agent": "browser"' ( #2636 )
2 years ago
8d543b040d
update docs
...
- document '"mode": "jsonl"' for metadata pp
- remove 'username' and 'password' from 'instagram'
- add 'ytdl' example (#3180 )
2 years ago
171262c1b6
[instagram] remove login support
...
broken feature that I cannot get to work anymore
2 years ago
895f36e53b
[postprocessor:metadata] add 'open' and 'encoding' options
2 years ago
870e6a48a0
implement 'http-metadata' option
...
or at least attempt to.
2 years ago
bca9f965e5
[downloader:http] add 'chunk-size' option ( #3143 )
...
and double the previous default from 16384 (2**14) to 32768 (2**15)
2 years ago
c12a97bcde
[postprocessor] add 'post-after' event ( #3117 )
2 years ago
7c6af27eb8
[tumblr] add 'fallback-*' options ( #2957 )
...
specifically 'fallback-delay' and 'fallback-retries'
and change default number of retries to 2 (down from 3)
2 years ago
cf86f68864
[instagram] add 'avatar' extractor ( #929 , #1097 , #2992 )
2 years ago
52d1eb928d
[pixiv] extend 'metadata' option ( #3057 )
...
make it usable for all 'pixiv' extractors
2 years ago
0714274f1f
[instagram] remove 'channel' extractor
2 years ago
f1f89b2436
[tumblr] add 'offset' option
2 years ago
15a4d7696f
[docs] update 'adjust-extensions' explanation ( #3031 )
2 years ago
7f30a0d7a7
add 'path-extended' option ( #3021 )
2 years ago
14717f3fc9
[deviantart] add 'group' option ( #3018 )
...
disabling this option allows to better download from deleted accounts
2 years ago
f7ba19a1c0
[nana] add 'nana' extractors ( #2967 )
2 years ago
4089bceddd
[sankaku] implement 'refresh' option ( #2958 )
2 years ago
aa49bf13d2
[instagram] add 'api' option
2 years ago
ac45ed2764
[skeb] implement 'filters' option ( #2945 )
2 years ago
0393e59535
[newgrounds] add 'games' extractor ( #2955 )
2 years ago
e99a9b2aff
[twitter] improve 'cards-blacklist' ( #2875 )
...
allow blacklisting domains and 'name:domain',
where 'domain' depends on a card's 'vanity_url' value
2 years ago
46fe469c53
[tumblr] implement 'ratelimit' option ( #2919 )
2 years ago
b36125333f
[postprocessor:zip] implement 'files' option ( #2872 )
2 years ago
3cb8327c60
[zerochan] add 'metadata' option ( #2861 )
2 years ago
9745b48830
[tumblr] attempt to fetch high-quality inline images ( #2877 )
...
* [tumblr] attempt to fetch high-quality images (again)
Fixes #1846 , and fixes #1344
* slight refactor
* update configuration.rst entry
2 years ago
4d78ca89db
[twitter] add 'cards-blacklist' option ( #2875 )
2 years ago
f16fbe9f93
document 'extractor.twitter.expand' ( #2848 )
2 years ago
574e38a287
[kemonoparty] add 'favorites' option ( #2826 ) ( #2831 )
...
* [kemonoparty] add 'favorites' option (#2826 )
* [kemonoparty] add regex for the url parameter and fallback on the config
option
* [kemonoparty] simplify
2 years ago
9ea521bec5
[oauth] add host config option ( #2806 )
2 years ago
df1c643dda
[tumblr] attempt to extract full-resolution photos
...
- for photos with apparent width == 2048 or height == 3072
- can be disabled with 'original' option
2 years ago
a69f198ac9
[wallhaven] implement 'metadata' option ( #2803 )
2 years ago
7d1a95ada6
implement 'path-metadata' option ( #2734 )
2 years ago
8cf5981ded
[skeb] add option to download 'article' images ( #1031 )
2 years ago
98af5a0409
[zerochan] implement login with username & password ( #1434 )
2 years ago
0c73914848
[postprocessor:metadata] implement 'mode: modify' ( #2640 )
2 years ago
f3de6b7a87
[postprocessor:metadata] implement 'mode: delete' ( #2640 )
2 years ago
eb68d45544
add global 'warnings' option ( #2762 )
2 years ago
f225247670
[gelbooru] add support for `api_key` and `user_id` ( #2767 )
2 years ago
74865adae5
implement 'format-separator' option ( #2737 )
...
a global option, that servers as a workaround for shortcomings due to
lack of a proper format string parser
2 years ago
117eeefda0
[postprocessor:mtime] add 'value' option ( #2739 )
2 years ago
1d14928bd9
[twitter] ignore previously seen Tweets ( #2712 )
...
occurs primarily for /with_replies results when logged in
2 years ago
4b2a0a0eda
[twitter] implement 'strategy' option ( #2712 )
...
to be able to better control what Tweets get used an returned
for twitter.com/USER URLs.
2 years ago
124b5c6410
[postprocessor:ugoira] enable 'mtime' by default ( #2714 )
2 years ago
3a5d5c3a91
update default User-Agent header to Firefox 102 ESR
...
snd update headers and ciphers for "browser": "firefox"
2 years ago
9d8e99af80
[itaku] support videos ( #1842 )
2 years ago
Mike Fährmann
