99b76628f7
implement '-e/--error-file' command-line option ( #4732 )
...
copying per-URL options from regular, read-only input files
does currently not work
10 months ago
1770c31e63
[urlgalleries] add support
10 months ago
bdb3ce7217
[foolslide] remove 'powermanga.org'
10 months ago
4e15513415
[docs] fix 'Example' -> 'Default' from b43be672
...
copy-paste mistake
10 months ago
b43be67206
[exhentai] add 'gp' option ( #4576 )
10 months ago
cb9a1176e6
[pixeldrain] add 'api-key' option ( #4839 )
10 months ago
e1404827a6
[pixeldrain] add 'file' and 'album' extractors ( #4839 )
10 months ago
ce7c4cb544
merge #4832 : [tmohentai] add 'gallery' extractor ( #4808 )
10 months ago
c4a201ed42
[tmohentai] simplify + tests
10 months ago
e17a48fe56
[blogger] inherit from BaseExtractor
...
- support www.micmicidol.club (#4759 )
10 months ago
aea15f6d17
add 'metadata-extractor' option ( #4549 )
10 months ago
34a387b6e2
support 'metadata-*' names for '*-metadata' options
...
For example, instead of 'url-metadata' it is now also possible to use
'metadata-url' as option name.
- metadata-url
- metadata-path
- metadata-http
- metadata-version
- metadata-parent
10 months ago
286d0cb098
[tmohentai] add support
10 months ago
07cb584231
[behance] add 'modules' option ( #4799 )
10 months ago
8bf161e574
reorder post processing options shown by --help
10 months ago
168331d147
replace '--ugoira-conv' etc with a general '--ugoira'
...
update --ugoira webm to use the same FFmpeg args as Danbooru
--ugoira-conv -> --ugoira vp8
--ugoira-conv-lossless -> --ugoira vp9-lossless
--ugoira-conv-copy -> --ugoira copy
(--ugoira-conv and co still work as before,
but --help now lists only --ugoira)
10 months ago
97357e65ee
replace '--mtime-from-date' with a more generic '--mtime'
...
--mtime-from-date -> --mtime date
for the same effect as before
(--mtime-from-date also still works,
but --help now lists only --mtime)
10 months ago
51e377e612
add '--cbz' command-line option
10 months ago
4700051562
rework and extend input file processing ( #4732 )
...
- add 2 command-line options to modify input file contents
- -I/--input-file-comment
- -x/--input-file-delete
- implement InputManager class
- move code from util.py to __init__.py
(mainly to avoid import cycles)
10 months ago
7a0f145cbe
[twitter] ignore promoted Tweets ( #4790 , #3894 )
...
add 'ads' option in case someone actually wants to
download promoted content for whatever reason
10 months ago
f4e61fd1d5
reword 'reddit.client-id' instructions ( #4749 )
11 months ago
007c433677
[patreon] support 'id:<campaign_id>' in place of a user name
...
https://patreon.com/id:12345
… and remove 'campaign-id' config option
11 months ago
a09df34bcf
merge #4714 : [4archive] add 'thread' and 'board' extractors
...
(#1262 , #2418 , #4400 , #4710 )
11 months ago
d2874c7724
[4archive] docs/supportedsites
11 months ago
6766877524
merge #4693 : [reddit] support Reddit Mobile share links
11 months ago
1042278bec
[misskey] support 'misskey.design' ( #4713 )
11 months ago
12a800ce21
[patreon] improve 'campaign_id' handling ( #4699 , #4715 )
...
- add ways to directly specify a 'campaign_id'
- 'campaign-id' config option
- 'c' or 'campaign_id' URL query parameter
- more descriptive error messages
- show 'campaign_id' value in debug log
11 months ago
b68aad3dab
[reddit] implement Reddit Mobile share links
11 months ago
95a74be2a5
release version 1.26.1
11 months ago
0d52b775cb
[kemonoparty] add 'revisions' option ( #4498 , #4597 )
11 months ago
390d14dbcc
[chevereto] support 'img.kiwi' and 'deltaporno.com' ( #4664 , #1381 )
11 months ago
2911ed1240
[chevereto] add generic extractors ( #4664 )
...
- support jpgfish
- support pixl.li / pixl.is (#3179 , #4357 )
11 months ago
2d41702762
[deviantart] implement '"group": "skip"' ( #4630 )
11 months ago
992e86ec94
[deviantart] disable 'jwt' ( #4652 )
11 months ago
2974b8e3c8
[moebooru] add 'metadata' option ( #4646 )
...
for extended 'pool' metadata
11 months ago
67ba4ee842
[pp:exec] support more replacement fields for '--exec' ( #4633 )
...
- {_directory}
- {_filename}
- {_path} (alias for {})
12 months ago
84fbbd96aa
[shimmie2] remove 'meme.museum'
12 months ago
0c5d8b1505
[deviantart] re-add 'quality' option and 'intermediary' transform
1 year ago
20d1683c47
[deviantart] fix JWT replacement ( #293 , #4548 , #4563 )
...
And again, a huge thank you to @Ironchest337
for discovering this.
1 year ago
82296b1f05
[reddit] add 'previews' option ( #4322 )
...
another way to disable this new behavior
1 year ago
9d8317d963
[deviantart] disable JWT updates ( #4548 , #4563 )
...
back to lowres images ...
1 year ago
3ecb512722
send Referer headers by default
1 year ago
6178177227
[twitter] fix '_extractor' of following results ( #4536 )
...
regression from 20ed647f
1 year ago
f2de70f254
[gfycat] remove module
1 year ago
6eca1fab9b
[gelbooru_v02] support 'xbooru.com' ( #4493 )
1 year ago
23bac772f2
[jpgfish] update domain to 'jpg1.su' ( #4494 )
1 year ago
ceb59e176f
fix default Firefox user agent string
...
note to self: do not trust some random third-party website
1 year ago
a783c4f0fe
[pornhub] add 'gif' support ( #4463 )
1 year ago
d12a5e440a
update docs/supportedsites
1 year ago
5ed245317d
[exhentai] add 'fav' option ( #4409 )
...
The name 'favorite' is already taken as extractor subcategory
1 year ago
410f783a33
implement 'subconfigs' option ( #4440 )
1 year ago
c1c73c0b0e
[pp:ugoira] add '"framerate": "uniform"' ( #4421 )
1 year ago
70bdf32a88
[pp:ugoira] extend 'ffmpeg-output' ( #4421 )
...
- when setting this option to a string value,
pass -hide-banner and -loglevel to FFmpeg
- change default to "error"
1 year ago
84089c7368
[docs] add note about 'path-restrict' implementation
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1671892152
1 year ago
f9fb276e81
[postprocessor] add 'prepare-after' event ( #4083 )
1 year ago
0ef1fcab20
[postprocessor] update 'finalize' events
...
Add 'finalize-error' and 'finalize-success' events that trigger
depending on whether error(s) did or did not happen.
'finalize' itself now always triggers regardless of error status.
(was supposed to have the same behavior as the new 'finalize-success')
1 year ago
9d67655397
add "ascii+" as a special 'path-restrict' value ( #4371 )
1 year ago
1ece3b92ff
[mangadex] allow multiple values for 'lang' ( #4093 )
...
This was already possible by setting 'lang' to a list of strings,
but now it can also be done as a more command-line friendly string.
-o lang=fr,it
1 year ago
c6d70c7483
[docs] fix typo ( #4350 )
1 year ago
7cd5130e88
[docs] small fixes/updates
1 year ago
54d974deb0
add 'python' post processor
...
similar to 'exec' but calls a Python function
1 year ago
ceebacc9e1
remove 'pyopenssl' option
1 year ago
0ba8d1f168
merge #4312 : [redgifs] add 'niches' extractor
1 year ago
c5565f79f7
merge #4096 : [danbooru] add support for booru.borvar.art instance
1 year ago
63326e3168
[danbooru] add tests for booruvar
1 year ago
5171d8975c
[E621] support 'e6ai.net' ( #4320 )
1 year ago
90231f2d5a
[twitter] add 'tweet-endpoint' option ( #4307 )
...
use the newer TweetResultByRestId only for guests by default
1 year ago
20ed647f6f
[twitter] add 'user' extractor and 'include' option ( #4275 )
1 year ago
86be197d11
[twitter] remove '/search/adaptive.json'
1 year ago
12cd85658b
[redgifs] add 'niches' extractor
1 year ago
ffbbbd3baf
[gelbooru_v01] 'vidyart' -> 'vidyart2'
1 year ago
2dd6942d1c
[jpgfish] update domain to 'jpeg.pet'
1 year ago
1137b89ed4
[lineblog] remove module
...
"LINE BLOGは2023年6月29日をもちましてサービスを終了いたしました"
1 year ago
86560fe0cd
[bcy] remove module
...
"The website was shut down on July 12, 2023"
https://danbooru.donmai.us/wiki_pages/bcy
1 year ago
fc43c74694
[docs] update reddit client-id instructions ( #4292 )
...
add clearing cached access tokens and getting a new refresh token
1 year ago
f079d9a703
[reddit] notify users about registering an oauth application
...
(#4292 , #4253 , #3943 )
1 year ago
7444fc125b
[gfycat] implement login support ( #3770 , #4271 )
...
For the record: '/webtoken' and '/weblogin' are not the same ...
1 year ago
5b59a0d143
update default User-Agent header to Firefox 115 ESR
1 year ago
1bf9f52c99
[twitter] add 'ratelimit' option ( #4251 )
1 year ago
f86fdf64a6
[twitter] use GraphQL search by default ( #4264 )
1 year ago
6ae3101fd0
[mangapark] add 'source' option ( #3969 )
1 year ago
c45a913bfd
[flickr] add 'exif' option
1 year ago
ccbc1a1d55
[flickr] add 'metadata' option ( #4227 )
1 year ago
c1cce4a80b
[twitter] extend 'conversations' option ( #4211 )
1 year ago
f50b81d3e9
[docs] add novel-related keys to 'pixiv.include' ( #4111 )
1 year ago
ce93c460a6
[formatter] implement 'H' conversion ( #4164 )
...
to remove HTML tags and unescape HTML entities
1 year ago
deff3b434d
[vipergirls] implement login support ( #4166 )
1 year ago
aad5e6490c
merge #4159 : [bunkr] update domain to bunkrr.su
1 year ago
e0522ffb3d
[bunkr] update
1 year ago
e04796e04b
merge #3447 : [jschan] add generic extractors for jschan imageboards
1 year ago
a90974178d
[jpgfish] update domain to 'jpg.pet' ( #4138 )
1 year ago
54cf1fa3e7
[twitter] use GraphQL search endpoint ( #3942 )
...
for guest users; selectable with 'search-endpoint' option.
adapted from 9c7b888ffa
1 year ago
0cf7282fa0
[pixiv] add 'full-series' option for novels ( #4111 )
1 year ago
b286efefcc
[pixiv] add 'novel-bookmark' extractor ( #4111 )
1 year ago
28f6487c64
[instagram] add 'metadata' option ( #3107 )
1 year ago
58f7480d46
[lensdump] update
...
- update docs/supportedsites.md
- add GPL2 header
- use BASE_PATTERN
- improve LensdumpImageExtractor
1 year ago
148bdc04a4
merge #2719 : [jpgfish] add 'jpgfish' extractors
1 year ago
2b1f875ef4
[jpgchurch] update to 'jpgfish'
1 year ago
5f86527cbe
merge #2781 : [mangaread] Add Mangaread extractor
1 year ago
f3cca50b9e
[mangadex] update links to API docs
1 year ago
65a9f4b124
merge #3950 : [misskey] add 'favorite' extractor
1 year ago
c76f0f3a1b
[misskey] update
...
- rename to 'MisskeyFavoriteExtractor'
- add 'access-token' option to docs
- add test URLs for other instances
- simplify 'pattern'
1 year ago
3fca455b82
[pixiv] add 'embeds' option ( #1241 )
1 year ago
9810ab35af
[docs] update formatting.md
...
- add element access with quotes
- fix \f formatting
- remove note about typing \f in a shell (9e6c9813
1 year ago
69865dcc05
[formatter] implement slicing strings as bytes ( #4087 )
...
prefixing a slice '[10:30]' with a lowercase b '[b10:30]' encodes
the string to bytes in filesystem encoding before applying the slice
1 year ago
e6f55d1555
[imagechest] add API support and 'access-token' option ( #4065 )
1 year ago
77abcf5ab3
[gofile] automatically fetch 'website-token' by default
...
the hardcoded token changed yet again
1 year ago
e3fed9bd17
[tcbscans] update domain to 'tcbscans.com' ( #4080 )
1 year ago
a83983c651
[instagram] add 'order-posts' option ( #4017 , #3993 )
1 year ago
d680623db3
[instagram] add 'order-files' option ( #4017 , #3993 )
1 year ago
5037013e2b
[gofile] update 'website-token' ( #4056 )
1 year ago
20dc13f832
[pixiv] initial 'novel' support ( #1241 , #4044 )
...
supported URLs are
- https://www.pixiv.net/novel/show.php?id= <ID>
- https://www.pixiv.net/novel/series/ <ID>
- https://www.pixiv.net/en/users/ <ID>/novels
1 year ago
1406f7125f
[4chanarchives] add 'thread' and 'board' extractors ( #4012 )
1 year ago
285391df43
add '-C' as short option for '--cookies'
...
and put cookie options into their own section
1 year ago
b9b1cdd71b
add '--cookies-export' command-line option
1 year ago
8520de57f0
[imgur] add 'favorite-folder' extractor ( #4016 )
1 year ago
a14b63d941
support selecting a domain for '--cookies-from-browser'
...
for example 'gallery-dl --cookies-from-browser firefox/twitter.com'
1 year ago
3ca5dac8b6
extend 'cookies-update' functionality
...
Allow writing cookies to a different file than a given cookies.txt,
making it possible to export cookies imported with --cookies-from-browser
To convert browser cookies to cookies.txt format:
gallery-dl --cookies-fr chromium \
-o cookies-update=cookies.txt \
--no-download \
http://example.org/file.jpg
1 year ago
79f47f98dd
[nana] remove module
...
permanently gone since 2023-03-13
1 year ago
0e74df1de8
[420chan] remove module
...
offline since 2022-06-01
1 year ago
0c46758a93
[foolslide] remove 'sensescans.com'
...
group moved to mangadex
https://mangadex.org/group/1071e71d-cc55-4fa6-81d1-4b5913a2fde5/sense-scans
1 year ago
a08fdfac6e
[foolfuuka] add 'archive.palanq.win'
1 year ago
1870df8b23
[foolfuuka] remove 'tokyochronos.net'
1 year ago
ef4e2d8178
[foolfuuka] remove 'archive.alice.al'
1 year ago
c182094ebf
merge #3748 : [downloader:http] add 'consume-content' option
1 year ago
7865067d19
[shimmie2] add generic extractors for Shimmie2 sites ( #3734 )
...
add support for
- loudbooru.com (#3734 )
- booru.cavemanon.xyz (#3734 )
- giantessbooru.com (#943 )
- tentaclerape.net
1 year ago
28419bf45a
[itchio] add 'game' extractor ( #3923 )
1 year ago
5297ee0cd9
[tumblr] add 'day' extractor ( #3951 )
1 year ago
b81ce381f0
fix backticks for 'extractor.redgifs.format' docs
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1518331927
1 year ago
aaf58a1259
[imgur] document 'client-id' option ( #3937 )
1 year ago
6f1e34ec69
[vipergirls] add 'thread' and 'post' extractors
...
(#731 , #2720 , #3812 )
1 year ago
81bd2af83e
[2chen] update domain to sturdychan.help
1 year ago
5b635f2317
[imxto] add 'gallery' extractor ( #1289 )
1 year ago
d253a3c542
merge #3841 : [urlshortener] add support for bit.ly & t.co
1 year ago
5e63942b37
[urlshortener] update
1 year ago
0a7eee3ee0
[deviantart] add 'public' option
1 year ago
75666cf9c3
[danbooru] reduce API requests for fetching extended 'metadata'
...
Instead of using one additional API request per post object (N+1),
this requires only one request per 200-post batch.
1 year ago
71b26adb9b
[urlshortener] add tinyurl.com as an example
2 years ago
421db26aff
[bunkr] update domain to 'bunkr.la'
2 years ago
9e2a945013
[urlshortener] add support for bit.ly & t.co
2 years ago
89a67c45e0
[nitter] support nitter.it ( #3819 )
2 years ago
197882cf12
[twitter] add 'hashtag' extractor ( #3783 )
2 years ago
46fdf46f21
[formatter] support loading an f-string from a template file
...
"\fTF ~/path/to/file.txt"
2 years ago
1a4d4a799b
[formatter] support filesystem paths for \fM
2 years ago
00f0233b28
[postprocessor:metadata] add 'skip' option ( #3786 )
2 years ago
a1ca2404f9
add 'globals' instead of overwriting the default ( #3773 )
2 years ago
17bd053d94
[hiperdex] fix extraction ( #3768 )
2 years ago
fcaeaf539c
[downloader:http] handle exceptions while consuming content
2 years ago
817fc0fbd1
[nitter] remove nitter.pussthecat.org
...
"Shutdown"
2 years ago
df77271438
[downloader:http] add 'consume-content' option
...
* fix connection not being released when the response is neither
successful nor retried
* add the ability to consume the HTTP response body instead of closing
the connection
reference:
https://docs.python-requests.org/en/latest/user/advanced/#body-content-workflow
2 years ago
da9840a39d
[reddit] update 'videos' option ( #3712 )
...
- add 'dash' to directly extract DASH manifest URLs
(was default behavior since a7c79531a7c79531
2 years ago
baf41d7437
[misskey] update ( #3717 )
...
- add module docstring
- add options to docs/gallery-dl.conf
2 years ago
e1867cf5eb
[misskey] add 'renotes' and 'replies' options
2 years ago
a95b5e0d8e
[misskey] add misskey extractors
2 years ago
0d142e403c
[szurubooru] add 'tag' and 'post' extractors ( #3583 , #3713 )
2 years ago
075c965512
add '--config-create' command-line option
...
(#2333 )
2 years ago
632d5d7745
allow loading config files in TOML format with --config-toml
2 years ago
d788e6c60c
implement 'globals' option
2 years ago
b14f8d5817
[gelbooru] add 'favorite' extractor ( #3704 )
...
requires logged in cookies to work
2 years ago
e480a93337
add 'output.stdout', '.stdin', and '.stderr' options
...
(#1621 , #2152 , #2529 )
Allow setting custom input/output encodings and options
without having to rely on Python's defaults.
2 years ago
6ed4309aba
[deviantart] add 'gallery-search' extractor ( #1695 )
2 years ago
e1df7f73b1
[deviantart] add 'search' extractor
...
(#538 , #1264 , #2954 , #2970 , #3577 )
Requires login to fetch any results, since the API endpoint raises an
error for not logged in requests.
TODO: parse HTML search results
2 years ago
bb11c2a576
merge #3662 : [redgifs] add 'collection' extractors
2 years ago
884f1848d6
[redgifs] fix syntax for older Python versions
...
and update docs/supportedsites
2 years ago
2bd8f2f4bd
[pornpics] add 'search' and 'tag' extractors
...
(#263 , #3544 , #3654 )
2 years ago
79bc82884c
[pornpics] add 'gallery' extractor ( #263 , #3544 , #3654 )
2 years ago
bd5d08abbc
[catbox] add 'file' extractor ( #3570 )
2 years ago
2a53e6445c
[bunkr] update domain ( #3636 )
2 years ago
b7337d810e
[postprocessor:metadata] add 'sort' and 'separators' options
2 years ago
762a68996b
implement 'archive-pragma' option
2 years ago
bbf0911a46
[e621] implement 'notes' and 'pools' metadata extraction
...
(#3425 )
2 years ago
925b467496
split e621 from danbooru module ( #3425 )
2 years ago
1ae48a54f8
[twitter] add 'transform' option
2 years ago
78d3960a31
[postprocessor:exec] implement archive options ( #3584 )
2 years ago
0f7e6c422a
merge #3596 : [shopify] support ohpolly.com
2 years ago
fcf7030b85
[shopify] support ohpolly.com
2 years ago
4572c122d8
document 'extractor.module-sources'
2 years ago
c2bc70593e
implement ability to load external extractor classes
...
- -X/--extractors
- extractor.module-sources
2 years ago
13a90969c7
merge #3575 : [nudecollect] add 'image' and 'album' extractors
2 years ago
abc3619feb
[lexica] add 'search' extractor ( #3567 )
2 years ago
f58215705a
add '-O/--postprocessor-option' command-line option ( #3565 )
2 years ago
2a5903dc16
[nudecollect] add 'image' and 'album' extractors
2 years ago
58c008e30a
[hiperdex] update domain ( #3572 )
2 years ago
caae8fefe1
merge #3541 : [deviantart] add extractor for status updates
2 years ago
3b32671fbd
[deviantart] add extractor for status updates
...
extract user status updates using the '/user/statuses/' endpoint
2 years ago
5cb263fdd2
[wikifeet/wikifeetx] add 'gallery' extractor
2 years ago
4e86aaa4db
document 'extractor.headers' and 'extractor.ciphers'
2 years ago
c881548a27
add 'extractor.retry-codes' option ( #3313 )
...
do not retry 429 and 430 by default
2 years ago
512abeb4ae
[booru] add 'url' option
2 years ago
c87bd1a752
[danbooru] extend 'metadata' option
...
make it possible to specify a custom list of metadata includes
2 years ago
c0d7d2be35
[downloader:http] add 'validate' option
2 years ago
4833ec323e
[imagefap] add 'folder' extractor ( #3504 )
2 years ago
cb9d100bac
update Authentication support in docs
...
https://github.com/mikf/gallery-dl/pull/3075#issuecomment-1374723412
2 years ago
362cd6991b
[pixiv] implement 'metadata-bookmark' option ( #3417 )
2 years ago
0895e6afee
merge #3462 : [docs] Update links and fix field typo
2 years ago
4b9bb632f2
[docs] update links
2 years ago
d1dd52349a
merge #3189 : [tcbscans] add 'chapter' and 'manga' extractors
2 years ago
ac78712618
merge #3496 : [docs] update OAuth guide in README.rst
2 years ago
2f31d21509
merge #3455 : [twitter] apply tweet type checks before uniqueness check
2 years ago
64435cc37e
[twitter] update docs
2 years ago
Mike Fährmann
