Mike Fährmann
58e0b17211
add note about using '\f' in shells to docs/formatting.md
...
(closes #2398 , closes #2406 )
3 years ago
lingegrino
b3d8b40690
Rename meme.museum ( #2393 )
3 years ago
Mike Fährmann
fc277fa45f
[seiga] require authentication with 'user_session' cookie ( #2372 )
...
Login with username & password would now require entering a 2FA token.
see also 7b009cc893
3 years ago
Mike Fährmann
47cf05c4ab
refactor proxy handling code ( #2357 )
...
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
- add 'downloader.proxy' option
- '-o extractor.proxy=–PROXY_URL -o downloader.proxy=null'
now has the same effect as youtube-dl's '--geo-verification-proxy'
3 years ago
Mike Fährmann
acbb0f5a57
clarify 'metadata.mtime' usage and shortcomings ( #2307 )
3 years ago
Mike Fährmann
bfa5e61900
[patreon] add explicit 'image_large' file type ( #2257 )
...
to allow more control over when and if to download 'large_url' images
4fee3a0e52
forced them to be downloaded
instead of regular images, even though 'large_url' images are most likely
an upscaled version of the original.
3 years ago
Alice
f1cab23724
[skeb] add 'sent-requests' option ( #2322 ) ( #2330 )
...
* Update skeb.py
* Update configuration.rst
* flake8
3 years ago
Mike Fährmann
4385a34e05
[twitter] fix handling of 429 responses ( fixes #2339 )
...
Twitter doesn't return a valid JSON response for 429 errors anymore.
3 years ago
Mike Fährmann
5a50569360
[toyhouse] support 'art' listings ( #1546 , #2331 )
3 years ago
Mike Fährmann
59749555b1
[postprocessor:mtime] add 'event' option ( #2307 )
3 years ago
Mike Fährmann
7958995398
[postprocessor:metadata] implement 'mtime' option ( #2307 )
3 years ago
Mike Fährmann
fdfdc1b614
[kissgoddess] add 'gallery' and 'model' extractors
...
(closes #1052 , #2304 )
3 years ago
Mike Fährmann
79a461a2c1
[mememuseum] add 'tag' and 'post' extractors ( closes #2264 )
3 years ago
Mike Fährmann
254a5b26e0
[twibooru] add extractors for searches, galleries, and posts
...
(#2219 )
3 years ago
Mike Fährmann
f5b2a9fcc6
add 'signals-ignore' option ( #2296 )
3 years ago
David Hoppenbrouwers
b17e2dcf93
[wallpapercave] add extractor for images ( #2205 )
3 years ago
Thomas Jost
a7de819aca
[lightroom] add Lightroom gallery extractor ( #2263 )
3 years ago
Mike Fährmann
563bd0ecf4
[danbooru] inherit from BaseExtractor
...
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283 )
- remove support for old e621 tag search URLs
3 years ago
Mike Fährmann
f1c853c6ef
[furaffinity] add 'layout' option ( #2277 )
...
to be able to force gallery-dl to parse according to a specific layout
in case its auto-detect fails
3 years ago
Mike Fährmann
4e4cb40663
do not sort BaseExtractor instances by name
3 years ago
Mike Fährmann
f351746483
release version 1.20.4
3 years ago
HRXN
f4a7c6eca2
Add two examples for custom instances of gelbooru ( #2262 )
...
* Add two examples for custom instances of gelbooru
- First example uses some custom options at the "gelbooru_v01" basecategory
These apply to all instances used herein
- Second example adds all options to the specific "gelbooru_v02" basecategory of TBIB
These settings are only used by TBIB
* move tbib settings into their own block
3 years ago
Mike Fährmann
cc7dce5755
[sexcom] add 'pins' extractor ( closes #2265 )
3 years ago
Mike Fährmann
02e18f56be
[e621] add 'favorite' extractor ( closes #2250 )
3 years ago
Mike Fährmann
86fa412b47
[hitomi] add 'format' option ( #2260 )
...
default is 'webp' since downloading original files is no longer allowed
3 years ago
Mike Fährmann
492436f936
[twitter] add 'warnings' option ( #2258 )
...
disable reporting any non-fatal errors by default
3 years ago
Mike Fährmann
9a221494c3
[twitter] add 'event' extractor ( closes #2109 )
3 years ago
Mike Fährmann
de754590e0
add --source-address command-line option ( closes #2206 )
3 years ago
Mike Fährmann
f2e8aedd74
[twitter] changes to 'cards' option
...
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
when 'cards' is set to "ytdl"
"cards": true --> only download card images
"cards": "ytdl" --> download card images and
use youtube_dl on otherwise unsupported cards
3 years ago
ImportTaste
c559c49781
formatting.md: add environment variable syntax ( #2065 )
3 years ago
Mike Fährmann
3c79c9b271
document extended blacklist/whitelist syntax ( #2025 )
...
and not just in the commit message of 010d65dc
3 years ago
Mike Fährmann
37beb1298e
[newgrounds] add 'search' extractor ( closes #2161 )
3 years ago
Mike Fährmann
fa7d92f7a9
add docs for 'extractor.generic.enabled'
3 years ago
Mike Fährmann
89bebe1bef
[500px] add 'favorite' extractor ( closes #1927 )
3 years ago
Mike Fährmann
882c614281
add album extractor for lolisafe/chibisafe instances
...
- support bunkr.is (closes #2038 )
- support zz.ht (closes #2105 )
3 years ago
Mike Fährmann
64cf26eaf4
allow specifying sleep-* options as string
...
either as single value or as range: "3.5", "2.1 - 5.0"
3 years ago
Mike Fährmann
62692c6842
[exhentai] add 'source' option
...
setting it to "hitomi" downloads the corresponding gallery from
hitomi.la; might be extended to other sources in the future
3 years ago
Mike Fährmann
099ed72de7
[hitomi] disable extra 'metadata' by default
...
safes one HTTP request that not needed with default filename settings
3 years ago
Mike Fährmann
f587458a3c
[twitter] include '4096x4096' as a default image fallback
...
(closes #2107 , closes #1881 )
3 years ago
Mike Fährmann
299bd2f1f5
[rule34us] add 'tag' and 'post' extractors ( #1527 )
3 years ago
Mike Fährmann
3cf1075d86
[inkbunny] add 'search' extractor ( closes #2094 )
3 years ago
Mike Fährmann
f4e3cee6ac
use yt-dlp by default ( #1850 , #2028 )
3 years ago
Mike Fährmann
f1b142e993
{kemonoparty[ change default 'files' order to attachments,file,inline
...
(#1991 )
3 years ago
Mike Fährmann
addb72e1bb
[reactor] support thatpervert.com ( closes #2029 )
3 years ago
Mike Fährmann
d8d9502e1e
[reactor] inherit from BaseExtractor
3 years ago
Mike Fährmann
f4ea216c95
[shopify] support loungeunderwear.com ( closes #2053 )
3 years ago
Alice
612850438e
[skeb] add 'thumbnails' option ( #2047 ) ( #2051 )
3 years ago
Mike Fährmann
11a3d96d13
[mangadex] load additional metadata using includes[] directives
...
- always provide 'artist', 'author', and 'group' metadata fields (#2049 )
- remove 'metadata' option
3 years ago
Mike Fährmann
c67756e187
[kemonoparty] add 'dms' option ( #2008 )
3 years ago
Mike Fährmann
3842cdcd8f
[formatter] implement 'D' format specifier
...
To be able to parse any string into a 'datetime' object
and format it as necessary.
Example:
{created_at:D%Y-%m-%dT%H:%M:%S%z}
->
"2010-01-01 00:00:00"
{created_at:D%Y-%m-%dT%H:%M:%S%z/%b %d %Y %I:%M %p}
->
"Jan 01 2010 12:00 AM"
with 'created_at' == "2010-01-01T01:00:00+0100"
3 years ago
Mike Fährmann
3a7a19c7b9
[dynastyscans] add 'manga' extractor ( closes #2035 )
3 years ago
Mike Fährmann
9bc83af3a6
[kemonoparty] 'postfile' -> 'file' ( #1991 )
...
to stay consistent with the existing file types for kemono
3 years ago
Mike Fährmann
d433735750
[kemonoparty] skip duplicate files ( #2032 , #1991 , #1899 )
...
Extract the SHA-256 file hash from URLs
and skip files with the same hash in the same post.
- provide a 'hash' metadata field (empty string if not available)
- remove 'patreon-skip-file' option
3 years ago
Mike Fährmann
d4ec245554
[kemonoparty] implement a 'files' option ( #1991 )
...
similar to 8d676151
3 years ago
Mike Fährmann
cfa4876848
[philomena] support furbooru.org ( closes #1995 )
3 years ago
Mike Fährmann
7c50f473e4
[ytdl] document 'cmdline-args' and 'config-file' options ( #1680 )
3 years ago
Mike Fährmann
37c9dedee1
[seisoparty] remove module
3 years ago
Mike Fährmann
b315a0ecef
[redgifs] update to API v2 ( #1984 )
3 years ago
Mike Fährmann
f0fc3b0ba1
[kemonoparty] add 'comments' option ( #1980 )
3 years ago
Mike Fährmann
4c49174579
[mangakakalot] update domain and fix extraction
3 years ago
ImportTaste
69cd573809
add special type format strings to docs ( #1987 )
3 years ago
Mike Fährmann
9156e90f1f
[twitter] add 'pinned' option
3 years ago
Mike Fährmann
06b414c9a3
[redgifs] 'gfyId' -> 'id' ( #1984 )
3 years ago
Alice
bfd7401b1e
[skeb] add 'user' and 'post' extractors ( #1031 ) ( #1971 )
...
* Create skeb.py
* Update __init__.py
* Update supportedsites.py
* Update supportedsites.md
* Update supportedsites.py
* Update skeb.py
3 years ago
Mike Fährmann
02a247f4e5
[deviantart] full resolution for non-downloadable images ( #293 )
...
Many thanks to @Ironchest337 for discovering this method
and providing a well-documented implementation.
3 years ago
Mike Fährmann
c19e762fdf
[vk] add 'album' extractor ( #474 , fixes #1952 )
...
todo: better metadata for albums
3 years ago
Mike Fährmann
8bb442f20d
[redgifs][gfycat] provide fallback URLs ( fixes #1962 )
...
and extend the 'format' option
3 years ago
Mike Fährmann
8d676151b7
[patreon] implement 'files' option ( #1935 )
3 years ago
Mike Fährmann
80314e981b
small corrections to gallery-dl-example.conf
...
use "'_reddit' in locals()"
instead of ""locals().get('_reddit')"
since the reddit metadata dict could be empty. The latter expression
would evaluate to False in that case, even though it has a reddit parent.
3 years ago
Mike Fährmann
3ecf4a9729
update gallery-dl-example.conf
...
- add comments
- remove/replace some older options
- display more post processor settings
3 years ago
Mike Fährmann
50098762e3
[nhentai] add 'tag' extractor ( closes #1950 )
3 years ago
Mike Fährmann
fe6ce5495a
[kemonoparty] add 'discord' extractor ( #1827 , #1940 )
3 years ago
Mike Fährmann
918fc9974d
[picarto] add 'gallery' extractor ( closes #1931 )
3 years ago
Mike Fährmann
e33125ad39
[pixiv] add 'sketch' extractor ( #1497 )
3 years ago
Mike Fährmann
b93915c113
[inkbunny] add 'pool' extractor ( #1937 )
3 years ago
Mike Fährmann
373d3e1c57
[seisoparty] implement login with username & password ( #1906 )
3 years ago
Mike Fährmann
7c5f62b453
[seisoparty] add 'favorite' extractor ( #1906 )
3 years ago
Mike Fährmann
d93b5474c3
[mangadex] update parameter handling for API requests
...
- move common parameters into '_pagination()'
- add 'ratings' (#1908 ) and 'api-parameters' options
3 years ago
Mike Fährmann
cd66c3c415
[twitter] add 'size' option ( #1881 )
3 years ago
Mike Fährmann
df8050b81d
[postprocessor:compare] add 'equal' option ( #1592 )
...
Move functionality from cdd72e14
to its own option,
where it can be used with any 'action'
3 years ago
Mike Fährmann
cdd72e1413
[postprocessor:compare] extend 'action' option ( #1592 )
...
allow setting it to "abort", "terminate", or "exit" as with 'skip'
3 years ago
Mike Fährmann
e0bdacd932
[fappic] add 'image' extractor ( closes #1898 )
3 years ago
Mike Fährmann
8c29a6e491
add example for 'url-metadata' option ( closes #1884 )
3 years ago
Mike Fährmann
d0761454b1
implement a download progress indicator ( #1519 )
3 years ago
Mike Fährmann
140a7bfdcd
update --clear-cache command in deviantart client-id instructions
3 years ago
Mike Fährmann
e69ee41f25
implement 'page-reverse' option ( #1854 )
3 years ago
Mike Fährmann
e4684c5cb9
[desktopography] simplify ( #1740 )
3 years ago
Mike Fährmann
7b05600056
[pinterest] only mention cookies for authentication ( #1856 )
...
regular login is broken due to CAPTCHA
3 years ago
Mike Fährmann
266ed9b62e
[aryion] add 'tag' extractor ( closes #1849 )
3 years ago
Mike Fährmann
c9e6693530
allow specifying a minimum/maximum for 'sleep-*' options ( #1835 )
...
for example '"sleep-request": [5.0, 10.0]' to wait between 5 and 10
seconds between each HTTP request
3 years ago
Mike Fährmann
bd845303ad
implement a way to shorten filenames with east-asian characters
...
(#1377 )
Setting 'output.shorten' to "eaw" (East-Asian Width) uses a slower
algorithm that also considers characters with a width > 1.
3 years ago
Mike Fährmann
7bbb1f92d7
[gelbooru_v02] add 'favorite' extractor ( closes #1834 )
3 years ago
Mike Fährmann
4ec11af6a4
[kemonoparty] implement login with username & password ( #1824 )
3 years ago
Mike Fährmann
83bbb628d8
[kemonoparty] add 'favorite' extractor ( #1824 )
3 years ago
Mike Fährmann
44f572c27f
[deviantart] implement a 'auto-unwatch' option ( #1466 , #1757 )
3 years ago
Mike Fährmann
268c9ca034
update default gallery-dl.conf
3 years ago
Mike Fährmann
3e36543c98
[nhentai] add 'favorite' extractor ( #1814 )
3 years ago
Mike Fährmann
96fec14ef7
[deviantart] rename 'watch' option to 'auto-watch'
...
(#1466 , #1757 )
Similar reason as in e05a96db
.
'watch' is already used by the WatchExtractor class.
3 years ago
Mike Fährmann
e75f2de9da
[deviantart] add 'comments' option ( #1800 )
3 years ago
Mike Fährmann
6ce16c6d31
[deviantart] add 'tag' extractor ( closes #1803 )
3 years ago