e0ac358aa5
[gofile] fix 401 Unauthorized errors ( #2632 )
2 years ago
4bf5bc2403
[weibo] support 'livephoto' entries ( #2146 )
2 years ago
603af48265
implement 'output.ansi' option ( #2628 )
2 years ago
9df4e0f65b
[twitter] disable 'cards' by default
2 years ago
79dce8ae68
[weasyl] implement 'metadata' option ( #2610 )
2 years ago
6b6eb0b8f6
[lolisafe] implement 'domain' option ( #2575 )
2 years ago
61887c895b
implement 'output.colors' options ( #2532 )
2 years ago
84756982e9
[pixiv] implement 'include' option
...
- split 'user' extractor and its 'avatar' and 'background' options into
separate extractors ('artworks', 'avatar', 'background')
- avatars can now be downloaded with
https://www.pixiv.net/en/users/ID/avatar
as URL and will use a proper archive key; similar for backgrounds
- options for the 'user' subcategory must be moved to 'artworks' to have
the same effect as before
2 years ago
f1c853c6ef
[furaffinity] add 'layout' option ( #2277 )
...
to be able to force gallery-dl to parse according to a specific layout
in case its auto-detect fails
3 years ago
86fa412b47
[hitomi] add 'format' option ( #2260 )
...
default is 'webp' since downloading original files is no longer allowed
3 years ago
f2e8aedd74
[twitter] changes to 'cards' option
...
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
when 'cards' is set to "ytdl"
"cards": true --> only download card images
"cards": "ytdl" --> download card images and
use youtube_dl on otherwise unsupported cards
3 years ago
099ed72de7
[hitomi] disable extra 'metadata' by default
...
safes one HTTP request that not needed with default filename settings
3 years ago
f4e3cee6ac
use yt-dlp by default ( #1850 , #2028 )
3 years ago
11a3d96d13
[mangadex] load additional metadata using includes[] directives
...
- always provide 'artist', 'author', and 'group' metadata fields (#2049 )
- remove 'metadata' option
3 years ago
b315a0ecef
[redgifs] update to API v2 ( #1984 )
3 years ago
9156e90f1f
[twitter] add 'pinned' option
3 years ago
02a247f4e5
[deviantart] full resolution for non-downloadable images ( #293 )
...
Many thanks to @Ironchest337 for discovering this method
and providing a well-documented implementation.
3 years ago
8bb442f20d
[redgifs][gfycat] provide fallback URLs ( fixes #1962 )
...
and extend the 'format' option
3 years ago
d0761454b1
implement a download progress indicator ( #1519 )
3 years ago
268c9ca034
update default gallery-dl.conf
3 years ago
e75f2de9da
[deviantart] add 'comments' option ( #1800 )
3 years ago
d3eab417ed
implement a 'path-strip' option
3 years ago
d320ee6251
implement a 'fallback' option ( closes #1770 )
3 years ago
fa6af46756
Added ability to download GIFs instead of mp4 from Luscious and Reactor ( #1701 )
3 years ago
cadfad4eea
[danbooru] add 'external' option ( closes #1747 )
3 years ago
5b1c62bfa9
[furaffinity] add 'external' option ( closes #1492 )
3 years ago
a316e44f8e
[bbc] add 'width' option ( #1706 )
3 years ago
7a0da4f93f
[newgrounds] add 'format' option ( closes #1729 )
3 years ago
74b457b2b5
remove unused entries
3 years ago
dff0da60f9
[ytdl] add 'generic' option ( #1680 )
3 years ago
36ac2197db
[ytdl] add extractor for sites supported by youtube-dl
...
(#1680 , #878 )
Can be used by prefixing any URL with 'ytdl:',
or by setting 'extractor,ytdl.enabled' to 'true'.
3 years ago
e661607e8b
[mangadex] document 'metadata' and 'lang' options ( #1535 )
3 years ago
a751afdfb3
[twitter] change some defaults
...
- 'retweets' option: true -> false
- 'quoted' option : true -> false
i.e. disable downloading tweets from other user's timelines by default
- search directory:
'["{category}", "Search", "{search}"]' ->
'["{category}", "{user[name]}"]'
i.e. change it to the same as other twitter extractors (#1308 )
3 years ago
b5affc62aa
[twitter] rename 'text-only' to 'text-tweets' ( #570 )
3 years ago
a7e4917ee1
[pillowfort] add 'inline' option ( #846 )
...
to support images present in a post's 'content',
but not listed in 'media'.
also separates the file hash present at the beginning
of each 'filename' into its own field.
3 years ago
efa6cc8ec3
[pillowfort] add 'external' option ( #846 )
...
for links to external Twitter posts etc.
3 years ago
4e1d33cea3
update default gallery-dl.conf
...
add (empty) entries for
- deviantart.client-id and deviantart.client-secret
- pixiv.refresh-token
3 years ago
e300da1424
add 'output.skip' option
3 years ago
fa519f9202
[pixiv] change 'translated-tags' option ( #1507 )
...
- rename to 'tags'
- use string-values: "japanese", "translated", "noop"
- remove duplicate entries for "translated" tags
3 years ago
9322c5e43b
[exhentai] restore limit config ( #1487 )
...
This partially reverts commit e9ec91c8
3 years ago
224b883ff4
[danbooru] add option for extended metadata extraction ( #1458 )
...
* [danbooru] add option for extended metadata extraction
* appease linter
* [danbooru] update docs/configuration.rst
* [danbooru] rename extended-metadata -> metadata
3 years ago
dff03a6605
[booru] add an option to extract notes (only gelbooru for now) ( #1457 )
...
* [booru] add an option to extract notes (currently implemented only for gelbooru)
* appease linter
* [gelbooru] rename "text" to "body" in note extraction
* add a code comment about reusing return value of _extended_tags
3 years ago
e9ec91c811
[exhentai] improve image limits check
...
- check if current image is the '509 Bandwidth Exceeded' notification
(https://ehgt.org/g/509.gif or https://exhentai.org/img/509.gif )
- remove 'limits' option
4 years ago
b0438c8f99
Revert "[deviantart] extend 'extra' option"
...
This reverts commit
5ad2b9c82b5c32a7bf5883f465faca#1387 , #1356 )
4 years ago
3378b39719
[twitter] implement 'users' option ( #1337 )
4 years ago
83f465faca
[deviantart] refactor 'extra' ( #1356 )
...
- change its expected type to string
- let users specify a list of sources (stash, posts) or 'all'
4 years ago
7440d1f112
[pixiv] add 'translated-tags' option ( closes #1354 )
...
(a lot more straight forward than I thought ...)
4 years ago
6a28b7045c
update 'docs/gallery-dl.conf'
4 years ago
b2c55f0a72
[sankaku] remove login support
...
The old login method for 'https://chan.sankakucomplex.com/user/login '
and the cookies it produces have no effect on the results from
'beta.sankakucomplex.com'.
4 years ago
c874071f5a
[kissmanga] remove module
4 years ago
3918b69677
remove 'extractor.blacklist' context manager
4 years ago
ac3036ef56
add 'filesize-min' and 'filesize-max' options ( closes #780 )
4 years ago
0e941553ec
[aryion] Add username/password support ( #960 )
...
* Add username/password support to aryion extractor
* Update docs to match
* Fix code style
4 years ago
ecaecc4064
[exhentai] add 'domain' option ( #897 )
4 years ago
9da2bc67f8
[twitter] add option to filter media from quoted tweets ( #854 )
4 years ago
c184cce876
update configuration.rst
...
- fix anonymous links
- update description of 'extractor.twitter.videos'
- document 'extractor.redgifs.format' (#724 )
4 years ago
0138e9c1d8
[twitter] remove 'content' option
...
The text content of each tweet is always available as 'full_text'
4 years ago
dfcf2a2c91
write OAuth token to cache by default ( #616 )
4 years ago
ddc253cf9a
implement a 'path-replace' option ( #662 , #755 )
4 years ago
e19f665a44
[danbooru] change default for 'ugoira' to 'false'
...
Downloading the pre-rendered versions should be a better default
than .zip files with individual frames.
4 years ago
dba87ca99e
[downloader:ytdl] change 'forward-cookies' default to 'false'
...
There are currently no situations where forwarding gallery-dl's cookies
to youtube-dl is necessary, and it only causes problems when forcing
youtube-dl for Twitter video downloads while logged in.
4 years ago
ff47641b13
test whether default/example config files contain valid JSON
4 years ago
5e2974d699
[weibo] add 'videos' option
4 years ago
9f638c2e01
[twitter] add 'replies' option ( closes #705 )
4 years ago
59edcdc822
[hitomi] restore metadata fields from before f33b13a
...
... and add a 'metadata' option to disable
visiting the gallery page and extracting data from it
if this is not needed.
5 years ago
a45fbc38ea
[pixiv] implement 'avatar' option ( #595 , #623 )
5 years ago
74e684e828
[twitter] change default value for 'videos' to 'true'
...
Every other 'videos' option defaulted to 'true', except Twitter.
5 years ago
da0d5f6092
[oauth] add 'port' option ( #604 )
5 years ago
dff33b260c
[reddit] add 'videos' option
5 years ago
6703b8a86b
[blogger] implement video extraction ( closes #587 )
5 years ago
25d5ec4ff3
[twitter] add option to extract TwitPic embeds ( #579 )
5 years ago
48e42e73fb
[reddit] change default value for 'comments' to '0'
5 years ago
bf658fd84b
[vsco] implement 'videos' option
5 years ago
95c90722ee
[instagram] implement 'videos' option ( closes #521 )
5 years ago
389d2d7e38
implement 'cookies-update' option ( #445 )
5 years ago
ee8b654464
[instagram] implement 'highlights' option ( closes #329 )
5 years ago
f02a768b5c
[danbooru] add 'ugoira' option ( #406 )
...
to choose between ZIP archives or converted video files
for Ugoira posts
5 years ago
7c09545f70
[downloader:ytdl] add 'outtmpl' option ( #395 )
5 years ago
5a210991b6
Remove control characters from filesystem paths
...
- add 'path-remove' option to specify the set of characters that
should be removed
- rename 'restrict-filenames' to 'path-restrict'
- #348 , #380
5 years ago
42bb0cdb9b
Update gallery-dl.conf ( #378 )
5 years ago
b7fb93e2b2
[downloader:http] add 'adjust-extensions' option
5 years ago
3957d27d79
[deviantart] add 'quality' option ( #369 )
5 years ago
547ea71463
[downloader.ytdl] add 'forward-cookies' option ( #352 )
...
The "long" name is necessary because just calling it 'cookies' would
clash with how the lookup for '--cookies' is implemented.
5 years ago
b1bea8aaeb
add 'restrict-filenames' option ( #348 )
5 years ago
21991acc49
add 'ciphers' option; update default User-Agent
5 years ago
feb98cf196
[twitter] improve 'content' formatting; add option ( #338 )
...
- include emoticons
- leave newlines intact
- remove pic.twitter.com/ links at the end
5 years ago
f7b5c4c3e7
use values of 'retries' options correctly
...
The RE-tries option now specifies exactly that: the maximum number a
failed HTTP request is re-tried. For example a value of 2 will now
correctly stop after 3 attempts: the initial one + 2 re-tries.
The maximum wait-time now also caps at 30min and increases exponentially
for both extractor.request() and downloader.http.download().
5 years ago
15e4ddf46d
implement custom logging formatter
...
supports custom log message formats for each loglevel and, by
extension, custom ANSI codes and colors for errors and warnings
(#304 )
5 years ago
db3f52881a
add 'mtime' option
5 years ago
e05a96db5e
[deviantart] rename 'stash' to 'extra' ( #302 )
...
'stash' is already used as a name for the StashExtractor and therefore
expected to be a dictionary.
5 years ago
c23bf263fe
[deviantart] rename 'external' to 'stash' ( #302 )
...
restrict extracted URLs to ones from https://sta.sh/ ...
5 years ago
2fb85178da
[deviantart] add 'external' option ( #302 )
...
If a description is available, this will extract URLs from the
description text and try to find Extractors for them.
5 years ago
d33f5a7423
[wallhaven] rewrite
...
- use API
- remove login support, add 'api-key' option
- remove support for "alpha" subdomain - alpha.wallhaven.cc used numeric
IDs that can't be translated to the new ID system
- support direct links to wallpapers
5 years ago
f1893b2b5b
[deviantart] add 'folders' option ( #276 )
5 years ago
4465a3ea68
[kissmanga][readcomiconline] add 'captcha' option ( #279 )
...
to configure how to handle CAPTCHA page redirects:
- either interactively wait for the user to solve the CAPTCHA
- or raise StopExtraction like before
5 years ago
9890bfdf23
[flickr] improve code and metadata
...
- simplify pagination
- add more metadata and slightly change its structure
- convert suitable values to int or list
- move keys from ["photo"] to the base level
- proper video support (#246 )
- rename method and variable names to better fit with other extractors
5 years ago
1e4d351ad3
[danbooru] add authentication support ( closes #151 )
...
... via HTTP Basic Auth with username and "password".
The password value in this case is not the account password itself,
but the"api_key" found in your user profile.
6 years ago
28f9539551
[tumblr] change default values for post types and inline media
6 years ago
5be95034ba
[tumblr] add option to download avatars ( #137 )
6 years ago
2fa28a2609
update default user-agent string ( closes #122 )
6 years ago
Mike Fährmann
