Mike Fährmann
0eea424a52
update supportedsites.rst
4 years ago
Federico Ravasio
25297815bc
[photovogue] added portfolio extractor ( #1253 )
4 years ago
Mike Fährmann
7a096c443f
[unsplash] add 'format' option ( #1197 )
4 years ago
v-delta
e707e060cb
[vipr] add image extractor ( #1258 )
...
* [vipr] add image extractor
Adds support for images hosted on https://vipr.im
* Fix codestyle issues
4 years ago
Mike Fährmann
89a2bcbb2d
[furaffinity] add 'descriptions' option ( #1231 )
4 years ago
Mike Fährmann
36f281330a
[newgrounds] fix flash file extraction ( closes #1257 )
...
… and add a 'flash' option to choose between flash and video formats.
4 years ago
Mike Fährmann
534194bf92
[unsplash] add extractors ( #1197 )
...
for
- single photos (/photos/ID)
- user profiles (/@USER)
- user likes (/@USER/likes)
- search results (/s/photos/SEARCH)
4 years ago
Mike Fährmann
c008cb5100
[pixiv] add 'related' option ( #1237 )
4 years ago
Mike Fährmann
01b9ccd4de
[derpibooru] use the "Everything" filter by default ( #1243 )
...
when neither 'api-key' nor 'filter' are set
4 years ago
Mike Fährmann
5d4494b15f
add "ascii" as a special 'path-restrict' value
4 years ago
Mike Fährmann
20bd9cd296
[wikiart] add extractor for single paintings ( closes #1233 )
...
There is no API endpoint for single paintings from what I can tell,
so this uses the site's search.
4 years ago
Mike Fährmann
fa8ee6eac4
[derpibooru] add search and gallery extractors ( #862 )
4 years ago
Mike Fährmann
8a88025dc4
[pinterest] support generic user URLs ( #1205 )
...
i.e. https://www.pinterest.com/USERNAME
also renames 'BoardsExtractor' to 'UserExtractor'
4 years ago
Mike Fährmann
56b460dcea
[foolfuuka] add 'search' extractors ( #1174 )
4 years ago
Mike Fährmann
fb64183d53
[foolfuuka] add 'board' extractors ( closes #1044 )
4 years ago
Mike Fährmann
b0beed7a06
[sankaku] add support for book searches ( closes #1204 )
4 years ago
Mike Fährmann
459a0af4f8
[sankaku] add support for sankaku.app URLs ( closes #1193 )
4 years ago
Mike Fährmann
371e9ca6df
[pinterest] implement video support ( closes #1189 )
4 years ago
Mike Fährmann
bf629a2818
[instagram] add 'include' option ( closes #1180 )
...
Split the functionality of the old 'user' extractor into separate
'posts' and 'highlights' extractors, which respond to virtual URLs
('/<user>/posts' and '/<user>/highlights')
4 years ago
Mike Fährmann
212ae0c399
[mangapanda] remove module
...
site now redirects to mangareader.net
4 years ago
Mike Fährmann
465015f75a
[sankaku] reimplement login support ( #1176 , #1182 )
4 years ago
Mike Fährmann
b2c55f0a72
[sankaku] remove login support
...
The old login method for 'https://chan.sankakucomplex.com/user/login '
and the cookies it produces have no effect on the results from
'beta.sankakucomplex.com'.
4 years ago
Mike Fährmann
cc15fbe71a
[moebooru] add generalized extractors for moebooru sites
...
- add support for sakugabooru.com (closes #1136 )
- add support for lolibooru.moe (closes #1050 )
This allows users to dynamically add support for moebooru/myimouto
based sites by adding an entry to their config file
(like for foolslide, foolfuuka, etc)
For example:
{
"extractor": {
"moebooru": {
"new-site-1": {"root": "https://site1.net "},
"new-site-2": {"root": "https://www.site2.moe "}
}
}
}
4 years ago
Mike Fährmann
0bc492c0fa
add docs for 'event' and 'filename' options
...
from 9c3568c3
and ca59bd69
4 years ago
Mike Fährmann
9b1bd09454
change 'extension-map' default
...
Replace all JPEG filename extensions with 'jpg'.
4 years ago
Mike Fährmann
de0c57886d
[twitter] add 'list-members' extractor ( closes #1096 )
4 years ago
Mike Fährmann
bc7b1d91bc
fix rST markup in configuration.rst
...
[ci skip]
4 years ago
Mike Fährmann
e3480bc8de
implement 'extension-map' option ( #318 )
4 years ago
Mike Fährmann
98a4d86a01
[sankakucomplex] extract videos and embeds ( closes #308 )
4 years ago
Mike Fährmann
d83b95fd28
[postprocessor:metadata] accept a string-list for 'content-format'
...
(closes #1080 )
4 years ago
Mike Fährmann
5bcf28de93
add a 'extractor.modules' option
4 years ago
Mike Fährmann
18213dc5ba
release version 1.15.2
4 years ago
Mike Fährmann
1686dc1757
[twitter] support media from Cards ( #1005 , #937 )
...
Can be enabled with 'extractor.twitter.cards', but for now disabled by
default because cards can redirect to rather large videos from YouTube
or Twitch.
4 years ago
Mike Fährmann
bac8af69e8
update configuration.rst
...
- add some lines to better explain post processor usage
- syntax highlighting for JSON blocks and other smaller stuff
4 years ago
Mike Fährmann
05d7009cc6
rename 'Authentication' entries in supportedsites.rst
...
- change 'Optional' to 'Supported'
- use 'OAuth' and 'Cookies' in their own
- add link to weasyl API key option
4 years ago
Mike Fährmann
d1c6d78477
fix rST markup in configuration.rst
4 years ago
Mike Fährmann
b8daabc3ca
[pinterest] implement login support ( closes #1055 )
...
being logged allows access to secret/protected boards
4 years ago
Zanny
3ec60e894a
[weasyl] api-key authentication ( #1057 )
...
* [weasyl] support api keys
* [weasyl] document api-key authentication
* [weasyl] usernames can contain ~
4 years ago
Mike Fährmann
35056a07d1
release version 1.15.1
4 years ago
Mike Fährmann
dd1e545597
[hentaifoundry] rename GalleryExtractor to PicturesExtractor
4 years ago
Mike Fährmann
c874071f5a
[kissmanga] remove module
4 years ago
Mike Fährmann
430b6d6e2e
[twitter] extend 'retweets' option ( closes #1026 )
...
Setting 'retweets' to '"original"' will use metadata from the
original retweeted Tweets, and not from the Retweet entry.
4 years ago
Mike Fährmann
b9bdd2c564
[hentaifoundry] add support for stories ( closes #734 )
4 years ago
Mike Fährmann
117582d798
use definition lists instead of tables in docs/configuration.rst
...
and strip some clutter from generated man pages
4 years ago
Mike Fährmann
cc4ac80302
[weasyl] add 'favorite' extractor ( #1032 )
4 years ago
Mike Fährmann
e9cc719497
[weasyl] update and simplify
...
- simplify 'pattern' regexps
- parse 'posted_at' as 'date'
- use unaltered 'title' ({title!l:R /_/} to lowercase and replace spaces)
4 years ago
Mike Fährmann
6514312126
[nijie] add 'include' option ( closes #1018 )
4 years ago
Mike Fährmann
0d43456323
[hentaifoundry] add 'include' option
4 years ago
Zanny
ebb7737b9b
Weasyl Extractor ( #977 )
...
* weasyl extractor
* @kattjevfel suggested changes
* @mikf changes
4 years ago
Mike Fährmann
1e313d5b84
implement 'sleep-request' option
4 years ago
Mike Fährmann
c43b3894be
[myhentaigallery] update and fix extraction ( #1001 )
...
- extract more metadata
- match "/show/" URLs
- complete test results
- fix missing images for lines starting with " <img"
- fix missing comma in supportedsites.py
4 years ago
Mike Fährmann
6a5f6396d4
add clearing cache to dA client-id instructions ( #996 )
4 years ago
Mike Fährmann
3afd362e2e
add 'sleep-extractor' option ( closes #964 )
...
(would have been nice if this were possible without code duplication)
4 years ago
Mike Fährmann
3108e85b89
[worldthree] remove extractors
...
http://www.slide.world-three.org/ hasn't been accessible for a long time.
4 years ago
Mike Fährmann
8fed3eb8cb
[jaiminisbox] remove extractors
...
https://jaiminisbox.com/post.html
4 years ago
Mike Fährmann
3918b69677
remove 'extractor.blacklist' context manager
4 years ago
Mike Fährmann
c78aa17506
add general 'blacklist' and 'whitelist' options ( #492 , #844 )
4 years ago
Mike Fährmann
5912727b88
support format string replacement fields in archive paths
...
(closes #985 )
4 years ago
Mike Fährmann
f6fd449b59
reduce wait time growth rate from exponential to linear
...
Waiting for 2**N seconds after each error grows too fast.
Simply waiting N seconds seems far more reasonable.
4 years ago
Mike Fährmann
ac3036ef56
add 'filesize-min' and 'filesize-max' options ( closes #780 )
4 years ago
Mike Fährmann
e33293fdd8
[hentaihand] update to new site layout
4 years ago
Mike Fährmann
ab1af66a97
[imgur] add 'search' extractor ( #934 )
4 years ago
ArtaxIsSleeping
0e941553ec
[aryion] Add username/password support ( #960 )
...
* Add username/password support to aryion extractor
* Update docs to match
* Fix code style
4 years ago
Mike Fährmann
063c71cd84
[furaffinity] add 'search' extractor ( closes #915 )
4 years ago
Mike Fährmann
dbbbb21180
[exhentai] add ability to specify custom image limit ( #940 )
4 years ago
Mike Fährmann
2b88c90f6f
[blogger] add search extractor ( #925 )
4 years ago
Mike Fährmann
901a897e6f
[inkbunny] mention login support
4 years ago
Mike Fährmann
b921fee24d
[inkbunny] fix submission order ( #283 )
...
Getting detailed submission info via /api_submissions.php reordered the
input submissions and sorted them by ID. InkbunnyAPI.detail() now sorts
them back and ensures they are returned in their original order.
This commit also removes the 'metadata' option and always requests
submission descriptions.
4 years ago
Mike Fährmann
c4ed9f4faa
[inkbunny] add 'metadata' option ( #283 )
4 years ago
Mike Fährmann
493cadb1e7
[inkbunny] add 'orderby' option ( #283 )
4 years ago
Mike Fährmann
8dbf827649
[bobx] remove module
4 years ago
Mike Fährmann
87202b8d74
[inkbunny] add 'user' and 'post' extractors ( #283 )
4 years ago
Mike Fährmann
b62ea72533
release version 1.14.3
4 years ago
Mike Fährmann
d5fcffcced
[subscribestar] add login capabilities ( #852 )
4 years ago
Mike Fährmann
ecaecc4064
[exhentai] add 'domain' option ( #897 )
4 years ago
Mike Fährmann
df7217c36d
update supportedsites.py ( #889 , #893 )
...
- mention optional auth access for more sites
- link to OAuth and Cookies sections in README
4 years ago
HRXN
51b49a361d
mention optional auth access for furaffinity ( #893 )
4 years ago
jakem72360
7dfdcc3fbf
[mangakakalot] Added extractors for MangaKakalot ( #876 )
4 years ago
Mike Fährmann
cb0132e441
[khinsider] add 'format' option ( closes #840 )
4 years ago
Mike Fährmann
c28db7a6ea
[8muses] support 'comics.8muses.com' URLs
4 years ago
Mike Fährmann
821524e4ee
[subscribestar] add 'user' and 'post' extractors ( #852 )
4 years ago
Mike Fährmann
f1ddbff0b5
[aryion] add 'recursive' option ( fixes #832 )
...
This is enabled by default and will recursively go through all
(sub)folders in an artist's gallery.
The old method of using "Latest Updates" lists can be restored by
disabling this option.
4 years ago
Mike Fährmann
9da2bc67f8
[twitter] add option to filter media from quoted tweets ( #854 )
4 years ago
Mike Fährmann
0e714b9a0e
[pinterest] add 'section' extractor ( #835 )
4 years ago
Mike Fährmann
5ba90f72ca
[pinterest] add support for sections ( closes #835 )
4 years ago
Mike Fährmann
c37a1c06c8
[twitter] add extractor for liked tweets ( closes #837 )
...
You need to be logged in to get access to anyone's liked tweets,
it seems.
4 years ago
Mike Fährmann
c184cce876
update configuration.rst
...
- fix anonymous links
- update description of 'extractor.twitter.videos'
- document 'extractor.redgifs.format' (#724 )
4 years ago
Mike Fährmann
cf863f60b3
[redgifs] add 'user' and 'search' extractors ( closes #724 )
4 years ago
Mike Fährmann
4aea5138dd
[sensescans] use https://
4 years ago
Mike Fährmann
41d03160ff
[deviantart] also search journals for sta.sh links ( #712 )
...
when 'extra' is enabled
4 years ago
Mike Fährmann
0138e9c1d8
[twitter] remove 'content' option
...
The text content of each tweet is always available as 'full_text'
4 years ago
Mike Fährmann
0f459f340b
[instagram] fix and re-enable login with username&password
...
This reverts commit 3e0848a482
.
(#756 , #771 , #797 , #803 )
https://github.com/althonos/InsaLooter/issues/287#issuecomment-630456522
4 years ago
Mike Fährmann
3e0848a482
[instagram] disable login with username&password ( #756 )
4 years ago
Mike Fährmann
da22ea8ced
use %APPDATA%\gallery-dl for config/cache on Windows
4 years ago
Mike Fährmann
45baa13615
update extractor test results
...
- don't run Instagram tests on Travis anymore
- replace Twitter test because timeline was made private
- update Hiperdex domain to '.com' (again ...)
4 years ago
Mike Fährmann
b489f4de12
fix internal links in configuration.rst
4 years ago
Mike Fährmann
dfcf2a2c91
write OAuth token to cache by default ( #616 )
4 years ago
Mike Fährmann
ddc253cf9a
implement a 'path-replace' option ( #662 , #755 )
4 years ago
Mike Fährmann
7003e61d1d
add a few more examples to gallery-dl-example.conf
...
- include 'igneous' and 'hath_perks' in Exhentai cookies
- add an example of how to write DeviantArt description to file
- add a 'path-restrict' mapping from invalid characters in Windows
paths to Unicode alternatives (taken from #662 )
4 years ago
Mike Fährmann
bc53302ad6
extend 'path-restrict' option
...
Allow its value to be a JSON object / Python dict that specifies
a mapping from invalid/unwanted input characters to specific
output characters.
For example {"/": "-", "*": "+"} will transform
"foo / ***bar***" into "foo - +++bar+++"
(closes #662 , #755 )
4 years ago
Mike Fährmann
e19f665a44
[danbooru] change default for 'ugoira' to 'false'
...
Downloading the pre-rendered versions should be a better default
than .zip files with individual frames.
4 years ago
Mike Fährmann
0378d079a5
[webtoons] fixes and simplifications ( #593 , #761 )
...
- fix episode listings for french comics
- allow input URLs without explicit scheme
- add 'lang'/'language' metadata
- use str.format() instead of '+' to assemble URLs
4 years ago
Mike Fährmann
ab11b1c896
[imagechest] simplify code ( #750 )
4 years ago
Mike Fährmann
dba87ca99e
[downloader:ytdl] change 'forward-cookies' default to 'false'
...
There are currently no situations where forwarding gallery-dl's cookies
to youtube-dl is necessary, and it only causes problems when forcing
youtube-dl for Twitter video downloads while logged in.
4 years ago
Mike Fährmann
2d6724180b
[hiperdex] update domain to hiperdex.info
4 years ago
Mike Fährmann
be16d77073
simplify and improve documentation markup ( #742 )
...
- use | to get a more "compact" representation
- link directly to example config files
4 years ago
Vrihub
fe224416bf
Some documentation improvements ( #742 )
...
* Link configuration examples in the intro, see #712
* Clarify overriding conf files by options, see #678
4 years ago
Mike Fährmann
f557cac074
[redgifs] add image extractor ( #724 )
4 years ago
Mike Fährmann
ba42ec321c
release version 1.13.6
4 years ago
Mike Fährmann
ff47641b13
test whether default/example config files contain valid JSON
4 years ago
Mike Fährmann
5e2974d699
[weibo] add 'videos' option
4 years ago
Mike Fährmann
9f638c2e01
[twitter] add 'replies' option ( closes #705 )
4 years ago
Mike Fährmann
a4fd620a25
[hiperdex] revert domain back to hiperdex.com
4 years ago
Mike Fährmann
a3de234e70
[hitomi] add extractor for tag searches ( closes #697 )
4 years ago
Mike Fährmann
d5273f9b0c
[hiperdex] update domain to hiperdex.net
4 years ago
Mike Fährmann
a6286bb551
[hiperdex] add 'artist' extractor ( #606 )
5 years ago
Mike Fährmann
6143050980
[aryion] add gallery and post extractors ( #390 , #673 )
5 years ago
Mike Fährmann
2587296deb
[mastodon] add access tokens for mastodon.social and baraag.net
...
(closes #665 )
5 years ago
Mike Fährmann
4203dc0bdc
[mangapark] fix metadata extraction
5 years ago
Mike Fährmann
88ebbadc58
remove dashes from subcategory names in supportedsites.rst
5 years ago
Mike Fährmann
09f2271528
[35photo] add 'tag' extractor
5 years ago
Mike Fährmann
90d15e3682
[instagram] use 'itertools.chain()'
5 years ago
Mike Fährmann
5b676ea59d
[e621] document username & password support ( #640 )
5 years ago
Mike Fährmann
fdd2dd5136
[kabeuchi] add 'user' extractor ( closes #561 )
5 years ago
Mike Fährmann
59edcdc822
[hitomi] restore metadata fields from before f33b13a
...
... and add a 'metadata' option to disable
visiting the gallery page and extracting data from it
if this is not needed.
5 years ago
Mike Fährmann
823fbeaae6
[newgrounds] add 'favorite' extractor ( #394 )
5 years ago
Mike Fährmann
a45fbc38ea
[pixiv] implement 'avatar' option ( #595 , #623 )
5 years ago
Mike Fährmann
32df8d06fe
[twitter] add 'bookmark' extractor ( closes #625 )
5 years ago
Mike Fährmann
cc5079c844
[hiperdex] add chapter and manga extractors ( closes #606 )
5 years ago
Mike Fährmann
6e759fe2f5
reword note for 'extractor.deviantart.refresh-token' ( #612 )
5 years ago
Mike Fährmann
765b2a0527
[hentaihand] add extractors ( closes #605 )
5 years ago
Mike Fährmann
67ea80222c
link to post processor example in 'pixiv.ugoira' docs
5 years ago
Mike Fährmann
74e684e828
[twitter] change default value for 'videos' to 'true'
...
Every other 'videos' option defaulted to 'true', except Twitter.
5 years ago
Mike Fährmann
ab13f81fd7
add example to 'cookies' docs
5 years ago
Mike Fährmann
c7cf9dd111
[furaffinity] support classic layout ( #284 )
5 years ago
Mike Fährmann
138135c190
[furaffinity] add extractors ( #284 )
5 years ago
Mike Fährmann
da0d5f6092
[oauth] add 'port' option ( #604 )
5 years ago
Mike Fährmann
719b63d0ca
[bcy] add user and post extractors ( #592 )
5 years ago
Mike Fährmann
091f9a107d
[docs] add examples of how to filter manga chapters by language
...
(closes #599 )
5 years ago
Mike Fährmann
1f4c9c5f9d
[8kun] add thread and board extractors ( closes #582 )
5 years ago
Mike Fährmann
dff33b260c
[reddit] add 'videos' option
5 years ago
Mike Fährmann
cf7a67d67f
[yaplog] remove module
...
Yaplog! ended its service on 2020-01-31
5 years ago
Mike Fährmann
56f1c96168
implement 'parent-directory' option ( #551 )
5 years ago
Mike Fährmann
6703b8a86b
[blogger] implement video extraction ( closes #587 )
5 years ago
Mike Fährmann
37247dbaff
miscellaneous fixes
5 years ago
Mike Fährmann
25d5ec4ff3
[twitter] add option to extract TwitPic embeds ( #579 )
5 years ago
Mike Fährmann
254f7c3999
implement a post-processor module to compare file versions
...
(#530 )
5 years ago
Mike Fährmann
a6b98d3bec
fix auto-generation of supportedsites.rst
5 years ago
Mike Fährmann
5cac79c3d9
[erolord] remove extractor
5 years ago
Mike Fährmann
87c8b89ddd
[postprocessor:metadata] add 'directory' option ( #520 )
5 years ago
Mike Fährmann
48e42e73fb
[reddit] change default value for 'comments' to '0'
5 years ago
Mike Fährmann
bf658fd84b
[vsco] implement 'videos' option
5 years ago
Mike Fährmann
95c90722ee
[instagram] implement 'videos' option ( closes #521 )
5 years ago
The Oddball
9a4ce20b8e
[realbooru] Add Realbooru extractor ( #514 )
5 years ago
Mike Fährmann
15f9bb3d14
add option to disable pyOpenSSL usage ( #508 )
...
(pyOpenSSL is now disabled by default)
5 years ago
Mike Fährmann
93b43f74f4
update docs/configuration.rst
...
- fix description of "skip": "enumerate"
- use line breaks for lists of multiple examples
- improve formatting of auto-generated man pages
5 years ago
Mike Fährmann
e17907ee2a
change default value of 'cookies-update' to 'true'
5 years ago
Mike Fährmann
26d2334550
[postprocessor:metadata] rename 'format' to 'content-format'
...
Just to be consistent with the other 'extension-format' option name,
and only 'format' is also still accepted.
5 years ago
Mike Fährmann
a412531451
[postprocessor:metadata] implement 'extension-format' option
...
closes #477
5 years ago
Mike Fährmann
d45fabb79d
match user profile handling on deviantart and newgrounds
5 years ago
Mike Fährmann
3ece3976ae
[newgrounds] implement login support ( #394 )
5 years ago
Mike Fährmann
94a94f3b86
miscellaneous stuff
5 years ago
Mike Fährmann
dd5d2b2eac
[deviantart] add user profile extractor ( #377 , #419 )
5 years ago
Mike Fährmann
9e88e7a344
[postprocessor:exec] improve ( #421 , #413 )
...
- add 'final' option
- include job status in pp finalization
- improve and extend documentation
5 years ago
Mike Fährmann
557e2c018b
[8chan] remove module
5 years ago
Mike Fährmann
c01ff78467
[twitter] extend 'videos' option to force extraction with ytdl
...
(closes #459 )
5 years ago
Mike Fährmann
702f2fbd1f
[issuu] add publication and user extractors ( #413 )
5 years ago
Mike Fährmann
109718a5e3
[blogger] add blog and post extractors ( closes #364 )
5 years ago
Mike Fährmann
8f38a35b91
[imgur] use API with "public" client_id ( #446 )
...
Using the API endpoints makes it possible to access NSFW content
without logging in.
5 years ago
Mike Fährmann
b23c822b23
[luscious] use GraphQL
5 years ago
Mike Fährmann
2057c6ba29
[naver] add blog and post extractors ( closes #447 )
5 years ago
Mike Fährmann
a5e6252137
update deviantart client-id/-secret instructions ( #451 , #424 )
5 years ago
Mike Fährmann
389d2d7e38
implement 'cookies-update' option ( #445 )
5 years ago
Mike Fährmann
5882b00f2f
[imgur] implement login support ( #446 )
5 years ago
Mike Fährmann
91643ca54b
[nozomi] add search extractor ( #388 )
5 years ago
Mike Fährmann
1adafdd3d0
document cache file requirement for DeviantArt refresh tokens
5 years ago
Mike Fährmann
6779512fc7
[nozomi] add post and tag extractors ( #388 )
5 years ago
Mike Fährmann
8af59a4bba
fix & update docs
...
- update Requests links
- add example for --exec
- set '-dev' version
5 years ago
Mike Fährmann
46ba173ded
[reddit] fix documentation inconsistencies ( closes #429 )
...
- Require 'reddit.comments' to be a number and convert it to an
integer to be extra sure
- Link to the README's OAuth section were appropriate
5 years ago
Mike Fährmann
5a1a0f5325
change text representation of user extractors to "User Profiles"
5 years ago
Mike Fährmann
ee8b654464
[instagram] implement 'highlights' option ( closes #329 )
5 years ago
Mike Fährmann
4330133114
[imgur] add 'favorite' extractor ( closes #420 )
...
… and use a newer site-internal API endpoint for user posts
5 years ago
Mike Fährmann
b1cddce865
Revert "[simplyhentai] fix extraction; remove image+video extractors"
...
This reverts commit d1db5180ab
.
5 years ago
Mike Fährmann
6fe9a134bf
[lineblog] add blog and post extractors ( closes #404 )
5 years ago
Mike Fährmann
6f3df3999a
[fuskator] add gallery and search extractor ( closes #407 )
5 years ago
Mike Fährmann
f02a768b5c
[danbooru] add 'ugoira' option ( #406 )
...
to choose between ZIP archives or converted video files
for Ugoira posts
5 years ago
Mike Fährmann
c6c5cb1898
improve 'deviantart.quality' description
5 years ago
Mike Fährmann
7c09545f70
[downloader:ytdl] add 'outtmpl' option ( #395 )
5 years ago
Mike Fährmann
d1db5180ab
[simplyhentai] fix extraction; remove image+video extractors
5 years ago
Mike Fährmann
b1728f512d
[patreon] support multi image posts and post URLs ( #383 )
5 years ago
Mike Fährmann
5a210991b6
Remove control characters from filesystem paths
...
- add 'path-remove' option to specify the set of characters that
should be removed
- rename 'restrict-filenames' to 'path-restrict'
- #348 , #380
5 years ago
Mike Fährmann
e661a1ce19
update links to Request's documentation
5 years ago
Mike Fährmann
829b1ccf04
[imgur] distinguish album and gallery URLs ( #380 )
...
A gallery can be either an album or a single image.
5 years ago
leezhi0513
42bb0cdb9b
Update gallery-dl.conf ( #378 )
5 years ago
Mike Fährmann
2ccf6a9e35
[instagram] make extractor tests happy ( #373 )
5 years ago
Mike Fährmann
8dc42bb178
implement 'enumerate' for 'extractor.skip' ( #306 )
...
[ci skip]
5 years ago
Mike Fährmann
b7fb93e2b2
[downloader:http] add 'adjust-extensions' option
5 years ago
Mike Fährmann
189acbeac9
[imgbb] add extractor for individual images ( closes #363 )
5 years ago
Mike Fährmann
f687052daf
have e-hentai and exhentai on supportedsites.rst ( #365 )
5 years ago
Mike Fährmann
3957d27d79
[deviantart] add 'quality' option ( #369 )
5 years ago
Mike Fährmann
0609afd1e4
update default cache directory ... again
...
Use a 'gallery-dl' subdirectory in ~/.cache to adhere to how other
programs store their cached data, and call os.makedirs() so it also
works without an existing ~/.cache directory.
5 years ago
Mike Fährmann
2c839f3760
[imgbb] add user extractor + login support ( #361 )
5 years ago
Mike Fährmann
a8b60b2bd9
change default cache directory for unix systems
...
Use either $XDG_CACHE_HOME or ~/.cache (if the former isn't set)
and store potentially sensitive cookies and tokens in a user's
home directory and not in the world-readable /tmp.
5 years ago
Mike Fährmann
2153206093
[imgbb] add album extractor ( #361 )
5 years ago
Mark Henrick
923e1bb714
[docs] Fix inconsistency about which sites have optional authentication ( #359 )
...
* [docs] Fix inconsistency about which sites have optional authentication
* update authentication docs
5 years ago
Mike Fährmann
a90280f4e7
[postprocessor:zip] add 'mode' option ( #355 )
5 years ago
Mike Fährmann
dc73d02d87
[exhentai] always use e-hentai.org as domain + set nw cookie
5 years ago
Mike Fährmann
279db2c5b2
[vsco] add collection & image extractor + video support ( #331 )
5 years ago
Mike Fährmann
547ea71463
[downloader.ytdl] add 'forward-cookies' option ( #352 )
...
The "long" name is necessary because just calling it 'cookies' would
clash with how the lookup for '--cookies' is implemented.
5 years ago
Mike Fährmann
b1bea8aaeb
add 'restrict-filenames' option ( #348 )
5 years ago
Mike Fährmann
60cf40380a
[vsco] add user extractor ( #331 )
5 years ago
Mike Fährmann
3fe5ccdfa6
[adultempire] add gallery extractor ( closes #340 )
5 years ago
Mike Fährmann
21991acc49
add 'ciphers' option; update default User-Agent
5 years ago
Mike Fährmann
feb98cf196
[twitter] improve 'content' formatting; add option ( #338 )
...
- include emoticons
- leave newlines intact
- remove pic.twitter.com/ links at the end
5 years ago
Mike Fährmann
8d1ae9b715
[tumblr] enable date-min/-max/-format options ( #337 )
5 years ago
Mike Fährmann
fb875d1ab8
add warning about NSFW sites in supportedsites.rst ( #335 )
5 years ago
Mike Fährmann
16c582aaf9
implement 'mtime' post-processor ( #332 )
...
This can set a file's modification time according to a UNIX timestamp
or a datetime object from its metadata.
5 years ago
Mike Fährmann
62097284fe
add 'download' option ( #220 )
5 years ago
Mike Fährmann
2ff73873f0
[erolord] add gallery extractor ( closes #326 )
5 years ago
Mike Fährmann
b4da8c5a97
[sexcom] add extractor for related pins ( #325 )
5 years ago
Mike Fährmann
69205df68d
allow '-1' for infinite retries ( #300 )
5 years ago
Mike Fährmann
f7b5c4c3e7
use values of 'retries' options correctly
...
The RE-tries option now specifies exactly that: the maximum number a
failed HTTP request is re-tried. For example a value of 2 will now
correctly stop after 3 attempts: the initial one + 2 re-tries.
The maximum wait-time now also caps at 30min and increases exponentially
for both extractor.request() and downloader.http.download().
5 years ago
Mike Fährmann
f2000a69aa
implement 'image-unique' and 'chapter-unique' options ( #303 )
...
The default value for both is 'false', i.e. duplicate URLs are NOT
ignored.
The previous behavior was to always ignore duplicate URLs to make
'--abort-on-skip' work properly when new images where added to the
beginning of a collection while gallery-dl is running.
5 years ago
Mike Fährmann
40da44b17f
Merge branch 'v1.9.0'
5 years ago
Mike Fährmann
a9c89085fb
[instagram] implement login support ( #195 )
5 years ago
Mike Fährmann
18a1f8c6cd
[vanillarock] add post and tag extractors ( closes #254 )
5 years ago
Mike Fährmann
f0c5093812
[nsfwalbum] add album extractor ( closes #287 )
5 years ago
Mike Fährmann
15e4ddf46d
implement custom logging formatter
...
supports custom log message formats for each loglevel and, by
extension, custom ANSI codes and colors for errors and warnings
(#304 )
5 years ago
Mike Fährmann
db3f52881a
add 'mtime' option
5 years ago
Mike Fährmann
ec3e8601f1
[slickpic] add user extractor ( #249 )
5 years ago
Mike Fährmann
d997c10320
[8muses] add album extractor ( #305 )
5 years ago
Mike Fährmann
e05a96db5e
[deviantart] rename 'stash' to 'extra' ( #302 )
...
'stash' is already used as a name for the StashExtractor and therefore
expected to be a dictionary.
5 years ago
Mike Fährmann
2184e3a86b
[slickpic] add album extractor ( #249 )
5 years ago
Mike Fährmann
c23bf263fe
[deviantart] rename 'external' to 'stash' ( #302 )
...
restrict extracted URLs to ones from https://sta.sh/ ...
5 years ago
Mike Fährmann
c73c2cda50
[pornhub] add gallery & user extractor ( #282 )
5 years ago
Mike Fährmann
035b850e82
update postprocessor entries in example config
...
- use whitelists
- add ugoira example (#299 )
5 years ago
Mike Fährmann
2fb85178da
[deviantart] add 'external' option ( #302 )
...
If a description is available, this will extract URLs from the
description text and try to find Extractors for them.
5 years ago
Mike Fährmann
096009367b
[xhamster] add gallery & user extractor ( #281 )
5 years ago
Mike Fährmann
d33f5a7423
[wallhaven] rewrite
...
- use API
- remove login support, add 'api-key' option
- remove support for "alpha" subdomain - alpha.wallhaven.cc used numeric
IDs that can't be translated to the new ID system
- support direct links to wallpapers
5 years ago
Mike Fährmann
f1893b2b5b
[deviantart] add 'folders' option ( #276 )
5 years ago
Mike Fährmann
c849574def
[keenspot] add comic extractor ( #223 )
...
Doesn't work for
- http://brawlinthefamily.keenspot.com/
- http://flipside.keenspot.com/
- http://lastblood.keenspot.com/
- http://mysticrevolution.keenspot.com/
- http://porcelain.keenspot.com/
- http://twokinds.keenspot.com/
yet, because of custom layouts.
5 years ago
Mike Fährmann
bcd1801aa8
[sankakucomplex] add 'tag' extractor ( #258 )
5 years ago
Mike Fährmann
74c2415138
[sankakucomplex] move article extractor to its own module ( #258 )
5 years ago
Mike Fährmann
4465a3ea68
[kissmanga][readcomiconline] add 'captcha' option ( #279 )
...
to configure how to handle CAPTCHA page redirects:
- either interactively wait for the user to solve the CAPTCHA
- or raise StopExtraction like before
5 years ago
Mike Fährmann
1e3e15c4f3
[sankaku] add article extractor ( #258 )
5 years ago
Mike Fährmann
e82cadac61
[patreon] add extractors ( #226 )
5 years ago
Mike Fährmann
4891f4a328
[hentainexus] add search extractor ( #256 )
5 years ago
Mike Fährmann
9890bfdf23
[flickr] improve code and metadata
...
- simplify pagination
- add more metadata and slightly change its structure
- convert suitable values to int or list
- move keys from ["photo"] to the base level
- proper video support (#246 )
- rename method and variable names to better fit with other extractors
5 years ago
Mike Fährmann
ba8eb1ffec
[hentainexus] add gallery extractor ( #256 )
5 years ago
Mike Fährmann
0318c610dc
[sexcom] add extractor for search results ( #147 )
5 years ago
Mike Fährmann
a247c94c34
[sexcom] add pin and board extractors ( #147 )
5 years ago
Mike Fährmann
70be494161
[plurk] add a 'comments' options ( #212 )
6 years ago
Mike Fährmann
0b2ff406f6
[plurk] add timeline- and post-extractors ( #212 )
6 years ago
Mike Fährmann
e7cd5510d5
[pixnet] add extractors ( closes #177 )
...
for:
- users/blogs: http://albertayu773.pixnet.net/
- folders: https://albertayu773.pixnet.net/album/folder/1405768
- sets : https://albertayu773.pixnet.net/album/set/15078995
- photos : https://albertayu773.pixnet.net/album/photo/159443828
6 years ago
Mike Fährmann
d9b94a585d
[mangoxo] add login support ( #184 )
...
A very recent change: It is now only possible to see more
than the first 5 images of an album if you are logged in.
6 years ago
Mike Fährmann
e730fc9045
[twitter] add login support ( #214 )
6 years ago
Mike Fährmann
35919a9bb8
[livedoor] add blog- and post-extractors ( #190 )
6 years ago
Mike Fährmann
2ff043edfa
[yaplog] add user- and post-extractors ( #190 )
6 years ago
Mike Fährmann
790f15a56f
[photobucket] use HTTPS
6 years ago
Mike Fährmann
6da665f32e
[mangoxo] add album- and channel-extractors ( closes #184 )
6 years ago
Mike Fährmann
c70b21248d
[wikiart] add extractors ( #179 )
...
for
- artists: https://www.wikiart.org/en/thomas-cole
- artist-listings: https://www.wikiart.org/en/artists-by-century/12
- artwork-listings: https://www.wikiart.org/en/paintings-by-media/grisaille
6 years ago
Mike Fährmann
bc2020e86c
release version 1.8.1
6 years ago
Mike Fährmann
369eb66125
consistently use '*' for rst lists
6 years ago
Mike Fährmann
c7a6b0ed90
[deviantart] add 'metadata' option ( #189 )
6 years ago
Mike Fährmann
8d96a8ce4c
[500px] add user-, gallery-, and image-extractors ( #185 )
6 years ago
Mike Fährmann
6277a739e4
[35photo] add user-, genre-, and image-extractors ( #162 )
6 years ago
Mike Fährmann
12482553bd
update links to youtube-dl
6 years ago
Mike Fährmann
6f57d44ec2
[seaotterscans] remove extractor
...
http://seaotterscans.com/ now redirects to their MangaDex profile
6 years ago
Mike Fährmann
25aaf55514
[smugmug] improve format selection ( closes #183 )
...
- use original image if available
- support video formats
- remove user info for ImageExtractor (it is no longer possible to get
image owner information for a single image)
6 years ago
Mike Fährmann
976ccb267f
[myportfolio] combine gallery and user extractors
...
An URL alone isn't good enough to distinguish between a gallery or a
gallery-listing, so the new extractor decides what to do based on the
page's content.
6 years ago
Mike Fährmann
9c0e2f294b
[shopify] add generic collection and product extractors ( #175 )
...
with fashionnova.com as a default domain
6 years ago
Mike Fährmann
13e0f2a78f
[deviantart] add 'scraps' extractor ( closes #168 )
6 years ago
Mike Fährmann
280531c8ff
[pururin] add gallery extractor ( closes #174 )
6 years ago
Mike Fährmann
3159dd79d5
[seiga] use HTTPS
6 years ago
Mike Fährmann
ea6fbfdd39
simplify build_supportedsites.py
6 years ago
Mike Fährmann
22d3a2fcc8
[artstation] add extractor for artwork listings ( #80 )
...
like https://www.artstation.com/artwork?sorting=latest
or https://www.artstation.com/artwork?sorting=picks
6 years ago
Mike Fährmann
937a802b49
[dynastyscans] add extractors for images and image searches
...
(closes #163 )
6 years ago
Mike Fährmann
19860655a3
[weibo] add 'user' and 'status' extractors
6 years ago
Mike Fährmann
ae353ed3b0
provide "extractor" and "job" keys for logging output
...
This allows for stuff like "{extractor.url}" and "{extractor.category}"
in logging format strings.
Accessing 'extractor' and 'job' in any way will return "None" if those
fields aren't defined, i.e. in general logging messages.
6 years ago
Mike Fährmann
53c2fd4664
add mastodon/foolslide/foolfuuka examples to example config
6 years ago
Mike Fährmann
d36ec51e5a
[tsumino] add extractor for search results ( #161 )
6 years ago
Mike Fährmann
bfbbac4495
[tsumino] add login capabilities ( #161 )
6 years ago
Mike Fährmann
6126615698
update URLs for supportedsites.rst
6 years ago
Mike Fährmann
80a75a1ecf
[tsumino] add gallery extractor ( #161 )
6 years ago
Mike Fährmann
0c32dc5858
[hentaifox] add extractor for search results ( #160 )
6 years ago
Mike Fährmann
0156189468
[hentaifox] add chapter extractor ( #160 )
6 years ago
Mike Fährmann
e4171d6baf
[luscious] add login capabilities ( closes #159 )
6 years ago
Mike Fährmann
134487ffb0
[exhentai] stop extraction if image limit is exceeded ( #141 )
...
can be turned off with the `exhentai.limits' option
6 years ago
Mike Fährmann
b50b30f1c9
[photobucket] download subalbums ( #117 )
6 years ago
Mike Fährmann
277b52101a
add 'category-transfer' option
...
[ci skip]
6 years ago
Mike Fährmann
4b441c162e
release version 1.6.3
6 years ago
Mike Fährmann
8aba2bdebf
[postprocessor:metadata] add 'tags' and 'custom' modes ( #135 )
6 years ago
Mike Fährmann
e53cdfd6a8
update build_supportedsites.py
6 years ago
Mike Fährmann
1e4d351ad3
[danbooru] add authentication support ( closes #151 )
...
... via HTTP Basic Auth with username and "password".
The password value in this case is not the account password itself,
but the"api_key" found in your user profile.
6 years ago
Mike Fährmann
06cbf5f9c4
implement 'chapter-reverse' option ( #149 )
...
Setting it to `true` will start with the latest chapter instead of the
first one.
6 years ago
Mike Fährmann
e95b24f056
[reactor] add wait-min & -max options ( #148 )
6 years ago
Mike Fährmann
a36f52a730
[joyreactor] add extractor for search results ( #148 )
6 years ago
Mike Fährmann
6c71e9cf5d
[deviantart] add separate 'sta.sh' extractor ( #113 )
...
- supports multiple stashed deviations per page
- explicitly mentions sta.sh support on supportedsites.rst
6 years ago
Mike Fährmann
28f9539551
[tumblr] change default values for post types and inline media
6 years ago
Mike Fährmann
5be95034ba
[tumblr] add option to download avatars ( #137 )
6 years ago