a45fbc38ea
[pixiv] implement 'avatar' option ( #595 , #623 )
5 years ago
a63a376ad2
[mangoxo] fix login
5 years ago
ebc70e87ce
[e621] update to new interface / API endpoints ( closes #635 )
5 years ago
d1cf7ccdb3
[instagram] add 'post_shortcode' metadata field ( #525 )
5 years ago
402025c3c3
fix some build issues
...
- use 'os.name' to decide between Windows/Linux build
- don't check Windows executable version number,
since Wine fails to run the executable and causes release.sh to stop
5 years ago
32df8d06fe
[twitter] add 'bookmark' extractor ( closes #625 )
5 years ago
3fb41c34c8
[bcy] reduce requests to '/item/detail/<id>' ( #613 )
...
The former implementation would try to use the embedded data from
'/item/detail/' pages for every post, even if that wasn't really
necessary.
This commit also fixes some issues with posts only visible to
logged in users.
5 years ago
f33b13aacf
[hitomi] simplify metadata extraction
...
Use the data from https://ltn.hitomi.la/galleries/ <id>.js for both
image URLs and metadata and ignore any gallery or reader pages.
This removes 'artist', 'characters', 'group', and 'parody' metadata
fields since this information is, as for now, only available in
gallery pages.
5 years ago
115fd2c6f2
"fix" incomplete MIME types ( #632 )
...
e-/exhentai's original image downloads currently send
incomplete/invalid Content-Type headers, "jpg" instead
of "image/jpg" etc, since the last update.
(https://forums.e-hentai.org/index.php?showtopic=236113 )
This change prepends any Content-Type value missing a
media type specification with "image/", transforming it
into a valid MIME type.
(A global solution to a local problem, but it shouldn't
cause any issues anywhere else)
5 years ago
72122eb9b3
release version 1.13.1
5 years ago
adcd7cb24a
[downloader:http] add another MIME type for '.rar' files ( #628 )
5 years ago
ce5e2a58fe
[imgbb] update test results
...
Image server domain changed from
https://image.ibb.co/ to https://i.ibb.co/
5 years ago
f117e32910
[danbooru] restore 'popular' functionality
5 years ago
39b48d665b
[hiperdex] use proper name for 'chapter_minor'
5 years ago
8fbbaa54ff
[bcy] fix partial image URLs ( #613 )
...
Images from new posts can have incomplete/partial URLs (1)
without any filename extension when fetching their data from
'/apiv3/user/selfPosts', so now all data gets taken from
'/item/detail/ID' pages.
It is currently unknown how to get the non-watermarked original version
of these images, or if that is possible at all. (2)
Images with a watermark will have their 'filter' metadata field set to
"watermark". For original images this field is an empty string "".
Enabling the 'noop' option will, in addition to the watermarked version,
yield the the '~noop.image' filter version (3),
where 'filter' is set to "noop".
(1) "https://img-bcy-qn.pstatp.com/banciyuan/3ccdff22479c4060aadc86718209b281 "
(2) "https://p1-bcy.byteimg.com/img/banciyuan/3ccdff22479c4060aadc86718209b281~tplv-banciyuan-logo-v3:wqnpnLLlhZLlpKfprZTnjotfCuWNiuasoeWFgyAtIEFDR-eIseWlveiAheekvuWMug==.image "
(3) "https://p1-bcy.byteimg.com/img/banciyuan/3ccdff22479c4060aadc86718209b281~noop.image "
5 years ago
86c00f9e66
[danbooru] move extractor logic from booru.py
5 years ago
1d4a369ea2
update extractor test results
5 years ago
7625912b31
[piczel] improve and update
...
- fix tag names
- fix a bug in _pagination()
- parse datetime in 'created_at' as 'date'
- rewrite main loop
- replace user profile test
5 years ago
ec85bf90de
use context managers in cache.py & add tests
5 years ago
913b8333cc
write DeviantArt refresh-tokens to cache ( #616 )
...
Writing the token is currently disabled by default and must be
enabled with 'extractor.oauth.cache'.
'extractor.deviantart.refresh-token' must be set to '"cache"'
to use the cached token.
5 years ago
2a4f227e08
warn about expired cookies
5 years ago
34887ae139
fix bugs in DatabaseCacheDecorator.update()/.invalidate()
...
- call db.commit() after changes have been made
- remove 'LIMIT 1' from the DELETE statement in invalidate()
(only available if SQLite3 was compiled with the right flags
enabled, syntax error otherwise)
5 years ago
380b693fad
[downloader:http] add more MIME types for '.bmp' files ( #621 )
5 years ago
4e361b3008
add tests for specific datetime values
5 years ago
80ecb99089
[hitomi] fix extraction
5 years ago
247c9e1416
[vsco] update gallery URL pattern
5 years ago
19ae6f3fc4
update test results
...
- twitter:
Don't test the whole kwdict, only the actual content, since the
keyword hash changes whenever that user changes his display name.
- khinsider:
Download host changed
5 years ago
cc5079c844
[hiperdex] add chapter and manga extractors ( closes #606 )
5 years ago
64bdec8430
[deviantart] check availability of intermediary URLs ( fixes #609 )
5 years ago
5607dd3646
[hitomi] follow multiple redirects
5 years ago
6e759fe2f5
reword note for 'extractor.deviantart.refresh-token' ( #612 )
5 years ago
765b2a0527
[hentaihand] add extractors ( closes #605 )
5 years ago
d94215d119
[tumblr] replace '-' with ' ' in tag searches ( fixes #611 )
...
To search for tags with actual minus signs in them
(there shouldn't be too many,) manually replace those
with url-encoded minus characters ('-' -> '%2d')
before inputting them into gallery-dl:
https://s679874.tumblr.com/tagged/tag-with-minus
->
https://s679874.tumblr.com/tagged/tag%2dwith%2dminus
5 years ago
5cdf1b1319
fix --verbose/--quiet
...
caused by 383795b
5 years ago
78e8d33c97
release version 1.13.0
5 years ago
e6cd49e78b
update extractor test results
5 years ago
90e4c645ba
[formatter] allow multiple "special" format specifiers ( #595 )
...
It is now, for example, possible to specify multiple replacement
operations per format replacement field: {name:Ra/b/Rc/d/}
5 years ago
5d9437b398
[vsco] skip "invalid" entities
5 years ago
650f2b6d58
[furaffinity] accept sfw.furaffinity.net URLs ( closes #608 )
...
Just as an alias for regular URLs with no extra content filtering.
5 years ago
219c4cc78c
[formatter] allow for numeric list and string indices
5 years ago
7d1da614d9
[formatter] implement field name alternatives ( #525 )
...
The format string '{a|b|c}' will now try to use the value from 'a' and
fall back to 'b' and 'c' if accessing a field raises an exception or
if its value is None.
5 years ago
67ea80222c
link to post processor example in 'pixiv.ugoira' docs
5 years ago
74e684e828
[twitter] change default value for 'videos' to 'true'
...
Every other 'videos' option defaulted to 'true', except Twitter.
5 years ago
ab13f81fd7
add example to 'cookies' docs
5 years ago
c7cf9dd111
[furaffinity] support classic layout ( #284 )
5 years ago
138135c190
[furaffinity] add extractors ( #284 )
5 years ago
b9c574bd1d
[patreon] log skipped files ( #590 )
5 years ago
80ea9104b8
[8kun] adjust URL pattern
5 years ago
c76c8b765a
[cloudflare] unescape challenge URL
5 years ago
ce26070231
[pixiv] reduce calls to '/user/detail'
5 years ago
Mike Fährmann