thatfuckingbird
141ca4ac0a
[pixiv] also save untranslated tags when translated-tags is enabled ( #1501 )
3 years ago
beesdotjson
5ad615f0db
fix PixivFavoriteExtractor regex ( #1405 )
...
* fix PixivFavoriteExtractor regex
* do not use lookbehind
4 years ago
Mike Fährmann
7440d1f112
[pixiv] add 'translated-tags' option ( closes #1354 )
...
(a lot more straight forward than I thought ...)
4 years ago
Mike Fährmann
8974f0361c
[pixiv] update ( #1304 )
...
- remove login with username & password
- require a refresh token
- add 'oauth:pixiv' functionality
See also:
- https://github.com/upbit/pixivpy/issues/158
- https://gist.github.com/ZipFile/c9ebedb224406f4f11845ab700124362
4 years ago
Mike Fährmann
193dca2ce1
update extractor test results
4 years ago
Mike Fährmann
c008cb5100
[pixiv] add 'related' option ( #1237 )
4 years ago
Mike Fährmann
3bd08acc8f
[pixiv] output debug message on failed login attempt
...
(#1192 )
4 years ago
Mike Fährmann
b58e605dc7
raise error when required username or password are missing
...
do not try to login as 'None' (#1192 )
4 years ago
Mike Fährmann
968d3e8465
remove '&' from URL patterns
...
'/?&#' -> '/?#' and '?&#' -> '?#'
According to https://www.ietf.org/rfc/rfc3986.txt , URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
4 years ago
Mike Fährmann
844793847c
update extractor test results
4 years ago
Mike Fährmann
844502cad5
update extractor test results
4 years ago
Mike Fährmann
a45fbc38ea
[pixiv] implement 'avatar' option ( #595 , #623 )
5 years ago
Mike Fährmann
ce26070231
[pixiv] reduce calls to '/user/detail'
5 years ago
Mike Fährmann
b7eb6cecbb
[pixiv] handle tags at the end of new bookmark URLs
5 years ago
Mike Fährmann
40fe062851
[pixiv] fix user id for bookmarks API calls ( closes #596 )
5 years ago
Mike Fährmann
91aaaf1a9e
[pixiv] add 'rating' metadata field ( #595 )
...
A human-friendlier representation of 'x_restrict'
5 years ago
Mike Fährmann
71851a6241
[pixiv] update URLs of followed users to the new format
5 years ago
Mike Fährmann
b9cbf932b4
[pixiv] update URL patterns ( fixes #568 )
...
Pixiv now uses new URLs for
- user profiles and illustration listings:
- https://www.pixiv.net/en/users/ <ID>
- https://www.pixiv.net/en/users/ <ID>/artworks
- bookmarks:
- https://www.pixiv.net/en/users/ <ID>/bookmarks/artworks
5 years ago
Mike Fährmann
ce54b8c04c
let extractors opt-out of cookie option usage
...
useful to avoid sending unnecessary cookies when all authentication
is done through OAuth tokens
5 years ago
Mike Fährmann
1f2a69f3c5
add '_extractor' information to redirect results
5 years ago
Mike Fährmann
a27f43dad1
[pixiv] wait and retry after rate limit error ( closes #535 )
5 years ago
Mike Fährmann
c0f391a4e2
[pixiv] support listing followed users ( #515 )
5 years ago
Mike Fährmann
1f209da4c0
[pixiv] match new search URLs ( closes #507 )
5 years ago
Mike Fährmann
4409d00141
embed error messages in StopExtraction exceptions
5 years ago
Mike Fährmann
22bac14452
[pixiv] match '/artworks/' URLs
5 years ago
Mike Fährmann
f9285f99e6
[pixiv] fix authentication
5 years ago
Mike Fährmann
32447d0d24
[pixiv] simplify default filename format
...
(#366 )
5 years ago
Mike Fährmann
ad3ac02fbc
[pixiv] update metadata entries ( #366 )
...
- change 'num' to a simple enumerating integer
- change default filename format
- provide content of the old 'num' field as 'suffix'
- add 'filename' for ugoira
5 years ago
Mike Fährmann
64b2935d8e
[pixiv] provide 'filename' and change default filename format
...
to '{filename}.{extension}' (closes #366 )
5 years ago
Mike Fährmann
fdec59f8e2
replace extractor.request() 'expect' argument
...
with
- 'fatal': allow 4xx status codes
- 'notfound': raise NotFoundError on 404
5 years ago
Mike Fährmann
b0e85a42e3
apply workaround from 4736912
in parse_datetime() itself
5 years ago
Mike Fährmann
4736912d4e
[pixiv] work around strptime limitations in Python < 3.7
...
"%z" doesn't allow a colon separator in older Python versions:
- "+0900" is OK
- "+09:00" raises an exception
5 years ago
Mike Fährmann
80fdb11508
[pixiv] add 'date' metadata field ( closes #248 )
5 years ago
Mike Fährmann
a2af2d2965
adjust cache maxage values
6 years ago
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor
6 years ago
Mike Fährmann
6284731107
simplify extractor constants
...
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
6 years ago
Mike Fährmann
34bab080ae
rewrite URL patterns to use only 1 per extractor
6 years ago
Mike Fährmann
7471933d5f
use extractor.request for all other API calls
...
- deviantart
- pawoo
- pixiv
- reddit
6 years ago
Mike Fährmann
f6bf66f72c
[pixiv] create directory for each "work" item ( #136 )
6 years ago
Mike Fährmann
8faf03ed84
[pixiv] use refresh_token based authentication
...
The first login will still use username and password, but everything
afterwards will use the refresh_token obtained from that.
This will prevent pixiv from sending a "New login to pixiv" email every
time a new access_token is requested.
6 years ago
Mike Fährmann
f4df6c2396
[pixiv] remove 'type' and 'page' query parameter handling
...
The "new and improved" /member_illust.php and /bookmark.php listings
don't quite work with how things were.
6 years ago
Mike Fährmann
b8c97d2295
use 'extractor.request()' for more HTTP requests
6 years ago
Mike Fährmann
91340d9d27
[pixiv] fix ugoira test
6 years ago
Mike Fährmann
eb7a1f3b98
[pixiv] rework ugoira handling
...
Frame information now gets attached to the ZIP file's keyword dict
instead of being written to a separate text file.
6 years ago
Mike Fährmann
a62bd81e9b
[pixiv] fix filter for 'type=all'
6 years ago
Mike Fährmann
1ff626db97
[pixiv] improve bookmark extraction
...
- combine 'favorite' and 'bookmark' extractors
- it is now one extractor class, but its subcategory still
distinguishes between your own bookmarks ('bookmark') and other
user's bookmarks ('favorite') like before
- allow filtering by bookmark tags and public/private bookmarks
- fix pagination for bookmark results
6 years ago
Mike Fährmann
0a1863fce3
[pixiv] respect more query parameters for user URLs
...
The API endpoint responsible for user illustrations does not
provide sufficient filter capabilities* to match the actual
website, so we are spinning our own filters.
Respected parameters are
'type': illust, manga, ugoira
'tag' : any image tag (this was already supported)
'p' : the page to start on
*
- API can filter for illustrations and manga, but not for ugoira.
- 'offset' is applied before filtering
- no 'tag' filter
6 years ago
Mike Fährmann
b8e53b8c6b
[pixiv] move query parsing out of constructor
...
better exception handling, among other things
6 years ago
Mike Fährmann
909d105ae6
[pixiv] add extractor for illusts from followed users
6 years ago
Mike Fährmann
fe69d01083
[pixiv] add extractor for search results
6 years ago