Mike Fährmann
7440d1f112
[pixiv] add 'translated-tags' option ( closes #1354 )
...
(a lot more straight forward than I thought ...)
4 years ago
Mike Fährmann
8974f0361c
[pixiv] update ( #1304 )
...
- remove login with username & password
- require a refresh token
- add 'oauth:pixiv' functionality
See also:
- https://github.com/upbit/pixivpy/issues/158
- https://gist.github.com/ZipFile/c9ebedb224406f4f11845ab700124362
4 years ago
Mike Fährmann
193dca2ce1
update extractor test results
4 years ago
Mike Fährmann
c008cb5100
[pixiv] add 'related' option ( #1237 )
4 years ago
Mike Fährmann
3bd08acc8f
[pixiv] output debug message on failed login attempt
...
(#1192 )
4 years ago
Mike Fährmann
b58e605dc7
raise error when required username or password are missing
...
do not try to login as 'None' (#1192 )
4 years ago
Mike Fährmann
968d3e8465
remove '&' from URL patterns
...
'/?&#' -> '/?#' and '?&#' -> '?#'
According to https://www.ietf.org/rfc/rfc3986.txt , URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
4 years ago
Mike Fährmann
844793847c
update extractor test results
4 years ago
Mike Fährmann
844502cad5
update extractor test results
4 years ago
Mike Fährmann
a45fbc38ea
[pixiv] implement 'avatar' option ( #595 , #623 )
5 years ago
Mike Fährmann
ce26070231
[pixiv] reduce calls to '/user/detail'
5 years ago
Mike Fährmann
b7eb6cecbb
[pixiv] handle tags at the end of new bookmark URLs
5 years ago
Mike Fährmann
40fe062851
[pixiv] fix user id for bookmarks API calls ( closes #596 )
5 years ago
Mike Fährmann
91aaaf1a9e
[pixiv] add 'rating' metadata field ( #595 )
...
A human-friendlier representation of 'x_restrict'
5 years ago
Mike Fährmann
71851a6241
[pixiv] update URLs of followed users to the new format
5 years ago
Mike Fährmann
b9cbf932b4
[pixiv] update URL patterns ( fixes #568 )
...
Pixiv now uses new URLs for
- user profiles and illustration listings:
- https://www.pixiv.net/en/users/ <ID>
- https://www.pixiv.net/en/users/ <ID>/artworks
- bookmarks:
- https://www.pixiv.net/en/users/ <ID>/bookmarks/artworks
5 years ago
Mike Fährmann
ce54b8c04c
let extractors opt-out of cookie option usage
...
useful to avoid sending unnecessary cookies when all authentication
is done through OAuth tokens
5 years ago
Mike Fährmann
1f2a69f3c5
add '_extractor' information to redirect results
5 years ago
Mike Fährmann
a27f43dad1
[pixiv] wait and retry after rate limit error ( closes #535 )
5 years ago
Mike Fährmann
c0f391a4e2
[pixiv] support listing followed users ( #515 )
5 years ago
Mike Fährmann
1f209da4c0
[pixiv] match new search URLs ( closes #507 )
5 years ago
Mike Fährmann
4409d00141
embed error messages in StopExtraction exceptions
5 years ago
Mike Fährmann
22bac14452
[pixiv] match '/artworks/' URLs
5 years ago
Mike Fährmann
f9285f99e6
[pixiv] fix authentication
5 years ago
Mike Fährmann
32447d0d24
[pixiv] simplify default filename format
...
(#366 )
5 years ago
Mike Fährmann
ad3ac02fbc
[pixiv] update metadata entries ( #366 )
...
- change 'num' to a simple enumerating integer
- change default filename format
- provide content of the old 'num' field as 'suffix'
- add 'filename' for ugoira
5 years ago
Mike Fährmann
64b2935d8e
[pixiv] provide 'filename' and change default filename format
...
to '{filename}.{extension}' (closes #366 )
5 years ago
Mike Fährmann
fdec59f8e2
replace extractor.request() 'expect' argument
...
with
- 'fatal': allow 4xx status codes
- 'notfound': raise NotFoundError on 404
5 years ago
Mike Fährmann
b0e85a42e3
apply workaround from 4736912
in parse_datetime() itself
5 years ago
Mike Fährmann
4736912d4e
[pixiv] work around strptime limitations in Python < 3.7
...
"%z" doesn't allow a colon separator in older Python versions:
- "+0900" is OK
- "+09:00" raises an exception
5 years ago
Mike Fährmann
80fdb11508
[pixiv] add 'date' metadata field ( closes #248 )
5 years ago
Mike Fährmann
a2af2d2965
adjust cache maxage values
6 years ago
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor
6 years ago
Mike Fährmann
6284731107
simplify extractor constants
...
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
6 years ago
Mike Fährmann
34bab080ae
rewrite URL patterns to use only 1 per extractor
6 years ago
Mike Fährmann
7471933d5f
use extractor.request for all other API calls
...
- deviantart
- pawoo
- pixiv
- reddit
6 years ago
Mike Fährmann
f6bf66f72c
[pixiv] create directory for each "work" item ( #136 )
6 years ago
Mike Fährmann
8faf03ed84
[pixiv] use refresh_token based authentication
...
The first login will still use username and password, but everything
afterwards will use the refresh_token obtained from that.
This will prevent pixiv from sending a "New login to pixiv" email every
time a new access_token is requested.
6 years ago
Mike Fährmann
f4df6c2396
[pixiv] remove 'type' and 'page' query parameter handling
...
The "new and improved" /member_illust.php and /bookmark.php listings
don't quite work with how things were.
6 years ago
Mike Fährmann
b8c97d2295
use 'extractor.request()' for more HTTP requests
6 years ago
Mike Fährmann
91340d9d27
[pixiv] fix ugoira test
6 years ago
Mike Fährmann
eb7a1f3b98
[pixiv] rework ugoira handling
...
Frame information now gets attached to the ZIP file's keyword dict
instead of being written to a separate text file.
6 years ago
Mike Fährmann
a62bd81e9b
[pixiv] fix filter for 'type=all'
6 years ago
Mike Fährmann
1ff626db97
[pixiv] improve bookmark extraction
...
- combine 'favorite' and 'bookmark' extractors
- it is now one extractor class, but its subcategory still
distinguishes between your own bookmarks ('bookmark') and other
user's bookmarks ('favorite') like before
- allow filtering by bookmark tags and public/private bookmarks
- fix pagination for bookmark results
6 years ago
Mike Fährmann
0a1863fce3
[pixiv] respect more query parameters for user URLs
...
The API endpoint responsible for user illustrations does not
provide sufficient filter capabilities* to match the actual
website, so we are spinning our own filters.
Respected parameters are
'type': illust, manga, ugoira
'tag' : any image tag (this was already supported)
'p' : the page to start on
*
- API can filter for illustrations and manga, but not for ugoira.
- 'offset' is applied before filtering
- no 'tag' filter
6 years ago
Mike Fährmann
b8e53b8c6b
[pixiv] move query parsing out of constructor
...
better exception handling, among other things
6 years ago
Mike Fährmann
909d105ae6
[pixiv] add extractor for illusts from followed users
6 years ago
Mike Fährmann
fe69d01083
[pixiv] add extractor for search results
6 years ago
Mike Fährmann
247f785af1
[pixiv] use App API
...
Transitioning to the App API breaks favorites archive IDs (there is
no longer any bookmark ID information), but the favorites API endpoint
of the public API was gone anyways ...
6 years ago
Mike Fährmann
0e3883303f
[pixiv] implement AppAPI wrapper
6 years ago