Mike Fährmann
3ecb512722
send Referer headers by default
1 year ago
Mike Fährmann
a453335a9f
remove test results in extractor modules
...
and add generic example URLs
1 year ago
Mike Fährmann
1d2b5d0c60
update test comment positions
...
always put them above the test they're referring to
1 year ago
Mike Fährmann
df5c7ee03e
[deviantart] fix search ( #4384 )
...
send correct usernames instead of 'u'
1 year ago
Mike Fährmann
a383eca7f6
decouple extractor initialization
...
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().
This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
1 year ago
Mike Fährmann
d97b8c2fba
consistent cookie-related names
...
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
1 year ago
Mike Fährmann
22099422ca
[deviantart] fix shortened URLs ( #4316 )
1 year ago
Mike Fährmann
98c9fdb414
[deviantart] revert e9353c63; retry downloads with private token
1 year ago
Mike Fährmann
e9353c63d6
[deviantart] keep using private access tokens
...
for deviations returned from a private API call
also fixes a bug from 0a7eee3e
where '_pagination()'
would never switch from unspecified (None) to private access token
1 year ago
Mike Fährmann
0a7eee3ee0
[deviantart] add 'public' option
1 year ago
Mike Fährmann
34699fbf64
[deviantart:search] detect login redirects ( #3860 )
1 year ago
ClosedPort22
c489aecb3e
[deviantart] add support for fxdeviantart.com URLs
...
fxdeviantart.com is a service that fixes embeds on Discord, similar to
fxtwitter.com
2 years ago
Mike Fährmann
6b03506655
[deviantart] allow searching when not logged in
2 years ago
Mike Fährmann
3fa456d989
[deviantart] remove mature scraps warning ( #3691 )
...
warn about private deviations
when paginating over eclipse results
2 years ago
Mike Fährmann
51301e0c31
replace remaining time.sleep() calls
...
with Extractor.sleep() or request_interval
2 years ago
Mike Fährmann
6ed4309aba
[deviantart] add 'gallery-search' extractor ( #1695 )
2 years ago
Mike Fährmann
e1df7f73b1
[deviantart] add 'search' extractor
...
(#538 , #1264 , #2954 , #2970 , #3577 )
Requires login to fetch any results, since the API endpoint raises an
error for not logged in requests.
TODO: parse HTML search results
2 years ago
Mike Fährmann
b4899c266f
merge #3656 : [deviantart] fix crash when handling deleted deviations in status updates
2 years ago
Mike Fährmann
725baedad3
[deviantart] use '/collections/all' endpoint for favorites
...
(#3666 ,#3668)
2 years ago
ClosedPort22
f4e211356d
[deviantart] slight refactor
2 years ago
ClosedPort22
a74114ef7a
[deviantart] fix crash when handling deleted deviations
...
in status updates
2 years ago
Mike Fährmann
7e2fd2e573
merge #3560 : [deviantart] add support for /deviation/ and fav.me URLs
2 years ago
ClosedPort22
c90b4ea8d9
[deviantart] add support for fav.me URLs
2 years ago
ClosedPort22
674c719646
[deviantart] refactor base36 conversion
2 years ago
ClosedPort22
293abb8921
[deviantart] add support for /deviation/ URLs
2 years ago
ClosedPort22
597b89245e
[deviantart] misc improvements to status extractor
...
- relax regex pattern
- handle invalid 'items' field
- add a test for shared sta.sh item
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2 years ago
ClosedPort22
013733c9e9
[deviantart] fix index fields for embedded/shared images
2 years ago
ClosedPort22
c4aeca7a5a
[deviantart] improve handling of statuses
...
- recursively yield statuses
- ignore items with missing or unexpected field(s)
2 years ago
ClosedPort22
3b32671fbd
[deviantart] add extractor for status updates
...
extract user status updates using the '/user/statuses/' endpoint
2 years ago
Mike Fährmann
77df8d3116
[deviantart] implement username&password login for scraps ( #1029 )
...
re-login when getting prematurely logged out by dA
is missing at the moment
2 years ago
Mike Fährmann
8d7585534e
Merge pull request #3367 from the-blank-x/deviantart-view
...
[deviantart] add /view URL support
2 years ago
blankie
6614d94b08
[deviantart] add /view URL support
2 years ago
ClosedPort22
bf3fd5951a
Merge branch 'master' into da-extra-stash
2 years ago
Mike Fährmann
cd931e1139
update extractor test results
2 years ago
ClosedPort22
13d825731e
[deviantart] fix test for sta.sh URL extraction
...
Without the 'count' assertion, the test would be essentially useless.
2 years ago
ClosedPort22
6356c9be96
[deviantart] extract sta.sh URLs from 'text_content'
2 years ago
Mike Fährmann
b0cb4a1b9c
replace 'text.extract()' with 'text.extr()' where possible
2 years ago
Mike Fährmann
2a1cb403ee
Revert "[Deviantart] [ #1776 ] Remove the "you need session cookies to download mature scraps" warning ( #1777 )"
...
This reverts commit 1f02878351
.
Mature scraps do yet again require cookies.
2 years ago
Mike Fährmann
14717f3fc9
[deviantart] add 'group' option ( #3018 )
...
disabling this option allows to better download from deleted accounts
2 years ago
Mike Fährmann
a12ce2bb41
[deviantart] fix 'deviation' extraction ( #2981 )
2 years ago
Mike Fährmann
6c76b5f90c
[deviantart] fix extraction ( #2981 , #2983 )
...
send a 'csrf_token' with every Eclipse API request
2 years ago
Mike Fährmann
43ec315a7f
[deviantart] use public access token for journals ( #2702 )
...
and retry with a private token if needed
2 years ago
Mike Fährmann
c6a9bab019
update extractor test results
2 years ago
Mike Fährmann
4f7fe9b4be
[deviantart] fix folder listings with 'pagination: manual'
...
(#2488 )
2 years ago
Mike Fährmann
97e8a15295
[deviantart] implement 'pagination' option ( #2488 )
2 years ago
Mike Fährmann
1f9a0e2fd8
update extractor test results
2 years ago
Mike Fährmann
b5236656d5
[deviantart] notify users about registering an oauth application
...
if they get repeated 429 errors and are using default API credentials
3 years ago
Mike Fährmann
d441888bfb
[deviantart] adjust API endpoints
...
Start all endpoints with a forward slash '/'
to be consistent with other API interfaces.
3 years ago
Mike Fährmann
8f0cf0bf71
[deviantart] use '/browse/newest' for most-recent searches
...
(#2096 )
3 years ago
Mike Fährmann
8bea02c38c
[deviantart] fix 'index' values for stashed deviations
3 years ago