06aaedded5
[twitter] extract 'source' metadata ( #4459 )
1 year ago
e0829ff0fd
[twitter] add 'date_original' metadata for retweets ( #4337 , #4443 )
1 year ago
2b88ad19e9
[twitter] accept 'x.com' URLs ( #4452 )
1 year ago
089d1a4f67
[twitter] fix 'TweetWithVisibilityResults' ( #4369 )
1 year ago
fb3f0453db
[twitter] improve error messages for single Tweets ( #4369 )
...
also fixes '"quoted": false' not having any effect
1 year ago
7fbc304ae9
[twitter] fix crash on private user ( #4349 )
1 year ago
a383eca7f6
decouple extractor initialization
...
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().
This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
1 year ago
d97b8c2fba
consistent cookie-related names
...
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
1 year ago
90231f2d5a
[twitter] add 'tweet-endpoint' option ( #4307 )
...
use the newer TweetResultByRestId only for guests by default
1 year ago
20ed647f6f
[twitter] add 'user' extractor and 'include' option ( #4275 )
1 year ago
86be197d11
[twitter] remove '/search/adaptive.json'
1 year ago
0b08e2e8a8
merge #4287 : [twitter] Fix following extractor not getting all users
1 year ago
f6553ffd2f
[twitter] simplify '_pagination_users'
...
- remove 'stop' variable
- call 'cursor.startswith()' only once
1 year ago
a27dbe8c82
[twitter] use 'TweetResultByRestId' endpoint ( #4250 )
...
allows accessing single Tweets without login
1 year ago
d3d639a159
[twitter] don't treat missing 'TimelineAddEntries' as fatal ( #4278 )
1 year ago
c321c773f2
make the code less ugly
1 year ago
a437a34bcf
fix lint i guess?
1 year ago
6cbc434b54
Fix users pagination
1 year ago
1bf9f52c99
[twitter] add 'ratelimit' option ( #4251 )
1 year ago
f86fdf64a6
[twitter] use GraphQL search by default ( #4264 )
1 year ago
c1cce4a80b
[twitter] extend 'conversations' option ( #4211 )
1 year ago
54cf1fa3e7
[twitter] use GraphQL search endpoint ( #3942 )
...
for guest users; selectable with 'search-endpoint' option.
adapted from 9c7b888ffa
1 year ago
864a654b25
[twitter] update query hashes
1 year ago
45cc7cee1a
[twitter] better error message for guest searches ( #3942 )
1 year ago
271f23d971
[twitter] extract 'conversation_id' metadata ( #3839 )
1 year ago
d0184fddcf
[twitter] optimize '_extract_twitpic()'
...
- use findall instead of finditer
- store URLs in a dict to discard duplicates
1 year ago
3dc862c7fc
merge #3796 : [twitter] extract TwitPic URLs in text ( #3792 )
1 year ago
1d505b39f8
[twitter] support 'profile-conversation' entries ( #3938 )
1 year ago
f500b45b5e
[twitter] improve 480bc34e
...
only check for double user assignment where necessary
1 year ago
480bc34e54
[twitter] do not overwrite previously assigned users ( #3922 )
1 year ago
f5a59c4170
[twitter] add 'date_bookmarked' metadata ( #3816 )
1 year ago
1c1f6fdc80
[twitter] fix regression from 160335ad
...
Tweets from 'homeConversation' or 'conversationthread' entries do not
contain a 'sortIndex' field. Accessing it raises a KeyError and would
erroneously get them labeled as 'deleted'.
1 year ago
160335ad44
[twitter] add 'date_liked' metadata for liked Tweets ( #3816 )
1 year ago
6d850ce629
[twitter] calculate 'date' from Tweet IDs
...
20 times faster than parsing 'created_at'
1 year ago
dbe06cdba1
[twitter] warn about 'withheld' Tweets and users ( #3864 )
1 year ago
3cc1dd1572
[twitter] update query hashes
1 year ago
3846ce0de5
[twitter] update to bookmark timeline v2 ( #3859 )
1 year ago
e6cb92864a
[twitter] allow setting custom features per API endpoint
1 year ago
bebbff6578
fix: graphql_timeline_v2_bookmark_timeline cannot be null
...
twitter: 400 Bad Request (The following features cannot be null: graphql_timeline_v2_bookmark_timeline)
1 year ago
197882cf12
[twitter] add 'hashtag' extractor ( #3783 )
2 years ago
d4fb4ff47f
[twitter] extract TwitPic URLs in text ( #3792 )
...
also ignore previously seen URLs
2 years ago
2bb937014f
[twitter] fall back to legacy /media endpoint when not logged in
2 years ago
b68094d326
[twitter] support 'note_tweet's
2 years ago
3dcabc97ed
[twitter] update API endpoints and parameters
2 years ago
9037128315
[twitter] fix some 'original' retweets not downloading ( #3744 )
2 years ago
dd884b02ee
replace json.loads with direct calls to JSONDecoder.decode
2 years ago
1ae48a54f8
[twitter] add 'transform' option
2 years ago
ab58c375b4
[twitter] fix search ( #3536 )
...
- partially revert 18fe4b334d
2 years ago
9683d79bb7
[twitter] "fix" search pagination ( #3536 , #3534 )
...
- properly process instructions
- do not expect a predetermined instruction order
2 years ago
4fec848858
[twitter] use "browser": "firefox" by default ( #3522 )
...
and reenable TLS 1.2 ciphers
2 years ago
Mike Fährmann