14867dad6b
[twitter] fix unified cards from search results
3 years ago
dffa440ede
[twitter] improve handling of deleted tweets ( #2212 )
3 years ago
54ef874ba4
[twitter] fix retweet filter ( #2212 )
3 years ago
cb43f7731b
[twitter] update to GraphQL API ( #2212 )
...
The old REST API endpoints, which were not used by Twitter since
summer 2021, are going to finally be phased out it seems, with
'/2/timeline/profile/USERID.json' being the first one.
Only Twitter's search doesn't have a GraphQL interface yet.
3 years ago
f2e8aedd74
[twitter] changes to 'cards' option
...
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
when 'cards' is set to "ytdl"
"cards": true --> only download card images
"cards": "ytdl" --> download card images and
use youtube_dl on otherwise unsupported cards
3 years ago
df2f0c09bb
[twitter] support "image_carousel_website" unified cards
3 years ago
f587458a3c
[twitter] include '4096x4096' as a default image fallback
...
(closes #2107 , closes #1881 )
3 years ago
ab8eea1a24
[twitter] fix extractor for direct image links ( fixes #2030 )
3 years ago
4377f1c284
[twitter] distinguish between fatal & nonfatal errors ( #2020 )
...
only show a warning for nonfatal errors
and do not raise a StopExtraction exception
3 years ago
9156e90f1f
[twitter] add 'pinned' option
3 years ago
cd66c3c415
[twitter] add 'size' option ( #1881 )
3 years ago
94143eb86c
[twitter] add 'quote_by' metadata field ( #1481 )
...
Only present for tweets quoted by another tweet.
Represents the tweet_id of said tweet quoting this one.
3 years ago
da16eabb82
[twitter] ensure card entries have a 'url' ( #1868 )
3 years ago
0fd959a2a7
[twitter] support '/with_replies' URLs ( closes #1833 )
3 years ago
6651da27e9
[twitter] fix 'url' extraction for users without 'expanded_url'
...
(#1532 , #1787 )
3 years ago
ae78d95a5f
[twitter] fix issue when filtering quote tweets ( #1792 )
...
When a user quotes his own Tweet and that Tweet gets filtered by
'"quoted": false', it could also get filtered when it appeared later
as regular Tweet.
3 years ago
0817f468ef
[twitter] expand t.co links in user descriptions ( #1532 , #1787 )
3 years ago
7c0ae88185
[twitter] add 'url' to user objects ( #1532 , #1787 )
3 years ago
5919dc5b5a
[twitter] slightly improve '_transform_user()'
3 years ago
6b56b3ebe1
[twitter] report API errors as generic StopExtraction exceptions
...
prevents duplicate logging messages for nonexistent users
(#1759 )
3 years ago
c866fcba48
[twitter] fix 'logout' ( #1719 )
...
delete 'auth_token' cookie and cookies.txt path
3 years ago
52984f7e22
[twitter] add option to log out when blocked ( #1719 )
3 years ago
e5a93e113f
[twitter] extend 'replies' option ( #1254 )
...
Allow setting 'replies to '"self"' to only download from self-replies.
3 years ago
229498b8aa
[twitter] warn about suspended accounts etc ( closes #1759 )
3 years ago
414bdc95a3
[twitter] set 'retweet_id' for original retweets ( #1481 )
3 years ago
5323c1c73a
[twitter] ensure guest tokens are returned as string ( #1665 )
3 years ago
035562bd11
[twitter] remove old-style URLs from image fallback lists
3 years ago
a751afdfb3
[twitter] change some defaults
...
- 'retweets' option: true -> false
- 'quoted' option : true -> false
i.e. disable downloading tweets from other user's timelines by default
- search directory:
'["{category}", "Search", "{search}"]' ->
'["{category}", "{user[name]}"]'
i.e. change it to the same as other twitter extractors (#1308 )
3 years ago
b5affc62aa
[twitter] rename 'text-only' to 'text-tweets' ( #570 )
3 years ago
724ca61f36
[twitter] add 'text-only' option ( #570 )
3 years ago
394fbb5f56
[twitter] strip useless t.co links ( #1532 )
...
The 'full_text' of Tweets with media content usually ends with a t.co
link to itself. This commit removes those.
3 years ago
41457dbb1b
[twitter] resolve t.co URLs in 'content' ( #1532 )
3 years ago
17b0ccb071
[twitter] add missing retweet media entities ( fixes #1555 )
...
from the original tweets
3 years ago
fd858eed7b
[twitter] add 'user_likes' metadata field for liked tweets
...
i.e. the 'screen_name' of the user whose liked tweets get extracted.
Ideally this would replace 'user' or at least be in the same format,
but that would break backwards compatibility or be impossible/too
complicated thanks to API result differences.
(#1421 )
4 years ago
8d124a3766
[twitter] rename variables
4 years ago
105f3c9666
[twitter] add extractor for direct image links ( closes #1417 )
4 years ago
ebd142e2a8
[twitter] don't use youtube-dl for cards when videos are disabled
...
(#1416 )
4 years ago
ccfa5a8694
[twitter] better error message when logging in with 2FA ( #1409 )
4 years ago
2846235669
[twitter] allow specifying a custom format for user results
...
(#1337 )
4 years ago
3378b39719
[twitter] implement 'users' option ( #1337 )
4 years ago
5d69e437d0
[twitter] add option to download all media from a conversation
...
(fixes #1319 )
4 years ago
de0656941b
[twitter] add extractor for followed users ( #1337 )
...
https://twitter.com/USER/following or
https://twitter.com/id:USERID/following
4 years ago
5542a11c46
[twitter] update GraphQL endpoints
4 years ago
24e8e398e0
[twitter] skip login if 'auth_token' cookie is present
4 years ago
95e5911895
[twitter] match '/i/user/ID' URLs
4 years ago
069b113cbf
[twitter] improve and fix retry after hitting rate limit
...
- replace recursive call with infinite loop
- fix function arguments for recursive call
4 years ago
780b6adb91
rename 'generate_csrf_token()' to just 'generate_token()'
...
and add a 'size' argument
4 years ago
25074aec47
[twitter] fetch media from pinned tweets ( #1203 )
4 years ago
2475176d99
[twitter] fetch tweets from 'homeConversation' entries
...
When logged in, some entries returned by Twitter's API are so called
'homeConversation's (they would be regular tweet entries otherwise.)
Those weren't picked up before and resulted in missing files compared
to accessing a timeline as guest.
('/media' timelines and search results were not affected)
4 years ago
3af9350648
[twitter] update API calls
...
- use 'https://twitter.com/i/api ' for all requests
except '/guest/activate.json'
- update (default) URL parameters
- update GraphQL endpoints
4 years ago
Mike Fährmann