Commit Graph

2134 Commits (5c27b25a8f08adef99e765d97915bc72241903de)

Author SHA1 Message Date
Mike Fährmann a27f43dad1
[pixiv] wait and retry after rate limit error (closes #535)
5 years ago
Mike Fährmann 6b373cb7e2
[exhentai] restrict default directory name length (#545)
5 years ago
Mike Fährmann b347bf68c7
[deviantart] add extractor for followed users (#515)
5 years ago
Mike Fährmann c0f391a4e2
[pixiv] support listing followed users (#515)
5 years ago
Mike Fährmann 2e2fc7f0ad
prevent infinite recursion when spawning extractors (closes #489)
5 years ago
Mike Fährmann 896896a490
[twitter] fix URLs forwarded to youtube-dl (closes #540)
5 years ago
Mike Fährmann 1e2713b895
[artstation] fix search result pagination (closes #537)
5 years ago
Mike Fährmann bf3df3d0b0
[directlink] send Referer headers (closes #536)
5 years ago
Mike Fährmann 83909ab5d4
release version 1.12.1
5 years ago
Mike Fährmann 9be7ff600e
[imagetwist] replace test image
5 years ago
Mike Fährmann 66905b1664
[foolslide] add fallback for chapter data extraction
5 years ago
Mike Fährmann 48e42e73fb
[reddit] change default value for 'comments' to '0'
5 years ago
Mike Fährmann 9c0928457a
[reddit] fix errors with 't1_…' submissions
5 years ago
Mike Fährmann 58391d492d
cache archive keys generated in __contains__() (#524)
5 years ago
Mike Fährmann bf658fd84b
[vsco] implement 'videos' option
5 years ago
Mike Fährmann 95c90722ee
[instagram] implement 'videos' option (closes #521)
5 years ago
Mike Fährmann 1921c127a5
make OSErrors during file downloads nonfatal (closes #512)
5 years ago
Mike Fährmann d0920e84e9
update test results
5 years ago
Mike Fährmann 8c11e81c9f
Merge commit '63e6993716db8d8bedfb7b0d445c7161493046b6'
5 years ago
Mike Fährmann 63e6993716
merge 'bypost' functionality into metadata postprocessor
5 years ago
Mike Fährmann 31a29835ff
[realbooru] simplify extractors and update tests (#514)
5 years ago
The Oddball 9a4ce20b8e [realbooru] Add Realbooru extractor (#514)
5 years ago
Mike Fährmann f9e74320de
retain trailing zeroes in Cloudflare challenge answers
5 years ago
Mike Fährmann 72b8fbfbad
[instagram] make post-page extraction nonfatal
5 years ago
Mike Fährmann 922b8a9595
[weibo] raise NotFoundError for unavailable/deleted statuses
5 years ago
Mike Fährmann 0cd157300e
[patreon] fix regex pattern for posts
5 years ago
Mike Fährmann fe19e233f3
[xvideos] improve
5 years ago
Mike Fährmann d3e44e899d
raise NotFoundErrors for 404 responses in GalleryExtractors
5 years ago
Mike Fährmann a4dd8b3dab
improve _check_cookies()
5 years ago
Mike Fährmann 76e60d10a6
[patreon] raise proper exception if creator/post doesn't exist
5 years ago
Mike Fährmann 9e63804347
[patreon] make retrieving user info nonfatal (#508)
5 years ago
Mike Fährmann 964dc57286
[vsco] improve image resolutions
5 years ago
Mike Fährmann 0629fe8fa4
[vsco] fix user profile extraction … again
5 years ago
Mike Fährmann ab17ea9632
[deviantart] only print warning if 'original' is enabled
5 years ago
Mike Fährmann 2188db6284
[gelbooru] fix non-API tag extraction
5 years ago
Mike Fährmann c4702ec9b6
simplify some logging calls
5 years ago
Gio c0b9ad678d Separate metadata from handle_url into handle_metadata, commenting
5 years ago
Mike Fährmann c9ef1b21c3
[patreon] get partial user info without /api/user/<id> (#507)
5 years ago
Mike Fährmann 0ab9bb1721
[4chan] add extractor for entire boards (closes #510)
5 years ago
Mike Fährmann c59b98c81b
[downloader:http] improve rate limit handling
5 years ago
Mike Fährmann bbbafc1c24
[downloader:http] catch both possible SSLException instances
5 years ago
Gio c20bb5c338 Naming convention, as per travis.
5 years ago
Gio 6ed4fc07ff Don't print intentional metadata skips to the console.
5 years ago
Gio cfc70a97ab Added an additional channel for downloading the metadata of an entire post or gallery.
5 years ago
Mike Fährmann f451be48c3
release version 1.12.0
5 years ago
Mike Fährmann 15f9bb3d14
add option to disable pyOpenSSL usage (#508)
5 years ago
Mike Fährmann c8e99e3b3b
[deviantart] fix crash on missing "token" field (#505)
5 years ago
Mike Fährmann 6ed2c7823c
[deviantart] disable original downloads if no cookies set
5 years ago
Mike Fährmann 50deab5265
[deviantart] fix URL generation from /extended_fetch results
5 years ago
Mike Fährmann 1f209da4c0
[pixiv] match new search URLs (closes #507)
5 years ago
Mike Fährmann e17907ee2a
change default value of 'cookies-update' to 'true'
5 years ago
Mike Fährmann 07dafad26d
[twitter] attempt to fix infinite loops (#499)
5 years ago
Mike Fährmann 71acbdabf4
[2chan] fix metadata extraction
5 years ago
Mike Fährmann c0a1241648
[livedoor] force https:// for image URLs
5 years ago
Mike Fährmann 6e23c0da09
[imgur] add extractor for subreddit links (closes #500)
5 years ago
Mike Fährmann 38c05df290
[oauth] add custom/default indicator to log messages (#501)
5 years ago
Mike Fährmann 372ffe95ee
[oauth] adjust Flickr redirect URI (fixes #503)
5 years ago
Mike Fährmann 004812258d
[hentaifox] fix extraction
5 years ago
Mike Fährmann e2710702d4
fix Cloudflare bypss
5 years ago
Mike Fährmann 8759403f37
[plurk] add delay between comment requests
5 years ago
Mike Fährmann a28552fd19
update test results
5 years ago
Mike Fährmann dcaa3d01bd
[imagefap] adapt to new image URL format
5 years ago
Mike Fährmann e62c209ca0
[nijie] fix 'date' parsing
5 years ago
Mike Fährmann 3bba763ab9
[twitter] improve
5 years ago
Mike Fährmann 26d2334550
[postprocessor:metadata] rename 'format' to 'content-format'
5 years ago
Mike Fährmann a412531451
[postprocessor:metadata] implement 'extension-format' option
5 years ago
Mike Fährmann 0f1538af78
split filename formatting into its own function
5 years ago
Mike Fährmann db35c3b581
[directlink] separate filenames from paths
5 years ago
Mike Fährmann 41a3169c67
[foolfuuka] use '{extension}' in default filename format
5 years ago
Mike Fährmann e9aed62c91
[imgur] unescape image titles
5 years ago
Mike Fährmann bca2222559
add '--exec-after'
5 years ago
Mike Fährmann ed6592ea1a
remove '--abort-on-skip'
5 years ago
Mike Fährmann 2c332edaad
[plurk] fix comment pagination
5 years ago
Mike Fährmann a3fa45bbb1
[behance] get images from 'media_collection' modules
5 years ago
Mike Fährmann 359c3bc1c5
[deviantart] revert to getting download URLs from OAuth API
5 years ago
Mike Fährmann 42b9633c7e
update test results
5 years ago
Mike Fährmann b28bd1c73e
[bobx] set generated session cookie (closes #482)
5 years ago
Mike Fährmann ae09f87602
improve SharedConfigMixin config lookups
5 years ago
Mike Fährmann b5c964332b
improve config.py test coverage
5 years ago
Mike Fährmann f5604492c3
update interface of config functions
5 years ago
Mike Fährmann 4ca883c66f
[smugmug] replace test for custom URLs
5 years ago
Mike Fährmann d45fabb79d
match user profile handling on deviantart and newgrounds
5 years ago
Mike Fährmann ea80dadd09
[deviantart] restore archive keys
5 years ago
Mike Fährmann 3fc1e12949
[postprocessor:metadata] filter private entries
5 years ago
Mike Fährmann ea094692c8
[vsco] fix collection extraction (#480)
5 years ago
Mike Fährmann 490831f84a
[bobx] "fix" image download URLs
5 years ago
Mike Fährmann 978cb03f81
update misc test results
5 years ago
Mike Fährmann fca87974fe
[sexcom] fix video downloads by sending specific Referer headers
5 years ago
Mike Fährmann bbbeff4c41
[downloader.http] implement file-specific HTTP headers
5 years ago
Mike Fährmann edc080468d
[instagram] make 'video_url' fields optional (fixes #479)
5 years ago
Mike Fährmann 9fdc5e74cb
[deviantart] ensure consistent username capitalization (#455)
5 years ago
Mike Fährmann b1f0609de5
[newgrounds] rewrite (#394)
5 years ago
Mike Fährmann 3ece3976ae
[newgrounds] implement login support (#394)
5 years ago
Mike Fährmann 3a07c06865
[newgrounds] update
5 years ago
Mike Fährmann 5513b66eb0
[vsco] fix user profile extraction
5 years ago
Mike Fährmann abfcb356fc
[flickr] support 3k, 4k, 5k, and 6k photo sizes (closes #472)
5 years ago
Mike Fährmann 521fcd2eb9
[imgbb] fix error in galleries without user info (closes #471)
5 years ago
Mike Fährmann 8061263d4c
[imgbb] improve pagination logic
5 years ago
Mike Fährmann da6789b2b0
disable unique archive id checks for some tests
5 years ago
Mike Fährmann 67e54ed8ea
release version 1.11.1
5 years ago
Mike Fährmann ce98a86c0e
fix data file inclusion in source distributions
5 years ago
Mike Fährmann 6c86fbfe2a
release version 1.11.0
5 years ago
Mike Fährmann 94a94f3b86
miscellaneous stuff
5 years ago
Mike Fährmann b0197098e6
[imgur] get title from webpage if missing in API response
5 years ago
Mike Fährmann dd5d2b2eac
[deviantart] add user profile extractor (#377, #419)
5 years ago
Mike Fährmann a437e78620
[deviantart] minimize cookie usage during scraps extraction
5 years ago
Mike Fährmann 1a197d2195
store the original cookiejar as Extractor._cookiejar
5 years ago
Mike Fährmann de83ae4576
make 'method' argument of Extractor.request keyword-only
5 years ago
Mike Fährmann a5be08a830
[downloader:ytdl] forward proxy settings
5 years ago
Mike Fährmann 4325695d74
[luscious] expand GraphQL queries
5 years ago
Mike Fährmann 94dbdbf506
[nijie] change default filename format
5 years ago
Mike Fährmann 9e88e7a344
[postprocessor:exec] improve (#421, #413)
5 years ago
Mike Fährmann c18fadc221
[instagram] extract videos without youtube-dl (#391)
5 years ago
Mike Fährmann f15eedb634
[sexcom] set Referer header for file downloads (closes #464)
5 years ago
Mike Fährmann 2a3bd4e3c7
rename extractor classes starting with a digit
5 years ago
Mike Fährmann b3b9da6d74
[photobucket] replace test URL
5 years ago
Mike Fährmann 64786363be
[4chan] simplify
5 years ago
Mike Fährmann 557e2c018b
[8chan] remove module
5 years ago
Mike Fährmann e14782a948
[instagram] simplify graphql extraction for post pages
5 years ago
Mike Fährmann c01ff78467
[twitter] extend 'videos' option to force extraction with ytdl
5 years ago
Mike Fährmann f8ac67ce50
[hitomi] extend URL pattern + follow redirects
5 years ago
Mike Fährmann e877ca97c3
[naver] adjust directory names and metadata structure
5 years ago
Mike Fährmann 702f2fbd1f
[issuu] add publication and user extractors (#413)
5 years ago
Mike Fährmann 8361d874d7
[hitomi] fix extraction
5 years ago
Mike Fährmann 5fa6ff04dd
[instagram] extract '__additionalDataLoaded' (#391)
5 years ago
Mike Fährmann 5af291ba5c
include failed downloads and child extractors in exit status
5 years ago
Mike Fährmann 322c2e7ed4
renaming variables
5 years ago
Mike Fährmann 87a87bff7e
[simplyhentai] fix image URLs
5 years ago
Mike Fährmann 4409d00141
embed error messages in StopExtraction exceptions
5 years ago
Mike Fährmann d5e3910270
adjust 'util.raises()'
5 years ago
Mike Fährmann d44f790e81
adjust output for HTTP status related errors
5 years ago
Mike Fährmann 03e0cec715
return with non-zero exit status on error
5 years ago
Mike Fährmann c887493a80
overhaul exception stuff
5 years ago
Mike Fährmann 109718a5e3
[blogger] add blog and post extractors (closes #364)
5 years ago
Mike Fährmann 244d396b0b
add '--ugoira-conv-lossless' command-line option (#432)
5 years ago
Mike Fährmann 49a6b1b6c0
[twitter] extract video stream info without youtube-dl (#452)
5 years ago
Mike Fährmann 9f0dbf2a72
[twitter] raise proper exception for protected Tweets
5 years ago
Mike Fährmann 083e14ad9a
[downloader:ytdl] add data from '_ytdl_extra' to info_dicts
5 years ago
Mike Fährmann 6e08ada4fe
[luscious] simplify some metadata entries
5 years ago
Mike Fährmann 9e3a8607ee
[deviantart] update usernames (#455)
5 years ago
Mike Fährmann 2eb38810c5
[twitter] fix image extraction when logged in (#452)
5 years ago
Mike Fährmann 8f38a35b91
[imgur] use API with "public" client_id (#446)
5 years ago
Mike Fährmann b23c822b23
[luscious] use GraphQL
5 years ago
Mike Fährmann ef17d94469
update test results
5 years ago
Mike Fährmann 2057c6ba29
[naver] add blog and post extractors (closes #447)
5 years ago
Mike Fährmann 389d2d7e38
implement 'cookies-update' option (#445)
5 years ago
Mike Fährmann fbc0a6a059
[nozomi] skip unavailable posts (#388)
5 years ago
Mike Fährmann ae98dbcbb3
[nozomi] implement searching for negated terms (#388)
5 years ago
Mike Fährmann 1c03a389df
[twitter] small improvements to search extractor
5 years ago
Mike Fährmann c3042978b8
[deviantart] match "/gallery/all" (closes #449)
5 years ago
Alice bcddcca6db Add search downloading to twitter.py (#448)
5 years ago
Mike Fährmann 1693d97bd3
update extractor class hierarchies
5 years ago
Mike Fährmann 7ebd984e8d
[imgur] print error message if no JSON data is found (#446)
5 years ago
Mike Fährmann 5882b00f2f
[imgur] implement login support (#446)
5 years ago
Mike Fährmann 91643ca54b
[nozomi] add search extractor (#388)
5 years ago
Mike Fährmann df2b3c6888
restore OAuth2 authentication error messages
5 years ago
Mike Fährmann 6779512fc7
[nozomi] add post and tag extractors (#388)
5 years ago
Mike Fährmann 6abe5f5bbb
[patreon] fix pagination (#444)
5 years ago
Mike Fährmann ff1e4a86aa
release version 1.10.6
5 years ago
Mike Fährmann d4ffd6c952
[yaplog] improve metadata extraction (#443)
5 years ago
Mike Fährmann 15af2f8464
[hitomi] fallback to /reader/ page if main page returns 404
5 years ago
Mike Fährmann 8af59a4bba
fix & update docs
5 years ago
Mike Fährmann dc6ad81e2e
[yaplog] prevent crash on empty posts (#443)
5 years ago
Mike Fährmann 94eb7c6cad
[deviantart] fix sta.sh extraction (436)
5 years ago
Mike Fährmann 1032cfa34b
[downloader:http] extend mimetype map with archive formats
5 years ago
Mike Fährmann 27b5b2497e
[deviantart] fix download URLs (#436)
5 years ago
Mike Fährmann 93aac8dfea
[yaplog] fix incomplete image URLs (#443)
5 years ago
Mike Fährmann a782b009b8
[yaplog] match blog names with '-' (#443)
5 years ago
Mike Fährmann cf5e716b9d
[hitomi] fix image URLs
5 years ago
Mike Fährmann ad81c07204
[postprocessor] match logger names of downloader modules
5 years ago
Mike Fährmann 03bc8adfc7
[postprocessor:exec] run after file moved to target location
5 years ago
Mike Fährmann 35958bebd4
[postprocessor:exec] fix filename quoting on Windows (#421)
5 years ago
Mike Fährmann b06c372e4d
[postprocessor:exec] improve; add command-line option (#421)
5 years ago
Mike Fährmann 5a54efa025
[xhamster] unescape 'title' and 'description'
5 years ago
Mike Fährmann 1b9bf4fc6e
[behance] fix 'tags' extraction
5 years ago
Mike Fährmann bb97e87989
[komikcast] ignore banner image
5 years ago
Mike Fährmann 0ff90a3f7d
[gfycat] include title in default filenames (closes #434)
5 years ago
Mike Fährmann fabdc3b0c6
release version 1.10.5
5 years ago
Mike Fährmann de4e2029d1
[nsfwalbum] update test album
5 years ago
Mike Fährmann 1faec285d1
[nijie] further improvements (closes #423)
5 years ago
Mike Fährmann 6d0a533d68
[reddit] respect 'comments:0' for single submissions (#429)
5 years ago
Mike Fährmann 803d8f814e
[oauth] update scope for reddit tokens (#428)
5 years ago
Mike Fährmann 46ba173ded
[reddit] fix documentation inconsistencies (closes #429)
5 years ago
Mike Fährmann 20eb6c401f
[nijie] improvements and fixes (#423)
5 years ago
Mike Fährmann d1ea08c67d
[weibo] fixes and improvements
5 years ago
Mike Fährmann 38d97f3da6
[deviantart] add debug message about API credentials (#424)
5 years ago
Mike Fährmann 80c2104fb5
[deviantart] fix 429 handling if 'fatal' is False (closes #424)
5 years ago
Mike Fährmann 913460240d
[reddit] fix 'extractor.blacklist()' arguments
5 years ago
Mike Fährmann 22bac14452
[pixiv] match '/artworks/' URLs
5 years ago
Mike Fährmann 66cac207ac
[twitter] match and use 'i/web' status URLs
5 years ago
Mike Fährmann 946f2751e2
[reddit] add 'user' extractor (closes #350)
5 years ago
Mike Fährmann c14abb9fb8
[reddit] improve URL parameter handling for subreddit links
5 years ago
Mike Fährmann ee8b654464
[instagram] implement 'highlights' option (closes #329)
5 years ago
Mike Fährmann f63c3097a9
[instagram] rework some code paths
5 years ago
Mike Fährmann 4330133114
[imgur] add 'favorite' extractor (closes #420)
5 years ago
Mike Fährmann ee5e20221f
[imgth] fix image URLs
5 years ago
Mike Fährmann b63b126808
[hentaicafe] extend URL pattern
5 years ago
Mike Fährmann d780f0357e
[imgur] add user extractor
5 years ago
Mike Fährmann 11ea689013
[simplyhentai] fix image and video URLs
5 years ago
Mike Fährmann 15632a1570
[tsumino] fix extraction
5 years ago