Commit Graph

2343 Commits (daeef8a5e3e680dae42d3b99162f6d89394ef8cc)

Author SHA1 Message Date
Mike Fährmann 999efec5cc
[deviantart] limit API wait times to 2**9=512 seconds (#721)
4 years ago
Mike Fährmann 504de79d8b
[vsco] fix extraction
4 years ago
Mike Fährmann 5e2974d699
[weibo] add 'videos' option
4 years ago
Mike Fährmann 9f638c2e01
[twitter] add 'replies' option (closes #705)
4 years ago
Mike Fährmann 19a7afdd9b
[downloader:http] add MIME types for .psd files (closes #714)
4 years ago
Mike Fährmann fc3e54275b
[patreon] respect filters and sort order in query params (#711)
4 years ago
Mike Fährmann 46b9a4d8ff
[patreon] improve hash extraction (#693, #713)
4 years ago
Mike Fährmann c56a751dae
[newgrounds] fix URLs produced by 'followng' extractors (#684)
4 years ago
Mike Fährmann baa877d853
release version 1.13.5
4 years ago
Mike Fährmann a4fd620a25
[hiperdex] revert domain back to hiperdex.com
4 years ago
Mike Fährmann 233b6f93a2
[patreon] recognize URLs with creator IDs (#711)
4 years ago
Mike Fährmann 38b6bd66b0
[500px] match 'web.500px.com' subdomains
4 years ago
Mike Fährmann d3b3b30107
update test results
4 years ago
Mike Fährmann 5d7ca76885
retry Cloudflare challenges
4 years ago
Mike Fährmann 3eab07739f
[twitter] ensure videos have a 'filename'
4 years ago
Mike Fährmann c4371a6970
[twitter] add 'reply' metadata field (#705)
4 years ago
Mike Fährmann 12ff23b6cc
[mastodon] improve account searches (fixes #704)
4 years ago
Mike Fährmann 400a0df661
[jaiminisbox] update decoding procedure (fixes #702)
4 years ago
Mike Fährmann 8fe858eb0e
improve parameter extraction when solving Cloudflare challenge
4 years ago
Mike Fährmann fb98b567fa
[gelbooru] improve post ID extraction for pools
4 years ago
Mike Fährmann d6facdee7b
[mastodon] add tests (#701)
4 years ago
Mike Fährmann 12eebb6f16
[xhamster] support xhamster.porncache.net domains (closes #700)
4 years ago
Mike Fährmann e749402191
[mastodon] fix pagination (#701)
4 years ago
Mike Fährmann 921914141e
[imgbb] improve redirect handling
4 years ago
Mike Fährmann 6cc800aad4
[instagram] add 'post_id' and 'num' metadata fields (closes #698)
4 years ago
Mike Fährmann a3de234e70
[hitomi] add extractor for tag searches (closes #697)
4 years ago
Mike Fährmann 456f6e8d05
[nozomi] move '_unpack()' method to global scope
4 years ago
Mike Fährmann 55ac408bdf
[hitomi] fix extraction of galleries without tags
4 years ago
Mike Fährmann db6685eeae
[aryion] support downloading from folders (fixes #694)
4 years ago
Mike Fährmann fa2952ac55
[furaffinity] add 'following' extractor (#515)
4 years ago
Mike Fährmann 9b194520db
[newgrounds] add 'following' extractor (closes #684)
4 years ago
Mike Fährmann 6386ee54e1
[deviantart] add extractor info to 'following' results
4 years ago
Mike Fährmann d5273f9b0c
[hiperdex] update domain to hiperdex.net
4 years ago
Mike Fährmann 08674a91f3
[patreon] fix hash extraction from download URLs (closes #693)
4 years ago
Mike Fährmann a31c1aae72
release version 1.13.4
5 years ago
Mike Fährmann a6286bb551
[hiperdex] add 'artist' extractor (#606)
5 years ago
Mike Fährmann 291033720a
[hiperdex] fix manga extraction
5 years ago
Mike Fährmann dfc0557807
[vsco] fix collection extraction
5 years ago
Mike Fährmann fd438f0d78
update extractor test results
5 years ago
Mike Fährmann bae1e8ed12
[deviantart] fix JPEG quality replacement pattern
5 years ago
Mike Fährmann cf4cef3d63
[aryion] adjust 'date' to UTC time
5 years ago
Mike Fährmann a0f4c295c0
add optional 'utcoffset' argument to 'parse_datetime()'
5 years ago
Mike Fährmann 6c531be294
[aryion] fix malformed 'last-modified' headers (#390)
5 years ago
Mike Fährmann 38bc6430d3
[downloader:http] don't overwrite existing '_mtime' fields
5 years ago
Mike Fährmann dc65f7d8dc
[aryion] use generic download URLs (#390)
5 years ago
Mike Fährmann 96b78bcf04
[aryion] include path in default directory format (#390)
5 years ago
Mike Fährmann 300264f676
read config files from PyInstaller exe directory (closes #682)
5 years ago
Mike Fährmann 6143050980
[aryion] add gallery and post extractors (#390, #673)
5 years ago
Mike Fährmann 9e7dfc0cfc
[myportfolio] fix extraction of galleries without title
5 years ago
Mike Fährmann 88fca0a172
[mastodon] update OAuth credentials for pawoo.net (#665)
5 years ago
Mike Fährmann 4ae8a25567
[mastodon] use 'combine_dict()' to combine extractor info dicts
5 years ago
Mike Fährmann 220c06b86e
[mastodon] handle rate limits
5 years ago
Mike Fährmann d02f7c1118
improve Extractor.wait()
5 years ago
Mike Fährmann 5d7404ab58
[oauth] use the new name for 'DeviantartAPI' (fixes #670)
5 years ago
Mike Fährmann 762c758af4
[hiperdex] fix extraction
5 years ago
Mike Fährmann f9a590f92b
[deviantart] apply HTTP request limits in more places
5 years ago
Mike Fährmann 2587296deb
[mastodon] add access tokens for mastodon.social and baraag.net
5 years ago
Mike Fährmann ff7c0b7eff
[deviantart] handle "Request blocked" errors (#655)
5 years ago
Mike Fährmann c874684f05
[deviantart] retrieve *all* download URLs through OAuth API
5 years ago
Mike Fährmann 5c27b25a8f
[deviantart] improve sta.sh extraction
5 years ago
Mike Fährmann e2fc4eaa6f
[deviantart] detect stash folders (fixes #659)
5 years ago
Mike Fährmann c034159701
[piczel] fix extraction for single images
5 years ago
Mike Fährmann 699036ea0c
[weibo] accept status URLs with non-numeric IDs (#664)
5 years ago
Mike Fährmann fe96f99e4b
[hentainexus] reduce line length (flake8) & update test
5 years ago
墨焓 6f81cac8fa
Add metadata to hentainexus: circle, event, title_conventional. (#661)
5 years ago
Mike Fährmann 3ed72f82dc
release version 1.13.3
5 years ago
Mike Fährmann 6f911aeb1c
[deviantart] add error message for cloudFront blocks (#655)
5 years ago
Mike Fährmann 7499d71d02
[simplyhentai] ignore certificate errors in video test
5 years ago
Mike Fährmann 4203dc0bdc
[mangapark] fix metadata extraction
5 years ago
Mike Fährmann 6ecb0a19cf
handle sys.stdin being None when using '-' as input file (#653)
5 years ago
Mike Fährmann 1b82d36ab2
[deviantart] handle decode errors for extended_fetch results (#655)
5 years ago
Mike Fährmann 09f2271528
[35photo] add 'tag' extractor
5 years ago
Mike Fährmann 77fda8190c
[35photo] simplify/remove tests for the 'genre' extractor
5 years ago
Mike Fährmann 4bc161ca0f
prevent crash when sys.stdout and co. are None (#653)
5 years ago
Mike Fährmann fb846c9ee5
[instagram] reduce line lengths and make flake8 happy
5 years ago
Mike Fährmann ad2efa8509
[e621] derive from Danbooru extractors (#651)
5 years ago
Mike Fährmann 9b39e1cd7e
[e621] fix bug in API rate limiting (#651)
5 years ago
Mike Fährmann b607d0ad7f
[twitter] fix typo in 'x-twitter-auth-type' header (#625)
5 years ago
Mike Fährmann 9159cb8fb3
remove trailing dots and spaces from directory names (#647)
5 years ago
Mike Fährmann 2c3b9e1450
[nozomi] support multiple images per post (#646)
5 years ago
Mike Fährmann c606d0c854
[instagram] update pattern for user profile URLs
5 years ago
Mike Fährmann 2530db3f4d
[mangadex] transform 'date' timestamps to datetime objects
5 years ago
Mike Fährmann ae2a33243b
[newgrounds] catch general Exceptions
5 years ago
Mike Fährmann 32e36d8f02
[sexcom] replace tests
5 years ago
Mike Fährmann 33b42dc847
[nozomi] sort search results (fixes #646)
5 years ago
Mike Fährmann eaa60a438b
[piczel] fix extraction
5 years ago
Mike Fährmann 5bcc7184c9
[danbooru][e621] increase page limits
5 years ago
Mike Fährmann 90d15e3682
[instagram] use 'itertools.chain()'
5 years ago
Leonardo Taccari 160328d21c
[instagram] Add support for user's saved medias (#644)
5 years ago
Mike Fährmann e0b0e8d62a
release version 1.13.2
5 years ago
Mike Fährmann d3482ace7f
[furaffinity] extract more metadata
5 years ago
Mike Fährmann f6c5edb76b
pre-compile regex pattern for remove_html() and split_html()
5 years ago
Mike Fährmann fdd2dd5136
[kabeuchi] add 'user' extractor (closes #561)
5 years ago
Mike Fährmann 59edcdc822
[hitomi] restore metadata fields from before f33b13a
5 years ago
Mike Fährmann 2d5703c493
[twitter] use a simpler data structure to store cookies in cache
5 years ago
Mike Fährmann 87d4f83597
[newgrounds] make post extraction nonfatal
5 years ago
Mike Fährmann 823fbeaae6
[newgrounds] add 'favorite' extractor (#394)
5 years ago
Mike Fährmann a45fbc38ea
[pixiv] implement 'avatar' option (#595, #623)
5 years ago
Mike Fährmann a63a376ad2
[mangoxo] fix login
5 years ago
Mike Fährmann ebc70e87ce
[e621] update to new interface / API endpoints (closes #635)
5 years ago
Mike Fährmann d1cf7ccdb3
[instagram] add 'post_shortcode' metadata field (#525)
5 years ago
Mike Fährmann 32df8d06fe
[twitter] add 'bookmark' extractor (closes #625)
5 years ago
Mike Fährmann 3fb41c34c8
[bcy] reduce requests to '/item/detail/<id>' (#613)
5 years ago
Mike Fährmann f33b13aacf
[hitomi] simplify metadata extraction
5 years ago
Mike Fährmann 115fd2c6f2
"fix" incomplete MIME types (#632)
5 years ago
Mike Fährmann 72122eb9b3
release version 1.13.1
5 years ago
Mike Fährmann adcd7cb24a
[downloader:http] add another MIME type for '.rar' files (#628)
5 years ago
Mike Fährmann ce5e2a58fe
[imgbb] update test results
5 years ago
Mike Fährmann f117e32910
[danbooru] restore 'popular' functionality
5 years ago
Mike Fährmann 39b48d665b
[hiperdex] use proper name for 'chapter_minor'
5 years ago
Mike Fährmann 8fbbaa54ff
[bcy] fix partial image URLs (#613)
5 years ago
Mike Fährmann 86c00f9e66
[danbooru] move extractor logic from booru.py
5 years ago
Mike Fährmann 1d4a369ea2
update extractor test results
5 years ago
Mike Fährmann 7625912b31
[piczel] improve and update
5 years ago
Mike Fährmann ec85bf90de
use context managers in cache.py & add tests
5 years ago
Mike Fährmann 913b8333cc
write DeviantArt refresh-tokens to cache (#616)
5 years ago
Mike Fährmann 2a4f227e08
warn about expired cookies
5 years ago
Mike Fährmann 34887ae139
fix bugs in DatabaseCacheDecorator.update()/.invalidate()
5 years ago
Mike Fährmann 380b693fad
[downloader:http] add more MIME types for '.bmp' files (#621)
5 years ago
Mike Fährmann 4e361b3008
add tests for specific datetime values
5 years ago
Mike Fährmann 80ecb99089
[hitomi] fix extraction
5 years ago
Mike Fährmann 247c9e1416
[vsco] update gallery URL pattern
5 years ago
Mike Fährmann 19ae6f3fc4
update test results
5 years ago
Mike Fährmann cc5079c844
[hiperdex] add chapter and manga extractors (closes #606)
5 years ago
Mike Fährmann 64bdec8430
[deviantart] check availability of intermediary URLs (fixes #609)
5 years ago
Mike Fährmann 5607dd3646
[hitomi] follow multiple redirects
5 years ago
Mike Fährmann 765b2a0527
[hentaihand] add extractors (closes #605)
5 years ago
Mike Fährmann d94215d119
[tumblr] replace '-' with ' ' in tag searches (fixes #611)
5 years ago
Mike Fährmann 5cdf1b1319
fix --verbose/--quiet
5 years ago
Mike Fährmann 78e8d33c97
release version 1.13.0
5 years ago
Mike Fährmann e6cd49e78b
update extractor test results
5 years ago
Mike Fährmann 90e4c645ba
[formatter] allow multiple "special" format specifiers (#595)
5 years ago
Mike Fährmann 5d9437b398
[vsco] skip "invalid" entities
5 years ago
Mike Fährmann 650f2b6d58
[furaffinity] accept sfw.furaffinity.net URLs (closes #608)
5 years ago
Mike Fährmann 219c4cc78c
[formatter] allow for numeric list and string indices
5 years ago
Mike Fährmann 7d1da614d9
[formatter] implement field name alternatives (#525)
5 years ago
Mike Fährmann 74e684e828
[twitter] change default value for 'videos' to 'true'
5 years ago
Mike Fährmann c7cf9dd111
[furaffinity] support classic layout (#284)
5 years ago
Mike Fährmann 138135c190
[furaffinity] add extractors (#284)
5 years ago
Mike Fährmann b9c574bd1d
[patreon] log skipped files (#590)
5 years ago
Mike Fährmann 80ea9104b8
[8kun] adjust URL pattern
5 years ago
Mike Fährmann c76c8b765a
[cloudflare] unescape challenge URL
5 years ago
Mike Fährmann ce26070231
[pixiv] reduce calls to '/user/detail'
5 years ago
Mike Fährmann da0d5f6092
[oauth] add 'port' option (#604)
5 years ago
Mike Fährmann 719b63d0ca
[bcy] add user and post extractors (#592)
5 years ago
Mike Fährmann 6426e3efc7
[khinsider] fix and improve metadata extraction
5 years ago
Mike Fährmann 4a3d2405de
[postprocessor:ugoira] small optimization
5 years ago
Mike Fährmann b7eb6cecbb
[pixiv] handle tags at the end of new bookmark URLs
5 years ago
Mike Fährmann 109f6c8685
[patreon] filter duplicate files per post (#590)
5 years ago
Mike Fährmann b38cf59711
[sexcom] fix image URLs & parse 'date' fields
5 years ago
Mike Fährmann 1f4c9c5f9d
[8kun] add thread and board extractors (closes #582)
5 years ago
Mike Fährmann facc5daa6d
[twitter] force old login page layout (fixes #584, fixes #598)
5 years ago
Mike Fährmann d1de7dc296
[hitomi] implement workaround for "broken" redirects
5 years ago
Mike Fährmann 40fe062851
[pixiv] fix user id for bookmarks API calls (closes #596)
5 years ago
Mike Fährmann 91aaaf1a9e
[pixiv] add 'rating' metadata field (#595)
5 years ago
Mike Fährmann dff33b260c
[reddit] add 'videos' option
5 years ago
Mike Fährmann 2ad43618cc
[piczel] fix extraction
5 years ago
Mike Fährmann cf7a67d67f
[yaplog] remove module
5 years ago
Mike Fährmann e0dd073ce0
[twitter] replace embedded tweet test
5 years ago
Mike Fährmann ec36df4851
[deviantart] fix video extraction from 'extended_fetch' results
5 years ago
Mike Fährmann 48be2266ed
[deviantart] better error message for 'extended_fetch' (#585)
5 years ago
Mike Fährmann 383795b550
prevent superfluous calls to Logger.makeRecord()
5 years ago
Mike Fährmann 71851a6241
[pixiv] update URLs of followed users to the new format
5 years ago
Mike Fährmann d086f30b42
[reddit] restore archive keys for i.redd.it images
5 years ago
Mike Fährmann 56f1c96168
implement 'parent-directory' option (#551)
5 years ago
Mike Fährmann ae07f92f7e
[reddit] rewrite extractor logic (closes #551)
5 years ago
Mike Fährmann 2852691d78
[paheal] replace test URL
5 years ago
Mike Fährmann 2a9be48511
improve util.load/save_cookiestxt() and add tests
5 years ago
Mike Fährmann e35c2ea1a6
[weibo] use youtube-dl to download from m3u8 manifests
5 years ago
Mike Fährmann 6703b8a86b
[blogger] implement video extraction (closes #587)
5 years ago
Mike Fährmann c1a6862863
implement functions to load/save cookies.txt files (closes #586)
5 years ago
Mike Fährmann 5d73b7f29c
release version 1.12.3
5 years ago
Mike Fährmann 37247dbaff
miscellaneous fixes
5 years ago
Mike Fährmann 0e9dc5c88e
fix AttributeError when accessing 'temppath'
5 years ago
Mike Fährmann 25d5ec4ff3
[twitter] add option to extract TwitPic embeds (#579)
5 years ago
Mike Fährmann 254f7c3999
implement a post-processor module to compare file versions
5 years ago
Mike Fährmann 32d7195d08
[pinterest] improve detection of invalid pin.it links
5 years ago
Mike Fährmann 0b84068d84
remove temp files before downloading from fallback URLs
5 years ago
Mike Fährmann 760b9b4db4
add remove_file() and remove_directory() helpers
5 years ago
Mike Fährmann b2d542ad40
improve PathFormat._enum_file()
5 years ago
Mike Fährmann 174117f827
allow multiple hashes for content tests
5 years ago
Alice f498a9057f [twitter] Fix stop before real end (#573)
5 years ago
Mike Fährmann 8bb32ee188
[hitomi] fix image URLs
5 years ago
Mike Fährmann bd5ce9855c
allow GalleryExtractors to set URL-independent extensions
5 years ago
Mike Fährmann af42c75152
[mangadex] revert domain to 'mangadex.org'
5 years ago
Mike Fährmann 200aea308a
[downloader:common] enable 'job'/'extractor' for logging messages
5 years ago
Mike Fährmann e89413da22
update test results
5 years ago
Mike Fährmann 33a6e0ac6e
[hentaifoundry] extract more metadata (closes #565)
5 years ago
Mike Fährmann 5cac79c3d9
[erolord] remove extractor
5 years ago
Mike Fährmann b9cbf932b4
[pixiv] update URL patterns (fixes #568)
5 years ago
Mike Fährmann 9d369de592
release version 1.12.2
5 years ago
Mike Fährmann 988cc2ec23
[mangadex] change domain to mangadex.cc (closes #559)
5 years ago
Mike Fährmann f8e137d6b4
[deviantart] show warning about private deviations only once
5 years ago
Mike Fährmann 939fec8ecd
[deviantart] match new search/popular URLs (closes #538)
5 years ago
Mike Fährmann 09cc88b715
[deviantart] match '/favourites/all' URLs (closes #555)
5 years ago
Mike Fährmann 3811fd8a25
fix time formatting for Python 3.4 and 3.5
5 years ago
Mike Fährmann 43ab9572b4
[twitter] handle API rate limits (#526)
5 years ago
Mike Fährmann 569747a78d
implement extractor.wait()
5 years ago
Mike Fährmann 5532e9c158
[twitter] handle quoted tweets (#526)
5 years ago
Mike Fährmann 0b4cb8e57a
[mangahere] send 'isAdult' cookie (fixes #556)
5 years ago