2749 Commits (1e59aa61239bde6612c776b25ec42d999a32ab5e)

 

All Branches   

Search

Author	SHA1	Message	Date
Mike Fährmann	1e59aa6123	update README.rst and setup.py ... - remove superfluous '-'s - use definition list for config paths - add Python 3.9 support to setup.py classifiers	4 years ago
Mike Fährmann	b8f2e42f7a	update pip install instructions	4 years ago
Mike Fährmann	476d563ec2	[downloader:http] add MIME type and signature for .swf files	4 years ago
Mike Fährmann	a00b60fbe7	[twitter] update 'x-csrf-token' header (fixes #1170) ... Twitter started using a bigger (80 instead of 16 bytes) CSRf token for logged in users, and expects those to be used as 'x-csrf-token' header when send via 'ct0' cookie. Generating an 80 byte token ourselves doesn't work, and Twitter will still insist on using its own.	4 years ago
Mike Fährmann	b88c97b873	[instagram] add 'cursor' option (#1149) ... To enable at least 'some' way to continue downloading from the middle of a user profile listing.	4 years ago
Mike Fährmann	0d406c8daf	[common] restrict values used in 'generate_extractors()'	4 years ago
Mike Fährmann	fe0265c7a5	[downloader.http] small improvements to file signature list ... - specify multiple entries for gif, mp3, zip - add entries for pdf	4 years ago
Mike Fährmann	b2c55f0a72	[sankaku] remove login support ... The old login method for 'https://chan.sankakucomplex.com/user/login' and the cookies it produces have no effect on the results from 'beta.sankakucomplex.com'.	4 years ago
Mike Fährmann	7f3d811d7b	[moebooru] inherit from BooruExtractor	4 years ago
Mike Fährmann	a3a863fc13	[booru] add generalized extractors for *booru sites ... similar to cc15fbe7	4 years ago
Mike Fährmann	5f23441e12	[piczel] update API URLs	4 years ago
Mike Fährmann	47114339a2	[webtoons] update 'ageGate' cookie	4 years ago
Mike Fährmann	4225f12783	[nozomi] handle empty 'date' fields (fixes #1163)	4 years ago
Mike Fährmann	2b93515ee0	[instagram] reimplement support for stories (#1149)	4 years ago
Mike Fährmann	ecdea799dd	[sankaku] use 'beta.sankakucomplex.com' API endpoints	4 years ago
Mike Fährmann	b3ecc89a9a	[instagram] use double quotes for strings when possible	4 years ago
Mike Fährmann	76285eb60d	[instagram] reimplement support for story highlights (#1149)	4 years ago
Mike Fährmann	8ca7f54750	rename '_request_…' variables ... - remove '_' at the beginning - _request_last -> request_timestamp	4 years ago
Mike Fährmann	15a122aff3	[instagram] update 'X-IG-WWW-Claim' headers	4 years ago
Mike Fährmann	e5d81bdc7b	[mangadex] handle 'external' chapters (closes #1154)	4 years ago
Mike Fährmann	447488fb18	[instagram] rewrite ... (#1113, #1122, #1128, #1130, #1149) Rely on the results of GraphQL queries instead of requesting data for each post separately via '/p/<shortcode>/?__a=1'. This might result in some missing metadata, and there might be some issues for '/channel/' and '/saved/' URLs, but at least downloading from the regular post listings should work without issues and without getting users blocked/banned. TODO: reimplement support for stories	4 years ago
Mike Fährmann	cc15fbe71a	[moebooru] add generalized extractors for moebooru sites ... - add support for sakugabooru.com (closes #1136) - add support for lolibooru.moe (closes #1050) This allows users to dynamically add support for moebooru/myimouto based sites by adding an entry to their config file (like for foolslide, foolfuuka, etc) For example: { "extractor": { "moebooru": { "new-site-1": {"root": "https://site1.net"}, "new-site-2": {"root": "https://www.site2.moe"} } } }	4 years ago
Mike Fährmann	43120407cc	[paheal] create directory for each post (closes #1147)	4 years ago
Mike Fährmann	63e61a0932	[twitter] update image URL format (#1145) ... use '/<name>?format=<fmt>&name=<size>' instead of the potentially deprecated '/<name>.<fmt>:<size>' but keep all of them as fallback URLs	4 years ago
Mike Fährmann	1a4b61f7eb	[downloader:http] fix issues with chunked transfer encoding ... (fixes #1144)	4 years ago
Mike Fährmann	536c088462	[downloader:http] improve 'adjust-extensions' (#776) ... Check file headers against a list of file signatures before downloading the whole file and writing it to disk. The file signature check needs some improvements (*), but it produces usable results for the most part. (*) - 'webp', 'wav', and others start with 'RFFI' - 'svg' uses the same "signature" as all XML documents - 'webm' has the same signature as 'mkv' files - only 'mp3' files in an ID3v2 container get recognized	4 years ago
Mike Fährmann	46323ae6ff	initialize 'hooks' as empty tuple ... follow-up to 9c29fc4e Prevent a "race" between initializing 'pathfmt' and 'hooks', and receiving a signal in between (e.g. ctrl+c), which would then crash in 'handle_finalize()'.	4 years ago
Mike Fährmann	06af57e84a	update CHANGELOG and README for 1.15.4	4 years ago
Mike Fährmann	9c29fc4e55	always initialize DownloadJob.hooks (fixes #1135) ... and not just when any (potential) post processors are defined	4 years ago
Mike Fährmann	ae6a1d5fbc	[mangoxo] fix extraction 2	4 years ago
Mike Fährmann	0bc492c0fa	add docs for 'event' and 'filename' options ... from 9c3568c3 and ca59bd69	4 years ago
Mike Fährmann	f6a684bc37	[hentainexus] update data decoding procedure (#1125)	4 years ago
Mike Fährmann	c57a918f4a	[e621] implement delay via '_request_interval_min'	4 years ago
Mike Fährmann	93ce7466e2	[2chan] skip external links	4 years ago
Mike Fährmann	547107307e	fix 'Metadata' messages in result tests	4 years ago
Mike Fährmann	b214e89b5c	[mangoxo] fix extraction	4 years ago
Mike Fährmann	578dcf805c	[mangapanda] don't force https://	4 years ago
Mike Fährmann	102c482f5e	[reddit] skip invalid/failed gallery items (fixes #1127)	4 years ago
Mike Fährmann	174945d2b2	[hentainexus] fix extraction (fixes #1125)	4 years ago
Mike Fährmann	ca59bd691c	[postprocessor:metadata] add 'event' and 'filename' options	4 years ago
Mike Fährmann	9c3568c397	[postprocessor:exec] add 'event' option ... and remove 'final' option -- use '"event": "finalize"' instead.	4 years ago
Mike Fährmann	9fffa9c343	rework post processor callbacks	4 years ago
Mike Fährmann	d6986be8b0	Move CI to GitHub Actions	4 years ago
Mike Fährmann	f99c6031e0	apply post processor blacklists/whitelists to basecategories ... (#1103)	4 years ago
Mike Fährmann	1e3dd7330e	merge SharedConfigMixin functionality into Extractor	4 years ago
Mike Fährmann	ddfb4fd07a	[twitter] use 'https://twitter.com/i/api/' for logged in users ... Doesn't seem to make a difference from what I can tell, i.e. downloaded files are the same, but the website does it.	4 years ago
Mike Fährmann	42ccae53c4	[mangadex] switch to API v2 ... https://mangadex.org/api/v2/ https://mangadex.org/thread/351011	4 years ago
Mike Fährmann	ca44111726	[flickr] update ... - ensure every photo has an 'owner' (#828) - change default directories to a more consistent schema - create directory for each photo	4 years ago
Mike Fährmann	9b1bd09454	change 'extension-map' default ... Replace all JPEG filename extensions with 'jpg'.	4 years ago
Mike Fährmann	e5438b8a29	release version 1.15.3	4 years ago