2006 Commits (25074aec4730064b86c2314f891b25a06900e25b)

Author	SHA1	Message	Date
Mike Fährmann	93ce7466e2	[2chan] skip external links	4 years ago
Mike Fährmann	b214e89b5c	[mangoxo] fix extraction	4 years ago
Mike Fährmann	578dcf805c	[mangapanda] don't force https://	4 years ago
Mike Fährmann	102c482f5e	[reddit] skip invalid/failed gallery items (fixes #1127)	4 years ago
Mike Fährmann	174945d2b2	[hentainexus] fix extraction (fixes #1125)	4 years ago
Mike Fährmann	1e3dd7330e	merge SharedConfigMixin functionality into Extractor	4 years ago
Mike Fährmann	ddfb4fd07a	[twitter] use 'https://twitter.com/i/api/' for logged in users ... Doesn't seem to make a difference from what I can tell, i.e. downloaded files are the same, but the website does it.	4 years ago
Mike Fährmann	42ccae53c4	[mangadex] switch to API v2 ... https://mangadex.org/api/v2/ https://mangadex.org/thread/351011	4 years ago
Mike Fährmann	ca44111726	[flickr] update ... - ensure every photo has an 'owner' (#828) - change default directories to a more consistent schema - create directory for each photo	4 years ago
Mike Fährmann	de0c57886d	[twitter] add 'list-members' extractor (closes #1096)	4 years ago
Mike Fährmann	904ba08568	[gfycat] fix default filename format	4 years ago
Mike Fährmann	a46561bc16	[500px] update query hashes	4 years ago
Mike Fährmann	2e3a0dff21	[8kun] fix file URLs of older posts (fixes #1101)	4 years ago
Mike Fährmann	00825cddf5	[hentaifoundry] use scheme from input URL (fixes #1095) ... Let the user choose between http and https, instead of always forcing https.	4 years ago
Mike Fährmann	8a98d3549a	[weasyl] create directory for each favorite submission ... (#1032)	4 years ago
Mike Fährmann	91db8df1c7	[deviantart] add 'index_base36' metadata field (closes #1099) ... This is the same ID as found in 'filename' without the 'd' in front, which is just 'index' encoded in base36.	4 years ago
Mike Fährmann	b9bfa4c675	update extractor test results	4 years ago
Mike Fährmann	1b5b789401	[mangoxo] fix metadata extraction	4 years ago
Mike Fährmann	41d4968866	[twitter] add 'list' extractor (#1096)	4 years ago
Mike Fährmann	5d10520f4c	[twitter] update GraphQL endpoint & fix width/height entries	4 years ago
Mike Fährmann	9b2e5f72d6	[exhentai] update image URL parsing (#1094)	4 years ago
Mike Fährmann	98a4d86a01	[sankakucomplex] extract videos and embeds (closes #308)	4 years ago
Mike Fährmann	558cde139c	[paheal] fix extraction (fixes #1088)	4 years ago
Mike Fährmann	0211af7ca8	[hentaifoundry] update 'YII_CSRF_TOKEN' cookie handling ... (fixes #1083)	4 years ago
Mike Fährmann	198c33ec36	also collect post processors from 'basecategory' entries ... (fixes #1084)	4 years ago
Mike Fährmann	350b1afe1c	speed up _list_classes() after iterating over all modules once	4 years ago
Mike Fährmann	18213dc5ba	release version 1.15.2	4 years ago
Mike Fährmann	b788712844	[fallenangels] fix extraction of '.5' chapters	4 years ago
Mike Fährmann	28d8541cb3	[mangafox] ensure download URLs have a scheme	4 years ago
Mike Fährmann	8e3a324c91	[mangakakalot] ignore "Go Home" buttons in chapter pages	4 years ago
Mike Fährmann	c14c5d82d6	[newgrounds] use generator for fallback URLs	4 years ago
Mike Fährmann	968d3e8465	remove '&' from URL patterns ... '/?&#' -> '/?#' and '?&#' -> '?#' According to https://www.ietf.org/rfc/rfc3986.txt, URLs are "organized hierarchically" by using "the slash ("/"), question mark ("?"), and number sign ("#") characters to delimit components"	4 years ago
Mike Fährmann	1686dc1757	[twitter] support media from Cards (#1005, #937) ... Can be enabled with 'extractor.twitter.cards', but for now disabled by default because cards can redirect to rather large videos from YouTube or Twitch.	4 years ago
Mike Fährmann	ffd38215a4	[hitomi] fix image URLs and URL pattern ... - non-webp files are now hosted on [a-c]b.hitomi.la - removed ampersand from invalid slug characters	4 years ago
Mike Fährmann	286718950c	[mangahere] ensure download URLs have a scheme (fixes #1070)	4 years ago
Mike Fährmann	76dfa11a65	[reddit] add 'date' metadata field (closes #1068)	4 years ago
Mike Fährmann	3f2ba629ea	[newgrounds] provide fallback URLs for video downloads (#1042)	4 years ago
Mike Fährmann	a3ca2f6080	update fallback URL handling ... remove Message.Urllist and use a '_fallback' field inside a kwdict	4 years ago
Mike Fährmann	43dab3a228	[mangadex] unescape more metadata fields (fixes #1066) ... like 'manga', 'author', 'artist', etc.	4 years ago
Mike Fährmann	5565025221	[xhamster] fix user profile extraction	4 years ago
Mike Fährmann	07432d6262	[seiga] fix flake8 and cookie test (#1063)	4 years ago
Mike Fährmann	b8daabc3ca	[pinterest] implement login support (closes #1055) ... being logged allows access to secret/protected boards	4 years ago
Mike Fährmann	1b1cf01d0d	add a general 'generate_csrf_token()' function	4 years ago
Mike Fährmann	7a0ba370d1	[gelbooru] rewrite mp4 video URLs (fixes #1048)	4 years ago
Mike Fährmann	6491db3eaf	[blogger] handle URLs with specified width/height (closes #1061) ... get highest quality for images with /wXXX-hXXX/ instead of the usual /sXXX/	4 years ago
Mike Fährmann	783e0af26d	[hentaifoundry] update and simplify	4 years ago
Mike Fährmann	5b844a72b7	[newgrounds] handle embeds without scheme (#1033)	4 years ago
kurumigi	7e0e872f4f	[seiga] Add metadata for single image downloads (#1063) ... * [seiga] Support image metadata. * [seiga] Update test data. * [seiga] Fix cookie check. * [test_cookies] [seiga] Fit test_cookies.py to the last commit.	4 years ago
Zanny	3ec60e894a	[weasyl] api-key authentication (#1057) ... * [weasyl] support api keys * [weasyl] document api-key authentication * [weasyl] usernames can contain ~	4 years ago
Mike Fährmann	844793847c	update extractor test results	4 years ago
Mike Fährmann	ddd6840509	[behance] fix 'collection' extraction	4 years ago
Mike Fährmann	c5e3971b18	[newgrounds] extract image embeds (closes #1033)	4 years ago
dawidsowa	43b156fb40	[reactor] match URLs without subdomain (#1053)	4 years ago
Mike Fährmann	3ebb174f2c	add missing extractor info when spawning new ones (fixes #1051) ... Not having this information causes the blacklist/whitelist logic to trigger and prevents things from functioning as intended when using default settings. Fixes issues for 8muses, deviantart, exhentai, and mangoxo.	4 years ago
Mike Fährmann	f9c1684af7	[newgrounds] restore original video URLs (#1042)	4 years ago
Mike Fährmann	73373c06ec	[weibo] handle posts with more than 9 images (closes #926) ... Responses from '/api/container/getIndex' don't list more than 9 images per 'status' object, but the embedded JSON from a '/detail/<ID>' page does.	4 years ago
Mike Fährmann	dd1e545597	[hentaifoundry] rename GalleryExtractor to PicturesExtractor	4 years ago
Mike Fährmann	c874071f5a	[kissmanga] remove module	4 years ago
Mike Fährmann	93e04bf9a9	[500px] update query hashes	4 years ago
Mike Fährmann	844502cad5	update extractor test results	4 years ago
Mike Fährmann	fad7748b6b	[xvideos] fix 'title' extraction	4 years ago
Mike Fährmann	5b927c15df	[newgrounds] fix video extraction (closes #1042)	4 years ago
Mike Fährmann	bdc6c8f074	improve message for 'oauth:deviantart' etc (closes #989)	4 years ago
Mike Fährmann	430b6d6e2e	[twitter] extend 'retweets' option (closes #1026) ... Setting 'retweets' to '"original"' will use metadata from the original retweeted Tweets, and not from the Retweet entry.	4 years ago
Mike Fährmann	b9bdd2c564	[hentaifoundry] add support for stories (closes #734)	4 years ago
Mike Fährmann	9a9d1924d8	[hentaicafe] add 'manga_id' metadata field (closes #1036) ... This field is only available when using a non-foolslide URL like '/hc.fyi/9874' or '/hazuki-yuuto-summer-blues/'	4 years ago
Mike Fährmann	cc4ac80302	[weasyl] add 'favorite' extractor (#1032)	4 years ago
Mike Fährmann	e9cc719497	[weasyl] update and simplify ... - simplify 'pattern' regexps - parse 'posted_at' as 'date' - use unaltered 'title' ({title!l:R /_/} to lowercase and replace spaces)	4 years ago
Mike Fährmann	6514312126	[nijie] add 'include' option (closes #1018)	4 years ago
Mike Fährmann	0d43456323	[hentaifoundry] add 'include' option	4 years ago
Zanny	ebb7737b9b	Weasyl Extractor (#977) ... * weasyl extractor * @kattjevfel suggested changes * @mikf changes	4 years ago
Mike Fährmann	aeb0d32333	[twitter] improve twitpic extraction (fixes #1019) ... - ignore twitpic.com/photos/… URLs - ignore empty image URLs	4 years ago
Mike Fährmann	7cd383c0f9	update extractor test results	4 years ago
Mike Fährmann	1e313d5b84	implement 'sleep-request' option	4 years ago
Mike Fährmann	c43b3894be	[myhentaigallery] update and fix extraction (#1001) ... - extract more metadata - match "/show/" URLs - complete test results - fix missing images for lines starting with " <img" - fix missing comma in supportedsites.py	4 years ago
choeronline	05b9ac8d37	[myhentaigallery] add extractor (#1001) ... * adds support for myhentaigallery * fixes linting issues in myhentaigallery extractor	4 years ago
Mike Fährmann	2626629117	[danbooru] handle posts without 'id' (fixes #1004)	4 years ago
Mike Fährmann	cc1fb0b4ea	[500px] update query hash	4 years ago
Mike Fährmann	da87a5fb7e	[exhentai] fix accessing config before main constructor ... bug introduced with 055c32e0 Making 'Extractor.config()' quite a bit faster is worth the "cost" of having to set _cfgpath in exhentai constructors, I think.	4 years ago
Mike Fährmann	f5b7ae01c1	update extractor test results	4 years ago
Mike Fährmann	136df52d1f	[deviantart] support watchers-only/paid deviations (#995)	4 years ago
Mike Fährmann	055c32e0f7	precompute extractor config paths	4 years ago
Mike Fährmann	231dd4c800	accumulate postprocessor objects (#994) ... Instead of one 'postprocessors' setting overwriting all others lower in the hierarchy, all postprocessors along the config path will now get collected into one big list. For example '--mtime-from-date' will therefore no longer cause other postprocessor settings in a config file to get ignored.	4 years ago
Mike Fährmann	3108e85b89	[worldthree] remove extractors ... http://www.slide.world-three.org/ hasn't been accessible for a long time.	4 years ago
Mike Fährmann	8fed3eb8cb	[jaiminisbox] remove extractors ... https://jaiminisbox.com/post.html	4 years ago
Mike Fährmann	dcf3ad7eef	[furaffinity] update download URL extraction (fixes #988) ... support the new 'd2.facdn.net' subdomain	4 years ago
Mike Fährmann	3918b69677	remove 'extractor.blacklist' context manager	4 years ago
Mike Fährmann	2b8d57f0ab	[twitter] support '/intent/user?user_id=…' URLs (#980)	4 years ago
Mike Fährmann	a3b473bd2f	[twitter] support specifying users by ID (#980) ... by using 'id:…' as their screen name, i.e. https://www.twitter.com/id:2976459548/media instead of https://twitter.com/supernaturepics/media The user ID can, for example, be obtained from the output of $ gallery-dl -j --range 1 https://twitter.com/<screen-name>	4 years ago
Mike Fährmann	a0d916ed41	[exhentai] update wait time before original image download (#978) ... depend on 'wait-max', don't use a hard-coded value	4 years ago
Mike Fährmann	f6fd449b59	reduce wait time growth rate from exponential to linear ... Waiting for 2**N seconds after each error grows too fast. Simply waiting N seconds seems far more reasonable.	4 years ago
Mike Fährmann	bc48514d84	[aryion] get post ID via gallery-item (fixes #981, closes #982) ... this even works when fetching post IDs from '/latest.php?id='	4 years ago
Mike Fährmann	799ca07fc8	[imgur] update ... - fix image/album detection for galleries - use new API endpoints for image/album data	4 years ago
Mike Fährmann	7876a03ece	[tumblr] create directories for each post (fixes #965) ... This changes the identifiers for directory format string fields. Everything blog related is now inside a 'blog' object and not at the "base level" anymore. E.g. '{name}' for directories is now '{blog[name]}' (or '{blog_name}', since that is also available)	4 years ago
Mike Fährmann	d50f3b333a	update extractor test results	4 years ago
Mike Fährmann	0f55b8e80a	[exhentai] fix type check from dbbbb21 (#940) ... 'bool' is a subclass of 'int', and therefore 'isinstance(self.limits, int)' also returns True when 'self.limits' has a boolean value	4 years ago
Mike Fährmann	e33293fdd8	[hentaihand] update to new site layout	4 years ago
Mike Fährmann	fda9e296dd	[gelbooru] fix extraction without API	4 years ago
Mike Fährmann	69e4871005	update extractor test results ... - sensescans: replace 404d chapters - mangapark: replace 404d chapters - subscribestar: update test for attached files	4 years ago
Mike Fährmann	ab1af66a97	[imgur] add 'search' extractor (#934)	4 years ago