153 Commits (4b441c162e4515d54a6aca543e548e129dae63dd)

Author	SHA1	Message	Date
Mike Fährmann	8e01cf0ef8	[reactor] generalize extractors (#148) ... - support *.reactor.cc domains - combine joyreactor and pornreactor modules	6 years ago
Leonardo Taccari	2655a2ea02	Add support for instagram.com user profiles and pages (#134) ... * [instagram] Add extractor for instagram.com user profiles and pages The extractor scrapes `instagram.com/<user>' timelines and `instagram.com/p/<shortcode>' by mimicking the behaviour of a web browser and extracting the sharedData JSON of the single pages. Please note that this mean that for user timelines we also do an extra request to the `instagram.com/p/<shortcode>' page but this permit to have consistent (and all) information about the media fetched. The MD5 logic used for X-Instagram-GIS was documented in <https://stackoverflow.com/questions/49786980/> * [instagram] Test for keywords, not url for GraphImage and GraphSidecar URLs returned by instagram seems not stable so avoid testing for them and instead test for keyword returned. * [instagram] Improve test of InstagramProfilepageExtractor Also check the count of media returned. * [instagram] Several cleanup and improvements - Change description, subcategories to generate a better description in docs/supportedsite.rst - Remove not needed InstagramExtractor.__init__() - Use text.parse_int() instead of directly using int() (the former is more robust) - Use self.request().json() instead of using json.loads() the self.request().text() - Add `pattern:' to check the URLs where we do not have a stable URLs. It seems that only the subdomain is not stable. Thanks to @mikf!	6 years ago
Mike Fährmann	ba56827f36	[newgrounds] add user-, video-, image-extractors (#119)	6 years ago
Mike Fährmann	a0ae156edc	[pornreactor] add tag-, user-, post-extractors (#114)	6 years ago
Mike Fährmann	503d42a1c2	[joyreactor] add tag-, user-, post-extractors (#114)	6 years ago
Mike Fährmann	2cf3f53839	[yuki] add thread extractor (closes #111)	6 years ago
Mike Fährmann	c9290d8212	[wallhaven] add wallpaper and search extractors ... todo: - login support to gain access to NSFW wallpapers - extractors for tag-, similar-, latest-listings - skip() support	6 years ago
Mike Fährmann	69fd61ea86	[bobx] add gallery and model extractors	6 years ago
Mike Fährmann	3ecea4cf36	[hentaicafe] add chapter and manga extractors (#101)	6 years ago
Mike Fährmann	ab2bfaeb46	[ngomik] add replacement for 'subapics' ... http://subapics.com/ got discontinued and replaced by http://ngomik.in/. ngomik.in is still displaying a link to the "old site" showing a big "Account Suspended" sign.	6 years ago
Mike Fährmann	3c90df6635	[piczel] add user, folder and image extractors	6 years ago
Mike Fährmann	df082e923c	[behance] add gallery extractor (#95)	6 years ago
Mike Fährmann	84854fcad7	[myportfolio] add user and gallery extractors (#95)	6 years ago
Mike Fährmann	a6fe2bb594	[whatisthisimnotgoodwithcomputers] remove extractor	6 years ago
Mike Fährmann	55b0913412	[simplyhentai] add gallery extractor (#89)	6 years ago
Mike Fährmann	16e014baaa	[smugmug] added image and album extractor ... just some initial code that still requires a lot of work ... TODO: - folders - old-style albums (which are nearly all of them ...) - images from users - OAuth It could also happen that the API credentials used will become invalid whenever my 14 day trial period ends (7 days remaining), but that would just require users to supply their own.	6 years ago
Mike Fährmann	d96b3474e5	[puremashiro] remove module ... site has been unreachable for a couple of weeks and now the DNS record is gone as well	6 years ago
Mike Fährmann	b44a296404	[gomanga] remove module ... site has been unreachable for a couple of weeks and the cloudflare status page shows host errors	6 years ago
Mike Fährmann	dd49127408	[spectrumnexus] remove module ... Site stopped hosting manga scans (http://view.thespectrum.net/)	7 years ago
Mike Fährmann	48a83a89e9	[loveisover] remove module ... archive.loveisover.me was shut down on 2018-03-29; https://www.archiveteam.org/index.php?title=4chan#archive.loveisover.me	7 years ago
Mike Fährmann	dd314279fb	[test] add unit tests for extractor module functions	7 years ago
Mike Fährmann	723cc66bb1	[artstation] add user-, image- and likes-extractors	7 years ago
Mike Fährmann	d6ef52897c	[imgchili] remove module ... All previously hosted images yield a 404 and the main page is just a logo.	7 years ago
Mike Fährmann	749fbbfa6c	[mangadex] add chapter- and manga-extractor	7 years ago
Mike Fährmann	edfd3d9fc9	[yeet] remove module ... - archive.yeet.net returns a 500 server error - yeet.net moved to yeet.rip, but the archive is gone	7 years ago
Mike Fährmann	54c36a8a34	[subapics] add chapter- and manga-extractor (#70)	7 years ago
Mike Fährmann	2dd3aeeeae	[komikcast] add chapter- and manga-extractor (#70)	7 years ago
Mike Fährmann	6a07e38366	implement extractor.add() and .add_module() ... ... as a public and non-hacky way to add (external) extractors to gallery-dl's pool and make them available for extractor.find()	7 years ago
Mike Fährmann	1e93955170	[batoto] remove module ... Site officially shut down on 2018.01.18	7 years ago
Mike Fährmann	91ed147cef	[oauth] use custom key/secret values during oauth:…	7 years ago
Mike Fährmann	40d35c87bc	[paheal] add tag- and post-extractors (closes #69)	7 years ago
Mike Fährmann	f10ffc0839	update extractor blacklist to also allow classes	7 years ago
Mike Fährmann	b33efc99a4	[idolcomplex] add support for idol.sankakucomplex.com	7 years ago
Mike Fährmann	5b094328b5	[puremashiro] add chapter- and manga-extractor (closes #66) ... Also adds support for region subtags in language codes (e.g. en-us)	7 years ago
Leonardo Taccari	a8d2dde8b2	[slideshare] Add a new extractor for slideshare.net (#54)	7 years ago
Mike Fährmann	300346ecdf	[mangazuki] remove extractors ... This site has been in "rebuild"-mode for a fairly long time and the current extractor code isn't going to work for the new version either.	7 years ago
Mike Fährmann	a393e6e538	[xvideos] add gallery extractor (#45)	7 years ago
Mike Fährmann	4d8387f93b	[pixiv] support mobile URLs (https://touch.pixiv.net/)	7 years ago
Mike Fährmann	cebf800a7f	[foolfuuka] add support for more sites (#18) ... - https://arch.b4k.co - https://archive.whatisthisimnotgoodwithcomputers.com - https://archive.yeet.net Notes: - The name "whatisthisimnotgoodwithcomputers" is way too long ... - archive.yeet.net is out of date and also blocked by 4chan servers - newest threads are 2 weeks old - using "https://archive.yeet.net" as Referer header results in "403 Forbidden" when accessing 4chan	7 years ago
Mike Fährmann	4ea82ea556	[warosu] add thread extractor	7 years ago
Mike Fährmann	abd7c559cd	[yonkouprod] remove module ... Every manga chapter on this site has been removed.	7 years ago
Mike Fährmann	da7219ba74	[kisscomic] remove module ... Image links on this site are dead.	7 years ago
Mike Fährmann	f53bf1a323	[thebarchive] add thread extractor	7 years ago
Mike Fährmann	b8cf434bb0	[rebeccablacktech] add thread extractor	7 years ago
Mike Fährmann	1cb1d2e0a3	[mangazuki] add chapter extractor	7 years ago
Mike Fährmann	f105782435	[fireden] add thread extractor	7 years ago
Mike Fährmann	c93f7d7496	[archiveofsins] add thread extractor	7 years ago
Mike Fährmann	96e13604da	[archivedmoe] add thread extractor	7 years ago
Mike Fährmann	98464d1f1b	[loveisover] add thread extractor	7 years ago
Mike Fährmann	47692f28da	[2chan] add thread extractor	7 years ago