594 Commits (6f30cf4c64af49d0023643bb89942b3eb133b849)

Author	SHA1	Message	Date
Mike Fährmann	6f30cf4c64	change keyword names to valid Python identifiers ... This commit mostly replaces all minus-signs ('-') in keyword names with underscores ('_') to allow them to be used in filter-expressions. For example 'gallery-id' got renamed to 'gallery_id'. (It is theoretically possible to access any variable, regardless of its name, with 'locals()["NAME"]', but that seems a bit too convoluted if just 'NAME' could be enough)	7 years ago
Mike Fährmann	54c0715135	allow users to set their own API access_tokens/client_ids	7 years ago
Mike Fährmann	49c7e70c10	[acidimg] add image extractor	7 years ago
Mike Fährmann	9b21d3f13c	add '--filter' command-line option ... This allows for image filtering via Python expressions by the same metadata that is also used to build filenames (--list-keywords). The usually shunned eval() function is used to evaluate filter-expressions, but it seemed quite appropriate in this case and shouldn't introduce any new security issues, as any attacker that could do > gallery-dl --filter "delete-everything()" ... could as well do > python -c "delete-everything()"	7 years ago
Mike Fährmann	00420ff202	[booru] consistent order for "popular" results	7 years ago
Mike Fährmann	83cf1e1d6d	[sankaku] unescape image URLs	7 years ago
Mike Fährmann	f98e3e8002	[luscious] fix tag extraction	7 years ago
Mike Fährmann	65997d835b	replace popular/ranking tests with older ones ... Metadata of several year old lists shouldn't change as much as it would for newer ones, which makes metadata-comparisons of the output of build_testresult_db.oy easier.	7 years ago
Mike Fährmann	be30fb2f98	add common config category for boorus and foolslide	7 years ago
Mike Fährmann	c0755a4d5e	[exhentai] revert login-method to its old version (#37) ... Additional cookies don't seem to help and have to be manually set anyway. The older method is more likely to succeed, so I'd rather use this one.	7 years ago
Mike Fährmann	3ee39ffd93	[exhentai] update login procedure (#37) ... This new version behaves pretty much exactly like a browser would and caches all cookies sent to it and not just "ipb_member_id" and "ipb_pass_hash".	7 years ago
Mike Fährmann	88a386977e	[booru] add "popular" extractors for more sites ... - konachan.com - behoimi.org - e621.net	7 years ago
Mike Fährmann	07214f4007	[booru] place subcategories into base classes	7 years ago
Mike Fährmann	60a888a1e4	[foolfuuka] add common config category ... All FoolFuuka based 4chan-archive extractors can now be configured using their own config keys (extractor.<category>) as well as a common shared one (extractor.foolfuuka).	7 years ago
Mike Fährmann	47bcf53ec1	implement support for additional unit test result types ... - "pattern" matches all resulting URLs against the given regex - "count" allows to specify the amount of returned URLs	7 years ago
Mike Fährmann	2d0dfe9d56	[exhenai] init headers before login and detect sadpanda ... - also debug-logs html after failed login - #37	7 years ago
Mike Fährmann	c7ec103e15	[batoto] fix extraction of chapter URLs	7 years ago
Mike Fährmann	18e6ed1c7e	[booru] add extractors for "Popular" images	7 years ago
Mike Fährmann	f7cdfd4c25	add a simplified version of 'parse_qs' ... This version only returns a dict of plain string to string key-value pairs and ignores multiple values for the same query variable.	7 years ago
Mike Fährmann	3b21e0703c	[deviantart] allow distinction between users and groups (#26) ... This is done by prepending "group-" to an extractor's subcategory if the URL belongs to a group ("folder" becomes "group-folder" and so on). This changes the configuration-path being used and is also reflected in the output of '--list-keywords'.	7 years ago
Mike Fährmann	e61a3a56d1	[hentai2read] fix and update keywords ... Added the "author" keyword and changed the name of a few others to be consistent with other manga/chapter extractors.	7 years ago
Mike Fährmann	c45770331a	use 'str.partition()' ... The (r)partition method is always faster then split() or any other method that has been replaced in this commit.	7 years ago
Mike Fährmann	017a72f448	[pixiv] improve input validation	7 years ago
Mike Fährmann	dcf42c5e89	[pixiv] add extractor for ranking lists	7 years ago
Mike Fährmann	4ea82ea556	[warosu] add thread extractor	7 years ago
Mike Fährmann	9aa95fba8c	[deviantart] adapt download URLs to use https ... Even though DeviantArt is "completely switching over to HTTPS"[1], every URL contained in an API response is still using HTTP [1] https://danlev.deviantart.com/journal/DeviantArt-Is-Switching-To-HTTPS-697996906	7 years ago
Mike Fährmann	02e89700fc	[foolfuuka] ensure sorted posts	7 years ago
Mike Fährmann	8bcf88bff7	[flickr] fix extraction ... This issue was only noticeable with older Python versions, as these don't exhibit a consistent ordering of dict keys.	7 years ago
Mike Fährmann	004456d5d5	properly update the config-dictionary ... When using 2 or more config files, the values of the second would improperly overwrite nested dictionaries of the first one. The new method properly combines these nested dictionaries as well.	7 years ago
Mike Fährmann	cfa479fab5	update error message for unspecified exceptions ... - ask user to report unexpected errors, which usually indicate extractor failure - handle OSErrors separately (permissions, disk full, etc) - revert 30eef52	7 years ago
Mike Fährmann	7e936e9c06	[luscious] simplify and remove dead code	7 years ago
Mike Fährmann	0245a0ba5f	fix extraction and update test results ... - fixes for hbrowse, imgyt, imgcandy, hosturimage - test updates for deviantart, gfycat	7 years ago
Mike Fährmann	abd7c559cd	[yonkouprod] remove module ... Every manga chapter on this site has been removed.	7 years ago
Mike Fährmann	da7219ba74	[kisscomic] remove module ... Image links on this site are dead.	7 years ago
Mike Fährmann	852e7acd31	[twitter] ignore "Promoted Tweets"	7 years ago
Mike Fährmann	915a0137de	improve 'extractor.request' ... - add 'fatal' argument - improve internal logic and flow - raise known exception on error - update exception hierarchy	7 years ago
rachmadani haryono	dcd573806e	chg: dev: fix error (#32) ... * fix: dev: error * fix: dev: AttributeError when getting artist * fix: dev: typo on luscious parser	7 years ago
Mike Fährmann	c4713404c8	[directlink] improve URL pattern	7 years ago
Mike Fährmann	d443822fdb	[luacious] get correct image URLs (fixes #33) ... Instead of using thumbnail URLs and modifying them the extractor now goes through every single image-page and gets its download URL from there.	7 years ago
Mike Fährmann	6950708e52	[hentaicdn] use HTTPS	7 years ago
Mike Fährmann	4f1e6c109f	[deviantart] remove 'invalid escape sequence' warning ... - use r"\w" or "\\w" instead of "\w"	7 years ago
Mike Fährmann	c864be479e	[directlink] update URL pattern & PEP 8 ... - combine some file extensions - don't match '.je' - line length < 80	7 years ago
H R X N	45f9d64c23	Update directlink.py with additional file exts. (#30) ... Add WebP, still not that common, but it's increasing. Add 3rd JPEG variant (https://en.wikipedia.org/wiki/JPEG#JPEG_filename_extensions) Never seen JFIF in the wild, would probably be overkill. Extend Ogg formats (https://en.wikipedia.org/wiki/Ogg; https://wiki.xiph.org/MIME_Types_and_File_Extensions)	7 years ago
Mike Fährmann	4357966a70	[kissmanga] make URL pattern case-insensitive (fixes 28)	7 years ago
Mike Fährmann	7aa9fa796a	code cleanup and fixes	7 years ago
Mike Fährmann	f08af03845	Merge branch 'cookies'	7 years ago
Mike Fährmann	55f048d02b	ignore case of cookiejar magic strings	7 years ago
Mike Fährmann	f53bf1a323	[thebarchive] add thread extractor	7 years ago
Mike Fährmann	b8cf434bb0	[rebeccablacktech] add thread extractor	7 years ago
Mike Fährmann	808f67ba7d	use 'cookiedomain' for cookies set by object-config-values ... otherwise these cookies would not be picked up by the _check_cookies() method.	7 years ago