552 Commits (eb94568e1f3ef436d004c4675363e455a048e9ed)

Author	SHA1	Message	Date
Mike Fährmann	1e3dd7330e	merge SharedConfigMixin functionality into Extractor	4 years ago
Mike Fährmann	e5438b8a29	release version 1.15.3	4 years ago
Mike Fährmann	b9bfa4c675	update extractor test results	4 years ago
Mike Fährmann	c3f01dc4e6	implement 'util.unique()'	4 years ago
Mike Fährmann	d83b95fd28	[postprocessor:metadata] accept a string-list for 'content-format' ... (closes #1080)	4 years ago
Mike Fährmann	350b1afe1c	speed up _list_classes() after iterating over all modules once	4 years ago
Mike Fährmann	18213dc5ba	release version 1.15.2	4 years ago
Mike Fährmann	ec61696316	add 't' format string conversion (closes #1065) ... to Trim whitespace from the beginning and end of strings. Example: '{field!t}' becomes 'foo' for 'field' == " \nfoo\t\r"	4 years ago
Mike Fährmann	07432d6262	[seiga] fix flake8 and cookie test (#1063)	4 years ago
Mike Fährmann	b8daabc3ca	[pinterest] implement login support (closes #1055) ... being logged allows access to secret/protected boards	4 years ago
kurumigi	7e0e872f4f	[seiga] Add metadata for single image downloads (#1063) ... * [seiga] Support image metadata. * [seiga] Update test data. * [seiga] Fix cookie check. * [test_cookies] [seiga] Fit test_cookies.py to the last commit.	4 years ago
Mike Fährmann	844793847c	update extractor test results	4 years ago
Mike Fährmann	c874071f5a	[kissmanga] remove module	4 years ago
Mike Fährmann	844502cad5	update extractor test results	4 years ago
Mike Fährmann	7cd383c0f9	update extractor test results	4 years ago
Mike Fährmann	65744a7a31	use alternative for all falsey values in format strings ... … and not just None (#525) It would be better to consistently use None for all non-existent fields and/or fields without a valid value, but this is a good enough workaround for now.	4 years ago
Mike Fährmann	f5b7ae01c1	update extractor test results	4 years ago
Mike Fährmann	392d022b04	implement 'config.accumulate()' (#994)	4 years ago
Mike Fährmann	3108e85b89	[worldthree] remove extractors ... http://www.slide.world-three.org/ hasn't been accessible for a long time.	4 years ago
Mike Fährmann	3918b69677	remove 'extractor.blacklist' context manager	4 years ago
Mike Fährmann	ac3036ef56	add 'filesize-min' and 'filesize-max' options (closes #780)	4 years ago
Mike Fährmann	fd0685d9b5	[postprocessor:zip] defer zip file creation (fixes #968) ... don't try to create zip files on postprocessor construction, wait until directory creation during file download,	4 years ago
Mike Fährmann	d50f3b333a	update extractor test results	4 years ago
Mike Fährmann	e33293fdd8	[hentaihand] update to new site layout	4 years ago
Mike Fährmann	69e4871005	update extractor test results ... - sensescans: replace 404d chapters - mangapark: replace 404d chapters - subscribestar: update test for attached files	4 years ago
Mike Fährmann	688bd046fc	release version 1.14.4	4 years ago
Mike Fährmann	422e69f187	skip external OAuth tests (closes #908)	4 years ago
Mike Fährmann	8dbf827649	[bobx] remove module	4 years ago
Mike Fährmann	87202b8d74	[inkbunny] add 'user' and 'post' extractors (#283)	4 years ago
Mike Fährmann	2ecf1efb16	update extractor test results ... - tumblr: remove deleted post - jaiminisbox: replace removed manga/chapters - smugmug: one inconsequential field got removed	4 years ago
Mike Fährmann	e62ebb4643	update CHANGELOG before building sdist and wheel packages	4 years ago
Mike Fährmann	0cac14c3bd	update extractor test results	4 years ago
Mike Fährmann	53cc498d9c	improve config lookup when there are multiple possible locations ... This specifically applies to all Mastodon extractors and all extractors with a 'basecategory', i.e. 'booru', 'foolslide', etc. Values inside those general config locations wouldn't be recognized when a value with the same was set on the 'extractor' level. For example 'extractor.mastodon.directory' should be used over 'extractor.directory' when both are set, but this was impossible with the previous implementation. (fixes #843)	4 years ago
Mike Fährmann	d81a8e6544	[twitter] update tests	4 years ago
Mike Fährmann	37d71f6e09	strip microseconds in text.parse_datetime()	4 years ago
Mike Fährmann	6db7ed90cb	release version 1.14.1	4 years ago
Mike Fährmann	087e3184dc	use a non-twitter URL when testing snap creation	4 years ago
Mike Fährmann	7daef6ee70	update extractor test results ... - certain posts on Instagram now return https://static.cdninstagram.com/rsrc.php/null.jpg for public users - MangaDex is deploying its new MangaDex@Home network similar to exhentai's Hentai@Home - realbooru has a new site layout, but the underlying booru API still works like before	4 years ago
Mike Fährmann	3bad1579ee	update extractor test results	4 years ago
Mike Fährmann	45baa13615	update extractor test results ... - don't run Instagram tests on Travis anymore - replace Twitter test because timeline was made private - update Hiperdex domain to '.com' (again ...)	4 years ago
Mike Fährmann	dfcf2a2c91	write OAuth token to cache by default (#616)	4 years ago
Mike Fährmann	6294e2c540	add 'text.ensure_http_scheme()'	4 years ago
Mike Fährmann	ece73b5b2a	make 'path' and 'keywords' available in logging messages ... Wrap all loggers used by job, extractor, downloader, and postprocessor objects into a (custom) LoggerAdapter that provides access to the underlying job, extractor, pathfmt, and kwdict objects and their properties. __init__() signatures for all downloader and postprocessor classes have been changed to take the current Job object as their first argument, instead of the current extractor or pathfmt. (#574, #575)	4 years ago
Mike Fährmann	4b606b68e4	skip OAuth tests when server is unreachable	4 years ago
Mike Fährmann	8b60bd6a91	mock 'time()' in cache tests ... instead of calling 'sleep()' to let time advance. This shortens the time needed to run those tests, and ensures consistent results. (Tests would randomly fail when using 'sleep()')	4 years ago
Mike Fährmann	8f2c1da041	skip example config tests if files are not available (#730)	4 years ago
Mike Fährmann	5df8f2959b	insert local directory into PYTHONPATH when running tests	4 years ago
Mike Fährmann	ff47641b13	test whether default/example config files contain valid JSON	4 years ago
Mike Fährmann	d6facdee7b	[mastodon] add tests (#701)	4 years ago
Mike Fährmann	fd438f0d78	update extractor test results	5 years ago
Mike Fährmann	a0f4c295c0	add optional 'utcoffset' argument to 'parse_datetime()'	5 years ago
Mike Fährmann	406449b0d6	ensure keys for mastodon instances are available during tests ... Calls to config.clear() from other tests are removing the API credentials set when importing mastodon.py for the first time.	5 years ago
Mike Fährmann	9e7dfc0cfc	[myportfolio] fix extraction of galleries without title	5 years ago
Mike Fährmann	3b50c4f49d	add tests for "Extractors" in oauth.py (#670)	5 years ago
Mike Fährmann	04bd0472de	add tests for Extractor.wait()	5 years ago
Mike Fährmann	7499d71d02	[simplyhentai] ignore certificate errors in video test	5 years ago
Mike Fährmann	4203dc0bdc	[mangapark] fix metadata extraction	5 years ago
Mike Fährmann	77fda8190c	[35photo] simplify/remove tests for the 'genre' extractor ... There is still a nice genre overview page (https://35photo.pro/genre/) but the individual sub-pages don't list photos anymore	5 years ago
Mike Fährmann	32e36d8f02	[sexcom] replace tests	5 years ago
Mike Fährmann	e0b0e8d62a	release version 1.13.2	5 years ago
Mike Fährmann	a63a376ad2	[mangoxo] fix login	5 years ago
Mike Fährmann	ebc70e87ce	[e621] update to new interface / API endpoints (closes #635)	5 years ago
Mike Fährmann	72122eb9b3	release version 1.13.1	5 years ago
Mike Fährmann	ce5e2a58fe	[imgbb] update test results ... Image server domain changed from https://image.ibb.co/ to https://i.ibb.co/	5 years ago
Mike Fährmann	1d4a369ea2	update extractor test results	5 years ago
Mike Fährmann	ec85bf90de	use context managers in cache.py & add tests	5 years ago
Mike Fährmann	4e361b3008	add tests for specific datetime values	5 years ago
Mike Fährmann	90e4c645ba	[formatter] allow multiple "special" format specifiers (#595) ... It is now, for example, possible to specify multiple replacement operations per format replacement field: {name:Ra/b/Rc/d/}	5 years ago
Mike Fährmann	219c4cc78c	[formatter] allow for numeric list and string indices	5 years ago
Mike Fährmann	7d1da614d9	[formatter] implement field name alternatives (#525) ... The format string '{a|b|c}' will now try to use the value from 'a' and fall back to 'b' and 'c' if accessing a field raises an exception or if its value is None.	5 years ago
Mike Fährmann	c7cf9dd111	[furaffinity] support classic layout (#284)	5 years ago
Mike Fährmann	40fe062851	[pixiv] fix user id for bookmarks API calls (closes #596)	5 years ago
Mike Fährmann	2852691d78	[paheal] replace test URL ... searching for 'k-on' doesn't yield any results anymore	5 years ago
Mike Fährmann	2a9be48511	improve util.load/save_cookiestxt() and add tests ... - take a file object as argument instead of an filename - accept whitespace before comments (" # comment") - map expiration "0" to None and not the number 0	5 years ago
Mike Fährmann	b3b5754f2d	update test_cookies.py	5 years ago
Mike Fährmann	174117f827	allow multiple hashes for content tests	5 years ago
Mike Fährmann	60a43f0264	fix downloader tests	5 years ago
Mike Fährmann	e89413da22	update test results	5 years ago
Mike Fährmann	5cac79c3d9	[erolord] remove extractor	5 years ago
Mike Fährmann	988cc2ec23	[mangadex] change domain to mangadex.cc (closes #559)	5 years ago
Mike Fährmann	87c8b89ddd	[postprocessor:metadata] add 'directory' option (#520)	5 years ago
Mike Fährmann	82f7f4172a	update test results	5 years ago
Mike Fährmann	d0920e84e9	update test results	5 years ago
Mike Fährmann	9e63804347	[patreon] make retrieving user info nonfatal (#508) ... … and fall back to the included data if an error occurs.	5 years ago
Mike Fährmann	15f9bb3d14	add option to disable pyOpenSSL usage (#508) ... (pyOpenSSL is now disabled by default)	5 years ago
Mike Fährmann	50deab5265	[deviantart] fix URL generation from /extended_fetch results ... (closes #505)	5 years ago
Mike Fährmann	004812258d	[hentaifox] fix extraction	5 years ago
Mike Fährmann	a412531451	[postprocessor:metadata] implement 'extension-format' option ... closes #477	5 years ago
Mike Fährmann	b5c964332b	improve config.py test coverage	5 years ago
Mike Fährmann	f5604492c3	update interface of config functions	5 years ago
Mike Fährmann	3fc1e12949	[postprocessor:metadata] filter private entries ... i.e. keys starting with an underscore	5 years ago
Mike Fährmann	978cb03f81	update misc test results ... - Livedoor now uses https:// for its image URLs - Instagram image URLs got simplified	5 years ago
Mike Fährmann	bbbeff4c41	[downloader.http] implement file-specific HTTP headers	5 years ago
Mike Fährmann	3ece3976ae	[newgrounds] implement login support (#394)	5 years ago
Mike Fährmann	abfcb356fc	[flickr] support 3k, 4k, 5k, and 6k photo sizes (closes #472)	5 years ago
Mike Fährmann	da6789b2b0	disable unique archive id checks for some tests ... - same image twice in a livedoor blog post - unreliable results for related pinterest items	5 years ago
Mike Fährmann	ba083b30b2	fix snap build ... … hopefully	5 years ago
Mike Fährmann	94a94f3b86	miscellaneous stuff	5 years ago
Mike Fährmann	9e88e7a344	[postprocessor:exec] improve (#421, #413) ... - add 'final' option - include job status in pp finalization - improve and extend documentation	5 years ago
Mike Fährmann	2a3bd4e3c7	rename extractor classes starting with a digit	5 years ago
Mike Fährmann	64786363be	[4chan] simplify ... - remove 'chan.py' - slight adjustments to directory and filenames	5 years ago
Mike Fährmann	557e2c018b	[8chan] remove module	5 years ago
Mike Fährmann	322c2e7ed4	renaming variables ... mostly 'keyword(s)' to 'kwdict'	5 years ago
Mike Fährmann	87a87bff7e	[simplyhentai] fix image URLs	5 years ago
Mike Fährmann	d5e3910270	adjust 'util.raises()'	5 years ago
Mike Fährmann	b23c822b23	[luscious] use GraphQL	5 years ago
Mike Fährmann	1693d97bd3	update extractor class hierarchies ... - let the GalleryExtractor class inherit directly from Extractor - make ChapterExtractor a subclass of GalleryExtractor - change enumeration field names of GalleryExtractors to 'num'	5 years ago
Mike Fährmann	7ebd984e8d	[imgur] print error message if no JSON data is found (#446)	5 years ago
Mike Fährmann	de4e2029d1	[nsfwalbum] update test album ... the old one is no longer available	5 years ago
Mike Fährmann	913460240d	[reddit] fix 'extractor.blacklist()' arguments ... The second argument must support 'append()'.	5 years ago
Mike Fährmann	1848788970	update test results etc	5 years ago
Mike Fährmann	d5fbb2d9de	[tumblr] ignore audio links from Spotify etc.	5 years ago
Mike Fährmann	c6c5cb1898	improve 'deviantart.quality' description	5 years ago
Mike Fährmann	c9b97dbca3	extend post processor tests	5 years ago
Mike Fährmann	49f6d7176d	[deviantart] restore filenames (#392) ... <title>_by_<user>_<id> --> <title>_by_<user>-<id>	5 years ago
Mike Fährmann	e528f3cb77	adjust postprocessor test results ... see 2495b99	5 years ago
Mike Fährmann	cb7f149974	fix mtime datetime test ... datetime.timestamp() uses local time for a naive datetime object	5 years ago
Mike Fährmann	23251356cb	require 'extension' data for each URL (#382)	5 years ago
Mike Fährmann	dd72ae7164	add postprocessor tests	5 years ago
Mike Fährmann	0bb873757a	update PathFormat class ... - change 'has_extension' from a simple flag/bool to a field that contains the original filename extension - rename 'keywords' to 'kwdict' and some other stuff as well - inline 'adjust_path()' - put enumeration index before filename extension (#306)	5 years ago
Mike Fährmann	748e37554c	update .travis.yml ... - install pyOpenSSL before running tests - simplify snap tests	5 years ago
Mike Fährmann	b7fb93e2b2	[downloader:http] add 'adjust-extensions' option	5 years ago
Mike Fährmann	eb7da159e2	[imagebam] update URL test results ... Image URLs are now using https://, but the website itself is still served as http://.	5 years ago
Mike Fährmann	fa60109e97	[exhentai] don't use e-hentai.org for exhentai URLs	5 years ago
Mike Fährmann	4a0c98bfc9	miscellaneous fixes and adjustments	5 years ago
Mike Fährmann	40637556fa	[ngomik] fix extraction	5 years ago
Mike Fährmann	d9d44ad953	[tsumino] update test results	5 years ago
Mike Fährmann	b1bea8aaeb	add 'restrict-filenames' option (#348)	5 years ago
Mike Fährmann	b3851e01d9	release version 1.9.0	5 years ago
Mike Fährmann	12da6bd0c9	[simplyhentai] fix/improve extraction	5 years ago
Mike Fährmann	b89f0d8d3c	update extractor result tests	5 years ago
Mike Fährmann	40da44b17f	Merge branch 'v1.9.0'	5 years ago
Mike Fährmann	7a99e85943	[kissmanga] fix download URLs and file extensions ... The current Blogspot image URLs hosted on Kissmanga end with an "invalid" query parameter (/000.png&upx=...), which doesn't get recognized by 'spliturl()' and 'parseurl()' as such and gets therefore included in the 'extension' field from 'text.nameext_from_url()'.	5 years ago
Mike Fährmann	a9c89085fb	[instagram] implement login support (#195)	5 years ago
Mike Fährmann	b1985d6579	test default format strings during extractor result tests ... A missing value or an invalid "syntax" for a format replacement field will raise an exception.	5 years ago
Mike Fährmann	95b1e4c3c0	implement R<old>/<new>/ format option (#318)	5 years ago
Mike Fährmann	70713f0f28	fix extractor result tests	5 years ago
Mike Fährmann	ee4d7c3d89	update downloader.find() and related code ... Instead of replacing 'https' with 'http' for every URL in 'get_downloader()', this now only happens once during downloader initialization. Also unit tests.	5 years ago
Mike Fährmann	179d112083	[downloader] overhaul http and text modules ... Get rid of the modular structure and simplify/specialize those modules.	5 years ago
Mike Fährmann	a77340c647	[keenspot] fix extraction for "TwoKinds"	5 years ago
Mike Fährmann	b171befa87	implement 'parse_unicode_escapes()'	5 years ago
Mike Fährmann	e05a96db5e	[deviantart] rename 'stash' to 'extra' (#302) ... 'stash' is already used as a name for the StashExtractor and therefore expected to be a dictionary.	5 years ago
Mike Fährmann	7c6cb908f9	[xhamster] update test results	5 years ago
Mike Fährmann	62335b9015	[paheal] adjust test results	5 years ago
Mike Fährmann	6a34f4b0c1	skip tests on read timeouts; print list of skipped tests	5 years ago
Mike Fährmann	d33f5a7423	[wallhaven] rewrite ... - use API - remove login support, add 'api-key' option - remove support for "alpha" subdomain - alpha.wallhaven.cc used numeric IDs that can't be translated to the new ID system - support direct links to wallpapers	5 years ago
Mike Fährmann	5499934ae2	[ngomik] fix extraction	5 years ago
Mike Fährmann	2b1999476e	implement 'text.rextract()'	5 years ago
Mike Fährmann	e30ada162d	fix cookie tests ... update _get_extractor(): - always return an Extractor instance with a _login_impl() method - use Extractor.from_url()	5 years ago
Mike Fährmann	2316e0ed3d	fix strptime workaround from b0e85a4 ... Don't return a modified version of 'date_time' if strptime fails.	5 years ago