544 Commits (0e5d2e128b02ecb88f7723e272f77814f8cafaad)

Author	SHA1	Message	Date
Mike Fährmann	9377543162	[mastodon] add 'following' extractor (#1891)	3 years ago
Mike Fährmann	bd845303ad	implement a way to shorten filenames with east-asian characters ... (#1377) Setting 'output.shorten' to "eaw" (East-Asian Width) uses a slower algorithm that also considers characters with a width > 1.	3 years ago
Mike Fährmann	292fffc83c	add 'j' format string conversion ... to convert to a JSON formatted string	3 years ago
Mike Fährmann	bb6a130942	automatically set required DDoS-GUARD cookies (#1779) ... for kemono.party and seiso.party	3 years ago
Mike Fährmann	2792ed6e4b	implement 'util.format_value()'	3 years ago
Mike Fährmann	9e42cd58ea	replace ChainPredicate class with 'functools.partial'	3 years ago
Mike Fährmann	36ac2197db	[ytdl] add extractor for sites supported by youtube-dl ... (#1680, #878) Can be used by prefixing any URL with 'ytdl:', or by setting 'extractor,ytdl.enabled' to 'true'.	3 years ago
Mike Fährmann	64240c8d42	[imagevenue] fix extraction ... (closes #1677)	3 years ago
Mike Fährmann	0179581340	add 'T' format string conversion (#1646) ... to convert 'date'/datetime to timestamp	3 years ago
Mike Fährmann	f74cf52e2b	[seisoparty] add 'user' and 'post' extractors (#1635)	3 years ago
Mike Fährmann	759735fb02	[kemonoparty] fix 'username' extraction (fixes #1652) ... The site's <title> content changed from <title>NAME | Kemono</title> to <title> NAME | Kemono </title>	3 years ago
Mike Fährmann	07c8adbd8b	[mangadex] implement login with username & password (#1535)	3 years ago
Mike Fährmann	4a747a31a3	[postprocessor:metadata] handle dicts in mode;tags (fixes #1598)	3 years ago
Mike Fährmann	3cbbefd4ed	support 'filter' option for post processors (#1460)	3 years ago
Mike Fährmann	0abad8bc12	implement 'compile_expression()'	3 years ago
Mike Fährmann	da6806a161	fix job tests for Python 3.4 and 3.5 ... assert_called() and assert_not_called() got added in Python 3.6	3 years ago
Mike Fährmann	8fd8126117	fix ISO 639-1 code for Japanese ... "jp" -> "ja"	3 years ago
Mike Fährmann	af9dba4684	add DataJob tests	3 years ago
Mike Fährmann	adf4d661b3	use '_extractor' info in UrlJobs	3 years ago
Mike Fährmann	1eabfa5c7a	[pillowfort] implement login with username & password (#846)	3 years ago
Mike Fährmann	559462789d	add some tests for job.py	3 years ago
Mike Fährmann	c5ca7905ce	add 'noop()' and 'identity()' functions	3 years ago
Mike Fährmann	bc868e7bb8	consider apparently long extensions as part of the filename ... (#1516)	3 years ago
Mike Fährmann	bdfcc9c4b1	update extractor test results	3 years ago
Mike Fährmann	387fe415d5	unescape items in text.split_html()	4 years ago
Mike Fährmann	78fd63b8f0	remove 'text.clean_xml()' ... was not used anywhere	4 years ago
Mike Fährmann	8553b218d9	replace calls to 'os.path.splitext()' with 'str.rpartition()' ... Makes functions who used it more than twice as fast and we can get rid of an import as well.	4 years ago
Mike Fährmann	bff71cde80	implement 'util.unique_squence()'	4 years ago
Mike Fährmann	5f1a6ff6fa	remove unneeded 'TRAVIS_SKIP' from test_results.py	4 years ago
Mike Fährmann	8821dceb79	use __import__() to dynamically load modules	4 years ago
Mike Fährmann	36bf76fa44	update 'oauth:mastodon:<instance>' code	4 years ago
Mike Fährmann	91308140ec	make 'generate_token()' compatible with Python 3.4	4 years ago
Mike Fährmann	780b6adb91	rename 'generate_csrf_token()' to just 'generate_token()' ... and add a 'size' argument	4 years ago
Mike Fährmann	0fdaea00a3	[postprocessor:metadata] sanitize filenames	4 years ago
Mike Fährmann	aac00a2024	add 'd' conversion for format strings ... to convert a timestamp to a formattable 'datetime' object. For example '{created_at!d:%Y-%m-%d}' transforms the timestamp in 'created_at' into a 'datetime' object and then formats its content using '%Y-%m-%d' as template. 1262304000 -> datetime(2010, 1, 1) -> "2010-01-01"	4 years ago
Mike Fährmann	912eea29bc	update extractor test results	4 years ago
Mike Fährmann	1f9121fecb	release version 1.16.0	4 years ago
Mike Fährmann	b2c55f0a72	[sankaku] remove login support ... The old login method for 'https://chan.sankakucomplex.com/user/login' and the cookies it produces have no effect on the results from 'beta.sankakucomplex.com'.	4 years ago
Mike Fährmann	547107307e	fix 'Metadata' messages in result tests	4 years ago
Mike Fährmann	578dcf805c	[mangapanda] don't force https://	4 years ago
Mike Fährmann	ca59bd691c	[postprocessor:metadata] add 'event' and 'filename' options	4 years ago
Mike Fährmann	9fffa9c343	rework post processor callbacks	4 years ago
Mike Fährmann	1e3dd7330e	merge SharedConfigMixin functionality into Extractor	4 years ago
Mike Fährmann	e5438b8a29	release version 1.15.3	4 years ago
Mike Fährmann	b9bfa4c675	update extractor test results	4 years ago
Mike Fährmann	c3f01dc4e6	implement 'util.unique()'	4 years ago
Mike Fährmann	d83b95fd28	[postprocessor:metadata] accept a string-list for 'content-format' ... (closes #1080)	4 years ago
Mike Fährmann	350b1afe1c	speed up _list_classes() after iterating over all modules once	4 years ago
Mike Fährmann	18213dc5ba	release version 1.15.2	4 years ago
Mike Fährmann	ec61696316	add 't' format string conversion (closes #1065) ... to Trim whitespace from the beginning and end of strings. Example: '{field!t}' becomes 'foo' for 'field' == " \nfoo\t\r"	4 years ago
Mike Fährmann	07432d6262	[seiga] fix flake8 and cookie test (#1063)	4 years ago
Mike Fährmann	b8daabc3ca	[pinterest] implement login support (closes #1055) ... being logged allows access to secret/protected boards	4 years ago
kurumigi	7e0e872f4f	[seiga] Add metadata for single image downloads (#1063) ... * [seiga] Support image metadata. * [seiga] Update test data. * [seiga] Fix cookie check. * [test_cookies] [seiga] Fit test_cookies.py to the last commit.	4 years ago
Mike Fährmann	844793847c	update extractor test results	4 years ago
Mike Fährmann	c874071f5a	[kissmanga] remove module	4 years ago
Mike Fährmann	844502cad5	update extractor test results	4 years ago
Mike Fährmann	7cd383c0f9	update extractor test results	4 years ago
Mike Fährmann	65744a7a31	use alternative for all falsey values in format strings ... … and not just None (#525) It would be better to consistently use None for all non-existent fields and/or fields without a valid value, but this is a good enough workaround for now.	4 years ago
Mike Fährmann	f5b7ae01c1	update extractor test results	4 years ago
Mike Fährmann	392d022b04	implement 'config.accumulate()' (#994)	4 years ago
Mike Fährmann	3108e85b89	[worldthree] remove extractors ... http://www.slide.world-three.org/ hasn't been accessible for a long time.	4 years ago
Mike Fährmann	3918b69677	remove 'extractor.blacklist' context manager	4 years ago
Mike Fährmann	ac3036ef56	add 'filesize-min' and 'filesize-max' options (closes #780)	4 years ago
Mike Fährmann	fd0685d9b5	[postprocessor:zip] defer zip file creation (fixes #968) ... don't try to create zip files on postprocessor construction, wait until directory creation during file download,	4 years ago
Mike Fährmann	d50f3b333a	update extractor test results	4 years ago
Mike Fährmann	e33293fdd8	[hentaihand] update to new site layout	4 years ago
Mike Fährmann	69e4871005	update extractor test results ... - sensescans: replace 404d chapters - mangapark: replace 404d chapters - subscribestar: update test for attached files	4 years ago
Mike Fährmann	688bd046fc	release version 1.14.4	4 years ago
Mike Fährmann	422e69f187	skip external OAuth tests (closes #908)	4 years ago
Mike Fährmann	8dbf827649	[bobx] remove module	4 years ago
Mike Fährmann	87202b8d74	[inkbunny] add 'user' and 'post' extractors (#283)	4 years ago
Mike Fährmann	2ecf1efb16	update extractor test results ... - tumblr: remove deleted post - jaiminisbox: replace removed manga/chapters - smugmug: one inconsequential field got removed	4 years ago
Mike Fährmann	e62ebb4643	update CHANGELOG before building sdist and wheel packages	4 years ago
Mike Fährmann	0cac14c3bd	update extractor test results	4 years ago
Mike Fährmann	53cc498d9c	improve config lookup when there are multiple possible locations ... This specifically applies to all Mastodon extractors and all extractors with a 'basecategory', i.e. 'booru', 'foolslide', etc. Values inside those general config locations wouldn't be recognized when a value with the same was set on the 'extractor' level. For example 'extractor.mastodon.directory' should be used over 'extractor.directory' when both are set, but this was impossible with the previous implementation. (fixes #843)	4 years ago
Mike Fährmann	d81a8e6544	[twitter] update tests	4 years ago
Mike Fährmann	37d71f6e09	strip microseconds in text.parse_datetime()	4 years ago
Mike Fährmann	6db7ed90cb	release version 1.14.1	4 years ago
Mike Fährmann	087e3184dc	use a non-twitter URL when testing snap creation	4 years ago
Mike Fährmann	7daef6ee70	update extractor test results ... - certain posts on Instagram now return https://static.cdninstagram.com/rsrc.php/null.jpg for public users - MangaDex is deploying its new MangaDex@Home network similar to exhentai's Hentai@Home - realbooru has a new site layout, but the underlying booru API still works like before	4 years ago
Mike Fährmann	3bad1579ee	update extractor test results	4 years ago
Mike Fährmann	45baa13615	update extractor test results ... - don't run Instagram tests on Travis anymore - replace Twitter test because timeline was made private - update Hiperdex domain to '.com' (again ...)	4 years ago
Mike Fährmann	dfcf2a2c91	write OAuth token to cache by default (#616)	4 years ago
Mike Fährmann	6294e2c540	add 'text.ensure_http_scheme()'	4 years ago
Mike Fährmann	ece73b5b2a	make 'path' and 'keywords' available in logging messages ... Wrap all loggers used by job, extractor, downloader, and postprocessor objects into a (custom) LoggerAdapter that provides access to the underlying job, extractor, pathfmt, and kwdict objects and their properties. __init__() signatures for all downloader and postprocessor classes have been changed to take the current Job object as their first argument, instead of the current extractor or pathfmt. (#574, #575)	4 years ago
Mike Fährmann	4b606b68e4	skip OAuth tests when server is unreachable	4 years ago
Mike Fährmann	8b60bd6a91	mock 'time()' in cache tests ... instead of calling 'sleep()' to let time advance. This shortens the time needed to run those tests, and ensures consistent results. (Tests would randomly fail when using 'sleep()')	4 years ago
Mike Fährmann	8f2c1da041	skip example config tests if files are not available (#730)	4 years ago
Mike Fährmann	5df8f2959b	insert local directory into PYTHONPATH when running tests	4 years ago
Mike Fährmann	ff47641b13	test whether default/example config files contain valid JSON	4 years ago
Mike Fährmann	d6facdee7b	[mastodon] add tests (#701)	4 years ago
Mike Fährmann	fd438f0d78	update extractor test results	5 years ago
Mike Fährmann	a0f4c295c0	add optional 'utcoffset' argument to 'parse_datetime()'	5 years ago
Mike Fährmann	406449b0d6	ensure keys for mastodon instances are available during tests ... Calls to config.clear() from other tests are removing the API credentials set when importing mastodon.py for the first time.	5 years ago
Mike Fährmann	9e7dfc0cfc	[myportfolio] fix extraction of galleries without title	5 years ago
Mike Fährmann	3b50c4f49d	add tests for "Extractors" in oauth.py (#670)	5 years ago
Mike Fährmann	04bd0472de	add tests for Extractor.wait()	5 years ago
Mike Fährmann	7499d71d02	[simplyhentai] ignore certificate errors in video test	5 years ago
Mike Fährmann	4203dc0bdc	[mangapark] fix metadata extraction	5 years ago
Mike Fährmann	77fda8190c	[35photo] simplify/remove tests for the 'genre' extractor ... There is still a nice genre overview page (https://35photo.pro/genre/) but the individual sub-pages don't list photos anymore	5 years ago