526 Commits (57508d3bb73a55db3a2f67b6c7104deb779589d0)

Author	SHA1	Message	Date
Mike Fährmann	5b43faffed	[postprocessor:metadata] write to stdout by setting filename to "-" ... (#2624)	2 years ago
Mike Fährmann	6ad39f2b68	add ytdl tests ... they only run when youtube-dl or yt-dlp are installed, i.e. if __import__("<ytdl-package>") succeeds	2 years ago
Mike Fährmann	688d6553b4	replace calls to print() with stdout_write() (#2529)	2 years ago
Mike Fährmann	f3408a9d92	implement string literals in replacement fields ... - either {_lit[foo]} or {'foo'} - useful as alternative for empty metadata fields: {title|'no title'} - due to using '_string.formatter_field_name_split()' to parse format strings, using certain characters will result in an error: [].:!	2 years ago
Mike Fährmann	c4b9f7bab8	update functions working with cookies.txt files ... - rename - load_cookiestxt -> cookiestxt_load - save_cookiestxt -< cookiestxt_store - in cookiestxt_load, add cookies directly to a cookie jar instead of storing them in a list first - other unnoticeable performance increases	2 years ago
Mike Fährmann	ca3a364db7	fix build_duration_func() (#2533) ... for extractors with request_interval_min > 0	2 years ago
Mike Fährmann	7fe54bab2a	attempt to fix some issues with 'contains()' (#2446) ... add a third argument that gets used when the values o search are given as a string	2 years ago
Mike Fährmann	d78a2c7163	re.escape() arguments for 'contains()' (#2446)	2 years ago
Mike Fährmann	413b77757b	implement 'contains()' (#2446) ... and add it to globals() in compiled expressions for --filter etc	3 years ago
Mike Fährmann	e7b30866d0	[postprocessor:mtime] fix timestamps from datetime objects (#2307) ... 'datetime.timestamp()', which got used to convert datetime objects to POSIX timestamps, assumes naive datetimes represent LOCAL time, while datetimes in 'date' metadata fields represent UTC time. Ref: https://docs.python.org/3/library/datetime.html#datetime.datetime.timestamp > Naive datetime instances are assumed to represent local time > you can obtain the POSIX timestamp by … calculating the timestamp directly	3 years ago
Mike Fährmann	29db716a63	implement 'datetime_to_timestamp()' ... and rename 'to_timestamp()' to the more descriptive 'datetime_to_timestamp_string()'	3 years ago
Mike Fährmann	8295bc6d97	fix loading/storing cookies without domain	3 years ago
Mike Fährmann	500a479026	fix a third(!) bug in _check_cookies() (#2372) ... turns out tests are worthless if you get em wrong ...	3 years ago
Mike Fährmann	cf44aba333	[formatter] allow evaluating f-string literals ... by starting a format string with '\fF'. This was technically already possible with '\fE', but this makes it a bit more convenient.	3 years ago
Mike Fährmann	94452761ed	fix cookies tests	3 years ago
Mike Fährmann	bddcec49f1	implement 'text.root_from_url()' ... use domain from input URL for kemono	3 years ago
Mike Fährmann	f5b2b9333f	fix another bug in _check:cookies (#2160) ... regression introduced in ed317bfc Added a couple of tests to hopefully catch such bugs before they land in a release.	3 years ago
Mike Fährmann	563bd0ecf4	[danbooru] inherit from BaseExtractor ... - merge danbooru and e621 code - support booru.allthefallen.moe (closes #2283) - remove support for old e621 tag search URLs	3 years ago
Mike Fährmann	b5b4f5a168	use 'build_extractor_filter' in test_results.py	3 years ago
Mike Fährmann	64cf26eaf4	allow specifying sleep-* options as string ... either as single value or as range: "3.5", "2.1 - 5.0"	3 years ago
Mike Fährmann	010d65dcec	extend blacklist/whitelist syntax (#2025) ... Each entry in such a list can now also include a subcategory '<category>:<subcategory>' and it is possible to use '*' or an empty string as placeholder '*:<subcategory>', ':<subcategory>', '<category>:*' For example "blacklist": "imgur,*:tag,gfycat:user" or "blacklist": ["imgur", "*:tag", "gfycat:user"] will filter all 'imgur' extractors, all extractors with a 'tag' subcategory (e.g. https://danbooru.donmai.us/posts?tags=bonocho), and all 'gfycat' user extractors.	3 years ago
Mike Fährmann	af6424f398	allow testing metadata in list elements	3 years ago
Mike Fährmann	3842cdcd8f	[formatter] implement 'D' format specifier ... To be able to parse any string into a 'datetime' object and format it as necessary. Example: {created_at:D%Y-%m-%dT%H:%M:%S%z} -> "2010-01-01 00:00:00" {created_at:D%Y-%m-%dT%H:%M:%S%z/%b %d %Y %I:%M %p} -> "Jan 01 2010 12:00 AM" with 'created_at' == "2010-01-01T01:00:00+0100"	3 years ago
Mike Fährmann	2ab190ce08	add tests for special format strings	3 years ago
Mike Fährmann	46e17c5e61	support accessing the current local datetime in format strings ... {_now}, {_now:%Y-%m-%d}, etc (#1968)	3 years ago
Mike Fährmann	38193dba46	support accessing environment variables in format strings (#1968) ... {_env[HOME]} to get the value of $HOME every other format string feature is supported as well	3 years ago
Mike Fährmann	f2d6b3e6b4	run tests without using 'nose' ... run_tests.sh -> run_tests.py	3 years ago
Mike Fährmann	12fc646c53	fix filename formatting tests	3 years ago
Mike Fährmann	e0bdacd932	[fappic] add 'image' extractor (closes #1898)	3 years ago
Mike Fährmann	c22ff97743	remove 'unit' argument from 'util.format_value()'	3 years ago
Mike Fährmann	cad85640de	move 'util.PathFormat' into its own 'path' module ... to prevent circular imports between 'formatter' and 'util'	3 years ago
Mike Fährmann	74145467dd	move 'util.Formatter' into its own 'formatter' module	3 years ago
Mike Fährmann	9377543162	[mastodon] add 'following' extractor (#1891)	3 years ago
Mike Fährmann	bd845303ad	implement a way to shorten filenames with east-asian characters ... (#1377) Setting 'output.shorten' to "eaw" (East-Asian Width) uses a slower algorithm that also considers characters with a width > 1.	3 years ago
Mike Fährmann	292fffc83c	add 'j' format string conversion ... to convert to a JSON formatted string	3 years ago
Mike Fährmann	bb6a130942	automatically set required DDoS-GUARD cookies (#1779) ... for kemono.party and seiso.party	3 years ago
Mike Fährmann	2792ed6e4b	implement 'util.format_value()'	3 years ago
Mike Fährmann	9e42cd58ea	replace ChainPredicate class with 'functools.partial'	3 years ago
Mike Fährmann	36ac2197db	[ytdl] add extractor for sites supported by youtube-dl ... (#1680, #878) Can be used by prefixing any URL with 'ytdl:', or by setting 'extractor,ytdl.enabled' to 'true'.	3 years ago
Mike Fährmann	64240c8d42	[imagevenue] fix extraction ... (closes #1677)	3 years ago
Mike Fährmann	0179581340	add 'T' format string conversion (#1646) ... to convert 'date'/datetime to timestamp	3 years ago
Mike Fährmann	f74cf52e2b	[seisoparty] add 'user' and 'post' extractors (#1635)	3 years ago
Mike Fährmann	759735fb02	[kemonoparty] fix 'username' extraction (fixes #1652) ... The site's <title> content changed from <title>NAME | Kemono</title> to <title> NAME | Kemono </title>	3 years ago
Mike Fährmann	07c8adbd8b	[mangadex] implement login with username & password (#1535)	3 years ago
Mike Fährmann	4a747a31a3	[postprocessor:metadata] handle dicts in mode;tags (fixes #1598)	3 years ago
Mike Fährmann	3cbbefd4ed	support 'filter' option for post processors (#1460)	3 years ago
Mike Fährmann	0abad8bc12	implement 'compile_expression()'	3 years ago
Mike Fährmann	da6806a161	fix job tests for Python 3.4 and 3.5 ... assert_called() and assert_not_called() got added in Python 3.6	3 years ago
Mike Fährmann	8fd8126117	fix ISO 639-1 code for Japanese ... "jp" -> "ja"	3 years ago
Mike Fährmann	af9dba4684	add DataJob tests	3 years ago