Mike Fährmann
844502cad5
update extractor test results
4 years ago
Mike Fährmann
7cd383c0f9
update extractor test results
4 years ago
Mike Fährmann
65744a7a31
use alternative for all falsey values in format strings
...
… and not just None (#525 )
It would be better to consistently use None for all non-existent
fields and/or fields without a valid value, but this is a good
enough workaround for now.
4 years ago
Mike Fährmann
f5b7ae01c1
update extractor test results
4 years ago
Mike Fährmann
392d022b04
implement 'config.accumulate()' ( #994 )
4 years ago
Mike Fährmann
3108e85b89
[worldthree] remove extractors
...
http://www.slide.world-three.org/ hasn't been accessible for a long time.
4 years ago
Mike Fährmann
3918b69677
remove 'extractor.blacklist' context manager
4 years ago
Mike Fährmann
ac3036ef56
add 'filesize-min' and 'filesize-max' options ( closes #780 )
4 years ago
Mike Fährmann
fd0685d9b5
[postprocessor:zip] defer zip file creation ( fixes #968 )
...
don't try to create zip files on postprocessor construction,
wait until directory creation during file download,
4 years ago
Mike Fährmann
d50f3b333a
update extractor test results
4 years ago
Mike Fährmann
e33293fdd8
[hentaihand] update to new site layout
4 years ago
Mike Fährmann
69e4871005
update extractor test results
...
- sensescans: replace 404d chapters
- mangapark: replace 404d chapters
- subscribestar: update test for attached files
4 years ago
Mike Fährmann
688bd046fc
release version 1.14.4
4 years ago
Mike Fährmann
422e69f187
skip external OAuth tests ( closes #908 )
4 years ago
Mike Fährmann
8dbf827649
[bobx] remove module
4 years ago
Mike Fährmann
87202b8d74
[inkbunny] add 'user' and 'post' extractors ( #283 )
4 years ago
Mike Fährmann
2ecf1efb16
update extractor test results
...
- tumblr: remove deleted post
- jaiminisbox: replace removed manga/chapters
- smugmug: one inconsequential field got removed
4 years ago
Mike Fährmann
e62ebb4643
update CHANGELOG before building sdist and wheel packages
4 years ago
Mike Fährmann
0cac14c3bd
update extractor test results
4 years ago
Mike Fährmann
53cc498d9c
improve config lookup when there are multiple possible locations
...
This specifically applies to all Mastodon extractors and all
extractors with a 'basecategory', i.e. 'booru', 'foolslide', etc.
Values inside those general config locations wouldn't be recognized
when a value with the same was set on the 'extractor' level.
For example 'extractor.mastodon.directory' should be used over
'extractor.directory' when both are set, but this was impossible
with the previous implementation.
(fixes #843 )
4 years ago
Mike Fährmann
d81a8e6544
[twitter] update tests
4 years ago
Mike Fährmann
37d71f6e09
strip microseconds in text.parse_datetime()
4 years ago
Mike Fährmann
6db7ed90cb
release version 1.14.1
4 years ago
Mike Fährmann
087e3184dc
use a non-twitter URL when testing snap creation
4 years ago
Mike Fährmann
7daef6ee70
update extractor test results
...
- certain posts on Instagram now return
https://static.cdninstagram.com/rsrc.php/null.jpg
for public users
- MangaDex is deploying its new MangaDex@Home network similar to
exhentai's Hentai@Home
- realbooru has a new site layout, but the underlying booru API still
works like before
4 years ago
Mike Fährmann
3bad1579ee
update extractor test results
4 years ago
Mike Fährmann
45baa13615
update extractor test results
...
- don't run Instagram tests on Travis anymore
- replace Twitter test because timeline was made private
- update Hiperdex domain to '.com' (again ...)
4 years ago
Mike Fährmann
dfcf2a2c91
write OAuth token to cache by default ( #616 )
4 years ago
Mike Fährmann
6294e2c540
add 'text.ensure_http_scheme()'
4 years ago
Mike Fährmann
ece73b5b2a
make 'path' and 'keywords' available in logging messages
...
Wrap all loggers used by job, extractor, downloader, and postprocessor
objects into a (custom) LoggerAdapter that provides access to the
underlying job, extractor, pathfmt, and kwdict objects and their
properties.
__init__() signatures for all downloader and postprocessor classes have
been changed to take the current Job object as their first argument,
instead of the current extractor or pathfmt.
(#574 , #575 )
4 years ago
Mike Fährmann
4b606b68e4
skip OAuth tests when server is unreachable
4 years ago
Mike Fährmann
8b60bd6a91
mock 'time()' in cache tests
...
instead of calling 'sleep()' to let time advance.
This shortens the time needed to run those tests,
and ensures consistent results.
(Tests would randomly fail when using 'sleep()')
4 years ago
Mike Fährmann
8f2c1da041
skip example config tests if files are not available ( #730 )
4 years ago
Mike Fährmann
5df8f2959b
insert local directory into PYTHONPATH when running tests
4 years ago
Mike Fährmann
ff47641b13
test whether default/example config files contain valid JSON
4 years ago
Mike Fährmann
d6facdee7b
[mastodon] add tests ( #701 )
4 years ago
Mike Fährmann
fd438f0d78
update extractor test results
5 years ago
Mike Fährmann
a0f4c295c0
add optional 'utcoffset' argument to 'parse_datetime()'
5 years ago
Mike Fährmann
406449b0d6
ensure keys for mastodon instances are available during tests
...
Calls to config.clear() from other tests are removing the API
credentials set when importing mastodon.py for the first time.
5 years ago
Mike Fährmann
9e7dfc0cfc
[myportfolio] fix extraction of galleries without title
5 years ago
Mike Fährmann
3b50c4f49d
add tests for "Extractors" in oauth.py ( #670 )
5 years ago
Mike Fährmann
04bd0472de
add tests for Extractor.wait()
5 years ago
Mike Fährmann
7499d71d02
[simplyhentai] ignore certificate errors in video test
5 years ago
Mike Fährmann
4203dc0bdc
[mangapark] fix metadata extraction
5 years ago
Mike Fährmann
77fda8190c
[35photo] simplify/remove tests for the 'genre' extractor
...
There is still a nice genre overview page (https://35photo.pro/genre/ )
but the individual sub-pages don't list photos anymore
5 years ago
Mike Fährmann
32e36d8f02
[sexcom] replace tests
5 years ago
Mike Fährmann
e0b0e8d62a
release version 1.13.2
5 years ago
Mike Fährmann
a63a376ad2
[mangoxo] fix login
5 years ago
Mike Fährmann
ebc70e87ce
[e621] update to new interface / API endpoints ( closes #635 )
5 years ago
Mike Fährmann
72122eb9b3
release version 1.13.1
5 years ago
Mike Fährmann
ce5e2a58fe
[imgbb] update test results
...
Image server domain changed from
https://image.ibb.co/ to https://i.ibb.co/
5 years ago
Mike Fährmann
1d4a369ea2
update extractor test results
5 years ago
Mike Fährmann
ec85bf90de
use context managers in cache.py & add tests
5 years ago
Mike Fährmann
4e361b3008
add tests for specific datetime values
5 years ago
Mike Fährmann
90e4c645ba
[formatter] allow multiple "special" format specifiers ( #595 )
...
It is now, for example, possible to specify multiple replacement
operations per format replacement field: {name:Ra/b/Rc/d/}
5 years ago
Mike Fährmann
219c4cc78c
[formatter] allow for numeric list and string indices
5 years ago
Mike Fährmann
7d1da614d9
[formatter] implement field name alternatives ( #525 )
...
The format string '{a|b|c}' will now try to use the value from 'a' and
fall back to 'b' and 'c' if accessing a field raises an exception or
if its value is None.
5 years ago
Mike Fährmann
c7cf9dd111
[furaffinity] support classic layout ( #284 )
5 years ago
Mike Fährmann
40fe062851
[pixiv] fix user id for bookmarks API calls ( closes #596 )
5 years ago
Mike Fährmann
2852691d78
[paheal] replace test URL
...
searching for 'k-on' doesn't yield any results anymore
5 years ago
Mike Fährmann
2a9be48511
improve util.load/save_cookiestxt() and add tests
...
- take a file object as argument instead of an filename
- accept whitespace before comments (" # comment")
- map expiration "0" to None and not the number 0
5 years ago
Mike Fährmann
b3b5754f2d
update test_cookies.py
5 years ago
Mike Fährmann
174117f827
allow multiple hashes for content tests
5 years ago
Mike Fährmann
60a43f0264
fix downloader tests
5 years ago
Mike Fährmann
e89413da22
update test results
5 years ago
Mike Fährmann
5cac79c3d9
[erolord] remove extractor
5 years ago
Mike Fährmann
988cc2ec23
[mangadex] change domain to mangadex.cc ( closes #559 )
5 years ago
Mike Fährmann
87c8b89ddd
[postprocessor:metadata] add 'directory' option ( #520 )
5 years ago
Mike Fährmann
82f7f4172a
update test results
5 years ago
Mike Fährmann
d0920e84e9
update test results
5 years ago
Mike Fährmann
9e63804347
[patreon] make retrieving user info nonfatal ( #508 )
...
… and fall back to the included data if an error occurs.
5 years ago
Mike Fährmann
15f9bb3d14
add option to disable pyOpenSSL usage ( #508 )
...
(pyOpenSSL is now disabled by default)
5 years ago
Mike Fährmann
50deab5265
[deviantart] fix URL generation from /extended_fetch results
...
(closes #505 )
5 years ago
Mike Fährmann
004812258d
[hentaifox] fix extraction
5 years ago
Mike Fährmann
a412531451
[postprocessor:metadata] implement 'extension-format' option
...
closes #477
5 years ago
Mike Fährmann
b5c964332b
improve config.py test coverage
5 years ago
Mike Fährmann
f5604492c3
update interface of config functions
5 years ago
Mike Fährmann
3fc1e12949
[postprocessor:metadata] filter private entries
...
i.e. keys starting with an underscore
5 years ago
Mike Fährmann
978cb03f81
update misc test results
...
- Livedoor now uses https:// for its image URLs
- Instagram image URLs got simplified
5 years ago
Mike Fährmann
bbbeff4c41
[downloader.http] implement file-specific HTTP headers
5 years ago
Mike Fährmann
3ece3976ae
[newgrounds] implement login support ( #394 )
5 years ago
Mike Fährmann
abfcb356fc
[flickr] support 3k, 4k, 5k, and 6k photo sizes ( closes #472 )
5 years ago
Mike Fährmann
da6789b2b0
disable unique archive id checks for some tests
...
- same image twice in a livedoor blog post
- unreliable results for related pinterest items
5 years ago
Mike Fährmann
ba083b30b2
fix snap build
...
… hopefully
5 years ago
Mike Fährmann
94a94f3b86
miscellaneous stuff
5 years ago
Mike Fährmann
9e88e7a344
[postprocessor:exec] improve ( #421 , #413 )
...
- add 'final' option
- include job status in pp finalization
- improve and extend documentation
5 years ago
Mike Fährmann
2a3bd4e3c7
rename extractor classes starting with a digit
5 years ago
Mike Fährmann
64786363be
[4chan] simplify
...
- remove 'chan.py'
- slight adjustments to directory and filenames
5 years ago
Mike Fährmann
557e2c018b
[8chan] remove module
5 years ago
Mike Fährmann
322c2e7ed4
renaming variables
...
mostly 'keyword(s)' to 'kwdict'
5 years ago
Mike Fährmann
87a87bff7e
[simplyhentai] fix image URLs
5 years ago
Mike Fährmann
d5e3910270
adjust 'util.raises()'
5 years ago
Mike Fährmann
b23c822b23
[luscious] use GraphQL
5 years ago
Mike Fährmann
1693d97bd3
update extractor class hierarchies
...
- let the GalleryExtractor class inherit directly from Extractor
- make ChapterExtractor a subclass of GalleryExtractor
- change enumeration field names of GalleryExtractors to 'num'
5 years ago
Mike Fährmann
7ebd984e8d
[imgur] print error message if no JSON data is found ( #446 )
5 years ago
Mike Fährmann
de4e2029d1
[nsfwalbum] update test album
...
the old one is no longer available
5 years ago
Mike Fährmann
913460240d
[reddit] fix 'extractor.blacklist()' arguments
...
The second argument must support 'append()'.
5 years ago
Mike Fährmann
1848788970
update test results etc
5 years ago
Mike Fährmann
d5fbb2d9de
[tumblr] ignore audio links from Spotify etc.
5 years ago
Mike Fährmann
c6c5cb1898
improve 'deviantart.quality' description
5 years ago