Mike Fährmann
47bcf53ec1
implement support for additional unit test result types
...
- "pattern" matches all resulting URLs against the given regex
- "count" allows to specify the amount of returned URLs
7 years ago
Mike Fährmann
2d0dfe9d56
[exhenai] init headers before login and detect sadpanda
...
- also debug-logs html after failed login
- #37
7 years ago
Mike Fährmann
c7ec103e15
[batoto] fix extraction of chapter URLs
7 years ago
Mike Fährmann
18e6ed1c7e
[booru] add extractors for "Popular" images
7 years ago
Mike Fährmann
f7cdfd4c25
add a simplified version of 'parse_qs'
...
This version only returns a dict of plain string to string key-value
pairs and ignores multiple values for the same query variable.
7 years ago
Mike Fährmann
3b21e0703c
[deviantart] allow distinction between users and groups ( #26 )
...
This is done by prepending "group-" to an extractor's subcategory
if the URL belongs to a group ("folder" becomes "group-folder" and
so on). This changes the configuration-path being used and is also
reflected in the output of '--list-keywords'.
7 years ago
Mike Fährmann
e61a3a56d1
[hentai2read] fix and update keywords
...
Added the "author" keyword and changed the name of a few others to be
consistent with other manga/chapter extractors.
7 years ago
Mike Fährmann
c45770331a
use 'str.partition()'
...
The (r)partition method is always faster then split() or any other
method that has been replaced in this commit.
7 years ago
Mike Fährmann
017a72f448
[pixiv] improve input validation
7 years ago
Mike Fährmann
dcf42c5e89
[pixiv] add extractor for ranking lists
7 years ago
Mike Fährmann
4ea82ea556
[warosu] add thread extractor
7 years ago
Mike Fährmann
6078ec5908
restructure the output of --help
...
Using argument groups is a definite improvement over how things looked
previously, but general group membership of individual items might be
a thing to reconsider.
7 years ago
Mike Fährmann
9aa95fba8c
[deviantart] adapt download URLs to use https
...
Even though DeviantArt is "completely switching over to HTTPS"[1],
every URL contained in an API response is still using HTTP
[1] https://danlev.deviantart.com/journal/DeviantArt-Is-Switching-To-HTTPS-697996906
7 years ago
Mike Fährmann
d70c66c516
fix "text:" downloader
7 years ago
Mike Fährmann
f7de048980
add additional debug output
7 years ago
Mike Fährmann
9bf9d64ad8
update unittests for util.py
7 years ago
Mike Fährmann
02e89700fc
[foolfuuka] ensure sorted posts
7 years ago
Mike Fährmann
8bcf88bff7
[flickr] fix extraction
...
This issue was only noticeable with older Python versions, as these
don't exhibit a consistent ordering of dict keys.
7 years ago
Mike Fährmann
e3bfb8325a
fix circular dependency
...
- util.py imported config.py and vice versa
- Python < 3.5 doesn't like this
7 years ago
Mike Fährmann
004456d5d5
properly update the config-dictionary
...
When using 2 or more config files, the values of the second would
improperly overwrite nested dictionaries of the first one.
The new method properly combines these nested dictionaries as well.
7 years ago
Mike Fährmann
ae2d61e5b3
handle format string exceptions separately
7 years ago
Mike Fährmann
3c9f190757
extend output of --list-keywords
7 years ago
Mike Fährmann
cfa479fab5
update error message for unspecified exceptions
...
- ask user to report unexpected errors, which usually indicate
extractor failure
- handle OSErrors separately (permissions, disk full, etc)
- revert 30eef52
7 years ago
Mike Fährmann
7e936e9c06
[luscious] simplify and remove dead code
7 years ago
Mike Fährmann
d74a635e41
[util] update 'default' values and improve test coverage
...
for 'code_to_language()' and 'language_to_code()'
7 years ago
Mike Fährmann
0245a0ba5f
fix extraction and update test results
...
- fixes for hbrowse, imgyt, imgcandy, hosturimage
- test updates for deviantart, gfycat
7 years ago
Mike Fährmann
abd7c559cd
[yonkouprod] remove module
...
Every manga chapter on this site has been removed.
7 years ago
Mike Fährmann
da7219ba74
[kisscomic] remove module
...
Image links on this site are dead.
7 years ago
Mike Fährmann
0b576cc131
mention '--upgrade' in README.rst
7 years ago
Mike Fährmann
852e7acd31
[twitter] ignore "Promoted Tweets"
7 years ago
Mike Fährmann
915a0137de
improve 'extractor.request'
...
- add 'fatal' argument
- improve internal logic and flow
- raise known exception on error
- update exception hierarchy
7 years ago
rachmadani haryono
dcd573806e
chg: dev: fix error ( #32 )
...
* fix: dev: error
* fix: dev: AttributeError when getting artist
* fix: dev: typo on luscious parser
7 years ago
Mike Fährmann
c4713404c8
[directlink] improve URL pattern
7 years ago
Mike Fährmann
d443822fdb
[luacious] get correct image URLs ( fixes #33 )
...
Instead of using thumbnail URLs and modifying them the extractor now
goes through every single image-page and gets its download URL from
there.
7 years ago
Mike Fährmann
6950708e52
[hentaicdn] use HTTPS
7 years ago
Mike Fährmann
4f1e6c109f
[deviantart] remove 'invalid escape sequence' warning
...
- use r"\w" or "\\w" instead of "\w"
7 years ago
Mike Fährmann
c864be479e
[directlink] update URL pattern & PEP 8
...
- combine some file extensions
- don't match '.je'
- line length < 80
7 years ago
H R X N
45f9d64c23
Update directlink.py with additional file exts. ( #30 )
...
Add WebP, still not that common, but it's increasing.
Add 3rd JPEG variant (https://en.wikipedia.org/wiki/JPEG#JPEG_filename_extensions )
Never seen JFIF in the wild, would probably be overkill.
Extend Ogg formats (https://en.wikipedia.org/wiki/Ogg ; https://wiki.xiph.org/MIME_Types_and_File_Extensions )
7 years ago
Mike Fährmann
4357966a70
[kissmanga] make URL pattern case-insensitive (fixes 28)
7 years ago
Mike Fährmann
493bd235cf
workaround for missing 'assert_called_once' method
...
this method was introduced in Python 3.6, but calling it still
works (i.e. it doesn't cause the test to fail) on Python 3.3/3.4
7 years ago
Mike Fährmann
7aa9fa796a
code cleanup and fixes
7 years ago
Mike Fährmann
f08af03845
Merge branch 'cookies'
7 years ago
Mike Fährmann
6db93a8b9e
add cookie tests
7 years ago
Mike Fährmann
55f048d02b
ignore case of cookiejar magic strings
7 years ago
Mike Fährmann
de68cf84a8
release version 0.9.1
7 years ago
Mike Fährmann
d7cb3c668a
update supportedsites.rst
7 years ago
Mike Fährmann
f53bf1a323
[thebarchive] add thread extractor
7 years ago
Mike Fährmann
b8cf434bb0
[rebeccablacktech] add thread extractor
7 years ago
Mike Fährmann
808f67ba7d
use 'cookiedomain' for cookies set by object-config-values
...
otherwise these cookies would not be picked up by the
_check_cookies() method.
7 years ago
Mike Fährmann
00a23fe208
update configuration.rst
7 years ago