4cf3d54718
[kissmanga] workaround for CAPTCHAs ( fixes #818 )
...
Requesting the same page again when being redirected to a CAPTCHA
lets us access that page without solving it.
4 years ago
8fe858eb0e
improve parameter extraction when solving Cloudflare challenge
4 years ago
1d4a369ea2
update extractor test results
5 years ago
d3e44e899d
raise NotFoundErrors for 404 responses in GalleryExtractors
5 years ago
4409d00141
embed error messages in StopExtraction exceptions
5 years ago
7a99e85943
[kissmanga] fix download URLs and file extensions
...
The current Blogspot image URLs hosted on Kissmanga end with an
"invalid" query parameter (/000.png&upx=...), which doesn't get
recognized by 'spliturl()' and 'parseurl()' as such and gets therefore
included in the 'extension' field from 'text.nameext_from_url()'.
5 years ago
4465a3ea68
[kissmanga][readcomiconline] add 'captcha' option ( #279 )
...
to configure how to handle CAPTCHA page redirects:
- either interactively wait for the user to solve the CAPTCHA
- or raise StopExtraction like before
5 years ago
48233f00c0
[readcomiconline] detect 'AreYouHuman' redirects ( #279 )
5 years ago
6dae6bee37
automatically detect and bypass cloudflare challenge pages
...
TODO: cache and re-apply cfclearance cookies
6 years ago
5530871b5a
change results of text.nameext_from_url()
...
Instead of getting a complete 'filename' from an URL and splitting that
into 'name' and 'extension', the new approach gets rid of the complete
version and renames 'name' to 'filename'. (Using anything other than
{extension} for a filename extension doesn't really work anyway)
Example: "https://example.org/path/filename.ext "
before:
- filename : filename.ext
- name : filename
- extension: ext
now:
- filename : filename
- extension: ext
6 years ago
32edf4fc7b
add '_extractor' info to manga extractor results
6 years ago
580baef72c
change Chapter and MangaExtractor classes
...
- unify and simplify constructors
- rename get_metadata and get_images to just metadata() and images()
- rename self.url to chapter_url and manga_url
6 years ago
4b1880fa5e
propagate 'match' to base extractor constructor
6 years ago
6284731107
simplify extractor constants
...
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
6 years ago
6126615698
update URLs for supportedsites.rst
6 years ago
b7b5456a32
[kissmanga] use HTTPS
6 years ago
fe96835d25
[kissmanga] add fallback for chapter-string parsing ( #20 )
6 years ago
cc36f88586
rename safe_int to parse_int; move parse_* to text module
7 years ago
1b80fa82a9
[imgur] update URL pattern and tests
7 years ago
6b72be8ee6
[tumblr] add 'hash' keyword
...
'hash' is the middle part of the filename in a tumblr image URL.
For example an image with '.../tumblr_p6tgemp1NZ1wgha4yo1_250.png' as
its URL would have 'p6tgemp1NZ1wgha4yo1' as hash.
7 years ago
d11fcf4804
smaller changes and fixes
...
- fix the cloudflare challenge result if the last decimal places
are zero (JS`s toFixed() removes trailing zeroes)
- fix downloading of kissmanga chapter-pages hosted on blogspot
(accessing blogspot with "kissmanga.com" as referrer yields a 401)
- disable certificate validation for 'mangahere' tests
- update flickr test result
7 years ago
179bcdd349
adjust archive-ids
7 years ago
5b3c34aa96
use generic chapter-extractor in more modules
7 years ago
0dd48d644f
update test results
...
nothing broke, but things got updated or changed
7 years ago
28cd78aae0
[kissmanga] extend chapter-string regex ( closes #58 )
7 years ago
038e3b3369
[kissmanga] handle "AreYouHuman" redirects ( #51 )
7 years ago
d68a24aa70
[kissmanga] fix extraction
...
site changed '\n' to '\r\n' for newlines
7 years ago
68a0a7579c
fix/improve some regular expressions
7 years ago
633b376f35
improve/adjust default filename formats for manga sites
7 years ago
9fc1d0c901
implement and use 'util.safe_int()'
...
same as Python's 'int()', except it doesn't raise any exceptions and
accepts a default value
7 years ago
04cc1ffe34
[kissmanga] extract manga metadata
7 years ago
6f30cf4c64
change keyword names to valid Python identifiers
...
This commit mostly replaces all minus-signs ('-') in keyword names with
underscores ('_') to allow them to be used in filter-expressions. For
example 'gallery-id' got renamed to 'gallery_id'.
(It is theoretically possible to access any variable, regardless of its
name, with 'locals()["NAME"]', but that seems a bit too convoluted if
just 'NAME' could be enough)
7 years ago
4357966a70
[kissmanga] make URL pattern case-insensitive (fixes 28)
7 years ago
2633337833
[kissmanga] update regex ( fixes #20 )
7 years ago
f226417420
simplify code by using a MangaExtractor base class
7 years ago
e41efbd2d9
[kissmanga] fix edge-case
8 years ago
ffd72424bf
[kissmanga] another attempt at getting the AES key
8 years ago
244ab75cad
[kissmanga] update AES key retrieval
8 years ago
fd3f84d913
[kissmanga] get AES key from site itself
8 years ago
f537ad5f2f
[kissmanga] re-enable module
8 years ago
bad623751f
[hentaihere] transition to https
8 years ago
331b413c60
update extractor test script
8 years ago
94e10f249a
code adjustments according to pep8 nr2
8 years ago
40dbea7ed2
rewrite parts of the cloudflare bypass system
8 years ago
2449825d53
[kissmanga] solve cloudflare challenge on demand
8 years ago
9e3788175e
implement decorator for cloudflare bypass
...
this method for enabling and caching a cloudflare bypass for a
requests.session object allows for different cache-timeouts for
different domains
8 years ago
5c69a380f0
[kissmanga] code simplifications
8 years ago
56d810c896
update keyword hashes for tests
8 years ago
19c2d4ff6f
remove explicit (sub)category keywords
8 years ago
d7e168799d
consistent extractor naming scheme + docstrings
8 years ago
Mike Fährmann