dd884b02ee
replace json.loads with direct calls to JSONDecoder.decode
2 years ago
a6305d031c
[hitomi] apply format check for every image ( #3030 ) ( #3280 )
2 years ago
b2b0b1c455
[hitomi] fall back to webp when format not available ( #3030 )
2 years ago
2eb0ddd083
[hitomi] fix error when number of tag results is multiple of 25
...
(#2870 )
2 years ago
946643c23c
[hitomi] use maxage for gg.js cache ( #2863 )
...
cached values become invalid after 1-2 hours
2 years ago
37d584a9b2
[hitomi] update metadata extraction ( fixes #2444 )
...
remove 'hitomi.metadata' option, as it is no longer necessary
to make additional HTTP requests to fetch all metadata.
3 years ago
dee0d22561
update extractor test results
3 years ago
86fa412b47
[hitomi] add 'format' option ( #2260 )
...
default is 'webp' since downloading original files is no longer allowed
3 years ago
17c9c47ca0
[hitomi] fix 'tag' extraction ( fixes #2189 )
3 years ago
8b910dd8ae
[hitomi] fix image URLs
...
again and again ...
3 years ago
38e2af29d6
[hitomi] fix image URLs
...
update '_parse_gg()' yet again
3 years ago
1e0278702d
[hitomi] update '_parse_gg()'
3 years ago
becc7f85a6
[hitomi] fix image URLs
3 years ago
099ed72de7
[hitomi] disable extra 'metadata' by default
...
safes one HTTP request that not needed with default filename settings
3 years ago
211de95dd0
update extractor test results
3 years ago
14852f7050
[hitomi] fix image path ( #1988 )
3 years ago
d4614e5ba4
[hitomi] fix image URLs ( #1982 )
3 years ago
6b6d92d51c
[hitomi]: fix image URLs ( #1975 )
3 years ago
47a780942c
update extractor test results
3 years ago
9429eaa0a3
[hitomi]: fix image URLs ( #1765 )
3 years ago
5612ca31c2
[hitomi] fix image URLs ( closes #1679 )
3 years ago
e98fa01c44
[hitomi] update image URL code ( fixes #1637 )
3 years ago
968d3e8465
remove '&' from URL patterns
...
'/?&#' -> '/?#' and '?&#' -> '?#'
According to https://www.ietf.org/rfc/rfc3986.txt , URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
4 years ago
ffd38215a4
[hitomi] fix image URLs and URL pattern
...
- non-webp files are now hosted on [a-c]b.hitomi.la
- removed ampersand from invalid slug characters
4 years ago
7cd383c0f9
update extractor test results
4 years ago
deaacc70bb
[hitomi] update URL pattern for tag searches
4 years ago
7140fe7e6d
[hitomi] fix redirect processing
4 years ago
a3de234e70
[hitomi] add extractor for tag searches ( closes #697 )
4 years ago
55ac408bdf
[hitomi] fix extraction of galleries without tags
4 years ago
59edcdc822
[hitomi] restore metadata fields from before f33b13a
...
... and add a 'metadata' option to disable
visiting the gallery page and extracting data from it
if this is not needed.
5 years ago
f33b13aacf
[hitomi] simplify metadata extraction
...
Use the data from https://ltn.hitomi.la/galleries/ <id>.js for both
image URLs and metadata and ignore any gallery or reader pages.
This removes 'artist', 'characters', 'group', and 'parody' metadata
fields since this information is, as for now, only available in
gallery pages.
5 years ago
80ecb99089
[hitomi] fix extraction
5 years ago
5607dd3646
[hitomi] follow multiple redirects
5 years ago
d1de7dc296
[hitomi] implement workaround for "broken" redirects
...
Some galleries redirect to a new "version" with different gallery id.
This new version might not be available any more, but the /reader/
page for the original gallery id can still work.
5 years ago
8bb32ee188
[hitomi] fix image URLs
5 years ago
f8ac67ce50
[hitomi] extend URL pattern + follow redirects
5 years ago
8361d874d7
[hitomi] fix extraction
5 years ago
1693d97bd3
update extractor class hierarchies
...
- let the GalleryExtractor class inherit directly from Extractor
- make ChapterExtractor a subclass of GalleryExtractor
- change enumeration field names of GalleryExtractors to 'num'
5 years ago
15af2f8464
[hitomi] fallback to /reader/ page if main page returns 404
...
Some galleries return a 404: Not Found error when trying to access
them through the main gallery URL, but their content is still
available on the respective /reader/ page.
5 years ago
cf5e716b9d
[hitomi] fix image URLs
5 years ago
a732e9c430
[instagram] update query hashes and headers
5 years ago
055102431f
[hitomi] handle Game CG galleries with scenes ( fixes #321 )
5 years ago
b51baa9a4b
[hitomi] fix empty language detection; parse datetime
5 years ago
fc5e4f2b21
[hitomi] simplify data extraction code
5 years ago
2756cc8dde
[hitomi] set Referer header ( fixes #239 )
5 years ago
26c4365baa
adjust metadata types for GalleryExtractors
6 years ago
3595cd582f
use GalleryExtractor as common base class
6 years ago
5530871b5a
change results of text.nameext_from_url()
...
Instead of getting a complete 'filename' from an URL and splitting that
into 'name' and 'extension', the new approach gets rid of the complete
version and renames 'name' to 'filename'. (Using anything other than
{extension} for a filename extension doesn't really work anyway)
Example: "https://example.org/path/filename.ext "
before:
- filename : filename.ext
- name : filename
- extension: ext
now:
- filename : filename
- extension: ext
6 years ago
580baef72c
change Chapter and MangaExtractor classes
...
- unify and simplify constructors
- rename get_metadata and get_images to just metadata() and images()
- rename self.url to chapter_url and manga_url
6 years ago
4b1880fa5e
propagate 'match' to base extractor constructor
6 years ago
Mike Fährmann