Mike Fährmann
059b1ee5e8
add base class for job types
9 years ago
Mike Fährmann
75ff08f73c
[nijie] proper login and other issues fixed
9 years ago
Mike Fährmann
2dfed4d40a
implement -g,--get-urls option
9 years ago
Mike Fährmann
50ec170b00
[mangapark] add manga extractor
9 years ago
Mike Fährmann
0ff437ca88
[mangapark] add chapter extractor
9 years ago
Mike Fährmann
995d8578bf
[deviantart] fix dates in image-extractor
9 years ago
Mike Fährmann
7d965a2897
[deviantart] add single image extractor
9 years ago
Mike Fährmann
e4a661fd6b
[deviantart] fix issue with small images
9 years ago
Mike Fährmann
3ebd126b35
[imgbox] add single image extractor
9 years ago
Mike Fährmann
a71805bacf
[chronos] add extractor
9 years ago
Mike Fährmann
30e47848cc
update setup.py and bump version
9 years ago
Mike Fährmann
cb53b29391
[imagetwist] adjust pattern and other fixes
9 years ago
Mike Fährmann
65c8566326
add provisional generic extractor
9 years ago
Mike Fährmann
e7512f626d
[turboimagehost] add extractor
9 years ago
Mike Fährmann
56876b76f1
[imagetwist] add extractor
9 years ago
Mike Fährmann
810aa4e146
add a space after status indicators
9 years ago
Mike Fährmann
b2401542eb
implement output.mode and .shorten options
9 years ago
Mike Fährmann
8d257510d0
rework output module
9 years ago
Mike Fährmann
bd87b5186d
[spectrumnexus] recognize volumes
9 years ago
Mike Fährmann
2ced575c4a
[spectrumnexus] add manga-extractor
9 years ago
Mike Fährmann
a8c0b4531d
fix issue with Ctrl+c on windows
9 years ago
Mike Fährmann
140f0475a6
allow lists to specify base-dir segments
9 years ago
Mike Fährmann
d26eec8470
workarounds for windows console
9 years ago
Mike Fährmann
4b377ccc09
use output-module during downloads
9 years ago
Mike Fährmann
c786843a6f
add output-module
9 years ago
Mike Fährmann
f7c47a6018
add subcategories to extractors
9 years ago
Mike Fährmann
81dcfbec90
initial support for extractor-subcategories
9 years ago
Mike Fährmann
1146ed95dc
improved get_base_directory
9 years ago
Mike Fährmann
1497da07de
remove unused format-strings
9 years ago
Mike Fährmann
c23eaccf46
[hentaifoundry] add single image extractor
9 years ago
Mike Fährmann
34210edf9a
[mangashare] add manga-extractor
9 years ago
Mike Fährmann
f48712c9c9
docstrings
9 years ago
Mike Fährmann
9cb184d409
[powermanga] support another url-pattern
9 years ago
Mike Fährmann
1eaf0a55a6
[mangamint] add manga-extractor
9 years ago
Mike Fährmann
914062d172
use text.extract_iter where applicable
9 years ago
Mike Fährmann
99b4fbb081
implement text.extract_iter
9 years ago
Mike Fährmann
332d9e393b
[mangahere] support sub-chapters (e.g. ch4.5)
9 years ago
Mike Fährmann
88739a3564
[mangahere] add manga-extractor
9 years ago
Mike Fährmann
e78937a580
cache cookies from cloudflare-bypass
9 years ago
Mike Fährmann
2b93fdba7c
[kissmanga] unify extractor metadata in base class
9 years ago
Mike Fährmann
763792f207
[kissmanga] add manga-extractor
9 years ago
Mike Fährmann
a1c09ff04d
[mangapanda] fix regex
9 years ago
Mike Fährmann
a7243b5ac5
bugfix: add patterns to cache before returning them
9 years ago
Mike Fährmann
d8f692f711
[mangapanda] add manga-extractor
9 years ago
Mike Fährmann
63d693866c
[mangareader] remove leading spaces from manga names
9 years ago
Mike Fährmann
20efe49f83
[mangareader] unify extractor metadata in base class
9 years ago
Mike Fährmann
d5349c8cb5
[mangareader] add manga-extractor (all chapters)
9 years ago
Mike Fährmann
3a93faa372
add and implement Queue message
9 years ago
Mike Fährmann
d96f4f8299
bump version
9 years ago
Mike Fährmann
d1673d912a
[mangahere] add chapter-extractor
9 years ago
Mike Fährmann
8b6f02a615
[mangamint] add chapter extractor
9 years ago
Mike Fährmann
3fb5a8b834
delay 'requests'-import
9 years ago
Mike Fährmann
a4c880f4c5
[pixiv] add bookmark extractor
9 years ago
Mike Fährmann
daf5fcd416
[pixiv] add user-favorite extractor
9 years ago
Mike Fährmann
20a4782173
[pixiv] get rid of an unnecessry api-call
9 years ago
Mike Fährmann
2a97296e69
[pixiv] add single work/illust extractor
9 years ago
Mike Fährmann
9251ea5771
[powermanga] update to new format
9 years ago
Mike Fährmann
68e7f7427d
Merge branch 'multi-extractor'
9 years ago
Mike Fährmann
c0aaf230d5
remove comments
9 years ago
Mike Fährmann
4d56b76aa8
update all other extractors
9 years ago
Mike Fährmann
ad12180027
[redhawkscans] remove module
9 years ago
Mike Fährmann
bcb23e44ce
[powermanga] transfer code from redhawkscans
9 years ago
Mike Fährmann
2dfe97dd00
[chan] update to new format
9 years ago
Mike Fährmann
2d498b19a1
remove 'info' parameter
9 years ago
Mike Fährmann
78234f0572
[safebooru] update to new format
9 years ago
Mike Fährmann
bb8f2a3e9d
[gelbooru] update to new format
9 years ago
Mike Fährmann
63a38edd05
[konachan] update to new format
9 years ago
Mike Fährmann
23208a49b8
[e621] update to new format
9 years ago
Mike Fährmann
1bce63124b
[3dbooru] update to new format
9 years ago
Mike Fährmann
70a6599904
[yandere] update to new format
9 years ago
Mike Fährmann
15dec86e6a
[danbooru] update to new format
9 years ago
Mike Fährmann
74199422a8
[booru] add base classes for tag/pool/post extractors
9 years ago
Mike Fährmann
900577b013
restructure info-parameters
9 years ago
Mike Fährmann
205ef3ca02
[danbooru] add single image/post extractor
9 years ago
Mike Fährmann
ee1ba22d05
[danbooru] add pool extractor
9 years ago
Mike Fährmann
80917f5392
[danbooru] rewrite to use multiple extractors
9 years ago
Mike Fährmann
8c3783706d
allow multiple extractors per module
9 years ago
Mike Fährmann
e9547ebdf3
[exhentai] configurable wait-times
9 years ago
Mike Fährmann
9c63027d2c
[exhentai] fix 'imgkey' handling
9 years ago
Mike Fährmann
4440b40234
[exhentai] better language keywords
9 years ago
Mike Fährmann
1c9296140c
[hitomi] provide proper language keywords
9 years ago
Mike Fährmann
7fd284a705
always provide lowercase fileextensions
9 years ago
Mike Fährmann
7547d8be3f
[spectrumnexus] add additional regex
9 years ago
Mike Fährmann
c2f0720184
code cleanup to use nameext_from_url
9 years ago
Mike Fährmann
ca523b9f64
add helper method to text module
9 years ago
Mike Fährmann
41f5971e73
[mangashare] add extractor
9 years ago
Mike Fährmann
3f2fbd874d
[hbrowse] add extractor
9 years ago
Mike Fährmann
5db2aa29cf
Merge branch 'options'
9 years ago
Mike Fährmann
b55870909c
properly implement -c,--config option
9 years ago
Mike Fährmann
b683427d42
properly implement -d,--dest options
9 years ago
Mike Fährmann
b041906c2c
make URL argument not necessary for some switches
9 years ago
Mike Fährmann
fdcc434b3d
implement --list-modules cmd-line option
9 years ago
Mike Fährmann
783e62b621
[hentaifoundry] code cleanup
9 years ago
Mike Fährmann
eb4a87b620
[hentaifoundry] add extractor
9 years ago
Mike Fährmann
247c5fcf8f
Merge branch 'options'
9 years ago
Mike Fährmann
90247977c9
implement --list-keywords cmd-line option
9 years ago
Mike Fährmann
edf3f7b69f
add KeywordJob class
9 years ago
Mike Fährmann
dd796e91c0
[spectrumnexus] add extractor
9 years ago
Mike Fährmann
db113bd87d
rename download.py
9 years ago
Mike Fährmann
47f016a016
remove DownloadManager class
9 years ago
Mike Fährmann
352950eebe
new method to import downloaders
9 years ago
Mike Fährmann
21a6416737
bugfixes
9 years ago
Mike Fährmann
73eeed3e72
[imgur] rewrite
9 years ago
Mike Fährmann
4427734b70
[imgur] fix parsing
9 years ago
Mike Fährmann
129ca282e4
[chan] unescape title
9 years ago
Mike Fährmann
20845d86f8
bump version
9 years ago
Mike Fährmann
3eaae19345
cmd line switch to set options
9 years ago
Mike Fährmann
3510faaed5
[sankaku] always use correct file-url
9 years ago
Mike Fährmann
3560935318
[exhentai] fix module dicstring
9 years ago
Mike Fährmann
60833abcc6
[sankaku] re-enable extractor
9 years ago
Mike Fährmann
36b376b5ba
[mangastream] add extractor
9 years ago
Mike Fährmann
009761fcd5
[mangapanda] add extractor
9 years ago
Mike Fährmann
5abe32a32c
[kissmanga] fix internal regex
9 years ago
Mike Fährmann
079330785a
document and fix smaller bugs
9 years ago
Mike Fährmann
1302f5a0f6
[kissmanga] add extractor
9 years ago
Mike Fährmann
822527a2ce
module to bypass cloudflare protection
9 years ago
Mike Fährmann
3b0fe8f544
unify booru filename-patterns
9 years ago
Mike Fährmann
c05dbd15b5
[gelbooru] support authentication to enable api access
9 years ago
Mike Fährmann
8469b3ae4f
[safebooru] add extractor
9 years ago
Mike Fährmann
6772b54bc4
[konachan] add extractor
9 years ago
Mike Fährmann
8e9bd1d793
sensible output if no extractor found
9 years ago
Mike Fährmann
6e67935b59
add more languages to iso639-1 dict
9 years ago
Mike Fährmann
6ed26864ea
[batoto] rewrite and fix
9 years ago
Mike Fährmann
686a4ddd3e
[8chan] fix file-urls
9 years ago
Mike Fährmann
950bebfd57
update setup.py and bump version
9 years ago
Mike Fährmann
c0efea339e
[imagebam] rewrite/fix
9 years ago
Mike Fährmann
e3e754eed7
[imgbox] use text.extract_all
9 years ago
Mike Fährmann
1352187758
[imgth] use text.extract_all
9 years ago
Mike Fährmann
353ac1e00b
[exhentai] use text.extract_all
9 years ago
Mike Fährmann
1fa6a99f18
[mangareader] rewrite
9 years ago
Mike Fährmann
d0bebd9ce3
allow adding values to existing dict
9 years ago
Mike Fährmann
629133a27a
document text.extract
9 years ago
Mike Fährmann
692d0c95cc
reimplement text.extract_all
9 years ago
Mike Fährmann
506242740f
fix capitalization issues
9 years ago
Mike Fährmann
cc5e37aeae
[powermanga] adjust regex
9 years ago
Mike Fährmann
ec6c1fd18e
[redhawkscans] use langcode to -name mapping
9 years ago
Mike Fährmann
b24bb1da7d
new module for simple language code to name mapping
9 years ago
Mike Fährmann
6dcfed16fc
[powermanga] just inherit from redhawkscans
9 years ago
Mike Fährmann
d6325865e8
[redhawkscans] generalize to allow subclassing
9 years ago
Mike Fährmann
b47f3b9d78
[exhentai] reenable extractor
9 years ago
Mike Fährmann
db479f881d
implement text.shorten_path/filename methods
9 years ago
Mike Fährmann
882b1f31b1
update setup.py and bump version
9 years ago
Mike Fährmann
2472d0722a
[hitomi] more metadata
9 years ago
Mike Fährmann
9cfe2c78a5
add extractor 'imgth'
9 years ago
Mike Fährmann
25f48101e5
add extractor 'hitomi'
9 years ago
Mike Fährmann
6ce181bcfe
add extractor 'nhentai'
9 years ago
Mike Fährmann
cf8f4cd179
[imgur] add to list of extractors
9 years ago
Mike Fährmann
c47bb4d570
add extractor 'imgur'
9 years ago
Mike Fährmann
89f938ee55
handle non string-like arguemnts for clean_path
9 years ago
Mike Fährmann
4cb419b5d9
[chan] unescape html entities in title
9 years ago
Mike Fährmann
7b83f88c64
replace illegal characters in paths
9 years ago
Mike Fährmann
c14b9895ad
add extractor 'powermanga'
9 years ago
Mike Fährmann
f5d5684119
[pixiv] reuse api-tokens
9 years ago
Mike Fährmann
bea33ae9cb
implement config.setdefault
9 years ago
Mike Fährmann
c8e0efe1ff
[pixiv] refresh access-token every 50 minutes
9 years ago
Mike Fährmann
e37846e1c1
bump version number
9 years ago
Mike Fährmann
b8b48ef795
[deviantart] add to extractor list
9 years ago
Mike Fährmann
76f32addf9
[booru] skip images without 'file-url' attribute
9 years ago
Mike Fährmann
0e5ef06ac0
Merge branch 'loader'
9 years ago
Mike Fährmann
733ae601da
enable user-specified patterns
9 years ago
Mike Fährmann
49e91bcb73
adjust loader to new config-interface
9 years ago
Mike Fährmann
e23aaa4298
Merge branch 'config' into loader
9 years ago
Mike Fährmann
539faa0322
remove SequentialExtractor class
9 years ago
Mike Fährmann
7d7c1afe20
[deviantart] adjust constructor paramters
9 years ago
Mike Fährmann
88442ad26d
Merge branch 'config'
9 years ago
Mike Fährmann
3c13548f29
rewrite extractors to use config-module
9 years ago
Mike Fährmann
608d3193a9
use new config-module in downloader
9 years ago
Mike Fährmann
2026223ed1
change argument format for config-calls
9 years ago
Mike Fährmann
ae535ba0c1
add extractor 'deviantart'
9 years ago
Mike Fährmann
9986a5ffb5
json-based config module
9 years ago
Mike Fährmann
42b8e81a68
rewrite extractors to use text-module
9 years ago
Mike Fährmann
c5801c9770
combine text related functions in new module
9 years ago
Mike Fährmann
d7e0d81bdd
[8chan] use api
9 years ago
Mike Fährmann
d8ef128e74
[4chan] update default filename and directory
9 years ago
Mike Fährmann
1998ec9b13
[pixiv] update user-agent to newest version
9 years ago
Mike Fährmann
c9ef181b3c
[4chan] use api
9 years ago
Mike Fährmann
c8e5b2e89e
base class for futaba-chan boards with api
9 years ago
Mike Fährmann
26bb9d62de
move and rework extractor-loading code
9 years ago
Mike Fährmann
4348694629
sanatize output
9 years ago
Mike Fährmann
63992363a6
[pixiv] handle timestamps at the end of urls
9 years ago
Mike Fährmann
13ebca2a48
[mangareader] supply correct width and height
9 years ago
Mike Fährmann
2afe15b217
[redhawkscans] let requests handle content decoding
9 years ago
Mike Fährmann
675937c77c
[mangareader] add extractor
9 years ago
Mike Fährmann
58dfd1a38f
[redhawkscans] fix chapter parsing
9 years ago
Mike Fährmann
a9e68c2745
[redhawkscans] fix json extraction
9 years ago
Mike Fährmann
b3f656ffd5
[redhawkscans] add extractor
9 years ago
Mike Fährmann
d9d88f1448
[pixiv] fix ugoira parsing
9 years ago
Mike Fährmann
68c4306040
[pixiv] transition to pixiv public api
9 years ago
Mike Fährmann
c0530a635c
[batoto] a few fixes
...
- removed 'volume' fields from default format strings
- fixed parsing by making volumes optional
- removed debug output
- updated docstring
- added a few unescape calls
10 years ago
Mike Fährmann
a145a73d05
add extractor 'nijie'
10 years ago
Mike Fährmann
9c25c15438
[3dbooru] fix default regex
10 years ago
Mike Fährmann
5993177ae3
[4chan] more metadata entries (size, width, height)
10 years ago
Mike Fährmann
a2cfbe445f
add extractor '3dbooru'
10 years ago
Mike Fährmann
762ad7426a
add extractor 'yandere'
10 years ago
Mike Fährmann
07f23dc2fa
add extractor '4chan'
10 years ago
Mike Fährmann
fae1b28c48
[pixiv] fix parsing of illust_ids
10 years ago
Mike Fährmann
ffec41f352
[imgchili] update to new extractor interface
10 years ago
Mike Fährmann
f3e208c0ab
[batoto] update to new extractor interface
10 years ago
Mike Fährmann
3c6161eea7
small whitespace fixes
10 years ago
Mike Fährmann
593a29b677
[imgbox] update to new extractor interface
10 years ago
Mike Fährmann
e41768d969
[imagebam] update to new extractor interface
10 years ago
Mike Fährmann
758fe00441
[e621] update to new extractor interface
10 years ago
Mike Fährmann
5cb46a2cb3
[gelbooru] update to new extractor interface
10 years ago
Mike Fährmann
1f4866fcd6
[danbooru] update to new extractor interface
10 years ago
Mike Fährmann
8c7da93d19
move and update BooruExtractor classes
10 years ago
Mike Fährmann
28fa7c53b4
docstrings and other small fixes for downloaders
10 years ago
Mike Fährmann
5545624da1
use seperate session in http downloader
10 years ago
Mike Fährmann
8f24329e30
[pixiv] fix http headers and cookies
10 years ago
Mike Fährmann
5806e02f97
better support for KeyboardInterrupt exceptions
10 years ago
Mike Fährmann
1cd25b5369
[pixiv] update to new extractor interface
10 years ago
Mike Fährmann
7c8d787077
add a few docstrings
10 years ago
Mike Fährmann
30b9db43a0
[8chan] combine patterns
10 years ago
Mike Fährmann
3872ccb435
pylint fixes
10 years ago
Mike Fährmann
ce2fab9c7e
allow user-specified formats for filenames
10 years ago
Mike Fährmann
652d60a5cd
implement new extractor-module selection
10 years ago
Mike Fährmann
cd4a699dd2
add 'Headers' and 'Cookies' message
10 years ago
Mike Fährmann
7b9fcea6bb
small fixes
10 years ago
Mike Fährmann
f31c05f522
[8chan] update to new extractor interface
10 years ago
Mike Fährmann
0abbee3710
update download-infrastructure
10 years ago
Mike Fährmann
513808d156
move code from util.py
10 years ago
Mike Fährmann
41f00809ff
update extractor base classes
10 years ago
Mike Fährmann
63bf0d1725
update __init__.py
10 years ago
Mike Fährmann
7c12ba1c31
move DownloadManager and ExtractorFinder
10 years ago
Mike Fährmann
d13608e909
[pixiv] fix parsing of illust-ids
10 years ago
Mike Fährmann
a66a5099f8
added extractor 'imgchili'
10 years ago
Mike Fährmann
ac002cc4b3
[8chan] fix image urls
10 years ago
Mike Fährmann
54d40b85fb
[pixiv] load balancing for image servers
10 years ago
Mike Fährmann
0f6cc964cc
[8chan] use new domainname for thread-urls
10 years ago
Mike Fährmann
34d13bc906
added extractor 'danbooru' + split BooruExtractor to handle XML and JSON
10 years ago
Mike Fährmann
2a7dbd8868
[booru] unquote tags to fix multiple tags in urls
10 years ago
Mike Fährmann
099fe7515e
added extractor 'e621'
10 years ago
Mike Fährmann
1244c933a1
[8chan] fix image urls
10 years ago
Mike Fährmann
729d2d8b20
[imagebam] fixed issue with destination direcotry name
10 years ago
Mike Fährmann
98dd5f9a90
added extractor 'imagebam'
10 years ago
Mike Fährmann
ff53b7f6fa
added extractor 'imgbox'
10 years ago
Mike Fährmann
4950cb0ab9
use 'safe_request' to handle connection errors
10 years ago
Mike Fährmann
b630753e5e
add 'method' parameter
10 years ago
Mike Fährmann
7588974306
fixed various bugs
...
- forgot "self." before "name_fmt"
- image keys where off by one
10 years ago
Mike Fährmann
deef91eddc
initial commit
10 years ago