563bd0ecf4
[danbooru] inherit from BaseExtractor
...
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283 )
- remove support for old e621 tag search URLs
3 years ago
cadfad4eea
[danbooru] add 'external' option ( closes #1747 )
3 years ago
221015e586
[downloader:http] disable filename extension changes for ugoira
...
(#1507 )
3 years ago
224b883ff4
[danbooru] add option for extended metadata extraction ( #1458 )
...
* [danbooru] add option for extended metadata extraction
* appease linter
* [danbooru] update docs/configuration.rst
* [danbooru] rename extended-metadata -> metadata
3 years ago
d781e6ac44
[e621] return pool posts in order ( closes #1195 )
...
… and add a 'num' enumeration index.
A bit more code than the PR version, but it prints some helpful messages
and doesn't call 'metadata()' twice.
4 years ago
e7d446a8f7
[danbooru] slight code refactoring
4 years ago
1e3dd7330e
merge SharedConfigMixin functionality into Extractor
4 years ago
2626629117
[danbooru] handle posts without 'id' ( fixes #1004 )
4 years ago
e19f665a44
[danbooru] change default for 'ugoira' to 'false'
...
Downloading the pre-rendered versions should be a better default
than .zip files with individual frames.
4 years ago
ad2efa8509
[e621] derive from Danbooru extractors ( #651 )
...
- use extractor implementations from 'danbooru'
- use "page": "b[ID]" to paginate over results instead of
"tags": "id:<[ID]", avoiding infinite loops with certain
post orders
- bump User-Agent version
5 years ago
5bcc7184c9
[danbooru][e621] increase page limits
5 years ago
f117e32910
[danbooru] restore 'popular' functionality
5 years ago
86c00f9e66
[danbooru] move extractor logic from booru.py
5 years ago
f02a768b5c
[danbooru] add 'ugoira' option ( #406 )
...
to choose between ZIP archives or converted video files
for Ugoira posts
5 years ago
6284731107
simplify extractor constants
...
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
6 years ago
1e4d351ad3
[danbooru] add authentication support ( closes #151 )
...
... via HTTP Basic Auth with username and "password".
The password value in this case is not the account password itself,
but the"api_key" found in your user profile.
6 years ago
e1d306cc48
update unit test results
6 years ago
6ac403c5d3
add postprocessor config example
6 years ago
3905474805
[booru] call update_page() with correct dict ( closes #82 )
7 years ago
974e73bdbb
[booru] smaller code adjustments
7 years ago
1219ebb7f5
[danbooru] use alternate subdomains; support safebooru
7 years ago
9e8a84ab6c
[booru] rewrite using Mixin classes ( #59 )
...
- improved code structure
- improved URL patterns
- better pagination to work around page limits on
- Danbooru
- e621
- 3dbooru
7 years ago
289d6b65d2
[danbooru] extend and improve URL regex
...
- add support for danbooru mirrors:
- hijiribe.donmai.us
- sonohara.donmai.us
- todo: actually use these domains instead of redirecting everything
to danbooru itself
- improve handling of query string parameters
7 years ago
65997d835b
replace popular/ranking tests with older ones
...
Metadata of several year old lists shouldn't change as much as it
would for newer ones, which makes metadata-comparisons of the output
of build_testresult_db.oy easier.
7 years ago
07214f4007
[booru] place subcategories into base classes
7 years ago
47bcf53ec1
implement support for additional unit test result types
...
- "pattern" matches all resulting URLs against the given regex
- "count" allows to specify the amount of returned URLs
7 years ago
18e6ed1c7e
[booru] add extractors for "Popular" images
7 years ago
c921b4f32a
code cleanup and fixing tests
7 years ago
cf79a47b59
update unit tests
8 years ago
ed94d9b92d
fix/improve various things
8 years ago
94e10f249a
code adjustments according to pep8 nr2
8 years ago
d7e168799d
consistent extractor naming scheme + docstrings
8 years ago
6f7d42b974
update tests
8 years ago
616e0aedd6
update booru testdata
9 years ago
ba99506c72
more extractor test-cases
9 years ago
2449532132
add more extractor test-cases
9 years ago
c8fdfbec42
[danbooru] add test cases
9 years ago
f7c47a6018
add subcategories to extractors
9 years ago
15dec86e6a
[danbooru] update to new format
9 years ago
205ef3ca02
[danbooru] add single image/post extractor
9 years ago
ee1ba22d05
[danbooru] add pool extractor
9 years ago
80917f5392
[danbooru] rewrite to use multiple extractors
9 years ago
3b0fe8f544
unify booru filename-patterns
9 years ago
3c13548f29
rewrite extractors to use config-module
9 years ago
3c6161eea7
small whitespace fixes
10 years ago
1f4866fcd6
[danbooru] update to new extractor interface
10 years ago
34d13bc906
added extractor 'danbooru' + split BooruExtractor to handle XML and JSON
10 years ago
Mike Fährmann