thatfuckingbird
dff03a6605
[booru] add an option to extract notes (only gelbooru for now) ( #1457 )
...
* [booru] add an option to extract notes (currently implemented only for gelbooru)
* appease linter
* [gelbooru] rename "text" to "body" in note extraction
* add a code comment about reusing return value of _extended_tags
3 years ago
Mike Fährmann
08d7934c6e
move extractors from booru.py into their own gelbooru_v02 module
4 years ago
Mike Fährmann
0978c1e184
[booru] use BaseExtractor
4 years ago
Mike Fährmann
e41e2be2f9
[booru] split '_prepare_post()'
4 years ago
Mike Fährmann
78061658ea
[booru] reduce exceptions caught during _prepare_post()
...
don't catch HttpErrors etc.
4 years ago
Mike Fährmann
8d2e4e5f13
[booru] improve error handling
...
e.g. for posts without a valid 'file_url' (#1176 )
4 years ago
Mike Fährmann
a3a863fc13
[booru] add generalized extractors for *booru sites
...
similar to cc15fbe7
4 years ago
Mike Fährmann
1e3dd7330e
merge SharedConfigMixin functionality into Extractor
4 years ago
Mike Fährmann
7a0ba370d1
[gelbooru] rewrite mp4 video URLs ( fixes #1048 )
4 years ago
Mike Fährmann
fb98b567fa
[gelbooru] improve post ID extraction for pools
4 years ago
Mike Fährmann
f117e32910
[danbooru] restore 'popular' functionality
5 years ago
Mike Fährmann
86c00f9e66
[danbooru] move extractor logic from booru.py
5 years ago
Mike Fährmann
f02a768b5c
[danbooru] add 'ugoira' option ( #406 )
...
to choose between ZIP archives or converted video files
for Ugoira posts
5 years ago
Mike Fährmann
7a5e78741c
[booru] build directory path for each file ( #385 )
5 years ago
Mike Fährmann
a4ba34c835
[booru] prevent crash when no tags are present ( #259 )
5 years ago
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor
6 years ago
Mike Fährmann
6284731107
simplify extractor constants
...
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
6 years ago
Mike Fährmann
4d656a81ca
replace SharedConfigExtractor class with a Mixin
6 years ago
Mike Fährmann
1d43cbbf52
[gelbooru] tag-splitting for non-api mode
6 years ago
Mike Fährmann
3dbc7c5f8d
[gelbooru] restore pool functionality
6 years ago
Mike Fährmann
a2c74bc6f0
[gelbooru] inherit from BooruExtractor class
...
Breaks pool functionality when using API calls (for now),
but reduces code clutter and enables the `tags` option.
6 years ago
Mike Fährmann
4a57509392
generalize tag-splitting option ( #92 )
...
- extend functionality to other booru sites:
- http://behoimi.org/
- https://konachan.com/
- https://e621.net/
- https://rule34.xxx/
- https://safebooru.org/
- https://yande.re/
6 years ago
Mike Fährmann
87853538b4
[yandere] add option to split tags by type ( #92 )
6 years ago
Mike Fährmann
95392554ee
use text.urljoin()
6 years ago
Mike Fährmann
3905474805
[booru] call update_page() with correct dict ( closes #82 )
7 years ago
Mike Fährmann
5008e105ee
update archive IDs
...
... to behave in a more straightforward way when dealing with
bookmarks/favourites/etc.
specific IDs are now grouped by their owner, album-id, ... to
allow for duplicates when it would be expected.
7 years ago
Mike Fährmann
34873dbd90
set 'archive_fmt' values
...
These are going to be used to create an unique id for each image.
7 years ago
Mike Fährmann
974e73bdbb
[booru] smaller code adjustments
7 years ago
Mike Fährmann
9e8a84ab6c
[booru] rewrite using Mixin classes ( #59 )
...
- improved code structure
- improved URL patterns
- better pagination to work around page limits on
- Danbooru
- e621
- 3dbooru
7 years ago
Mike Fährmann
00420ff202
[booru] consistent order for "popular" results
7 years ago
Mike Fährmann
be30fb2f98
add common config category for boorus and foolslide
7 years ago
Mike Fährmann
07214f4007
[booru] place subcategories into base classes
7 years ago
Mike Fährmann
18e6ed1c7e
[booru] add extractors for "Popular" images
7 years ago
Mike Fährmann
58e95a7487
share extractor and downloader sessions
...
There was never any "good" reason for the strict separation
between extractors and downloaders. This change allows for
reduced resource usage (probably unnoticeable) and less lines
of code at the "cost" of tighter coupling.
7 years ago
Mike Fährmann
c184e47ee3
put common directory- and filename formats in base classes
7 years ago
Mike Fährmann
85a46ed700
[booru] fix issue with multiple tags
7 years ago
Mike Fährmann
ff92674379
[booru] simple skip functionality
...
(#6 )
8 years ago
Mike Fährmann
94e10f249a
code adjustments according to pep8 nr2
8 years ago
Mike Fährmann
19c2d4ff6f
remove explicit (sub)category keywords
8 years ago
Mike Fährmann
d7e168799d
consistent extractor naming scheme + docstrings
8 years ago
Mike Fährmann
bb8f2a3e9d
[gelbooru] update to new format
9 years ago
Mike Fährmann
1bce63124b
[3dbooru] update to new format
9 years ago
Mike Fährmann
74199422a8
[booru] add base classes for tag/pool/post extractors
9 years ago
Mike Fährmann
80917f5392
[danbooru] rewrite to use multiple extractors
9 years ago
Mike Fährmann
c2f0720184
code cleanup to use nameext_from_url
9 years ago
Mike Fährmann
3b0fe8f544
unify booru filename-patterns
9 years ago
Mike Fährmann
76f32addf9
[booru] skip images without 'file-url' attribute
9 years ago
Mike Fährmann
539faa0322
remove SequentialExtractor class
9 years ago
Mike Fährmann
3c13548f29
rewrite extractors to use config-module
9 years ago
Mike Fährmann
42b8e81a68
rewrite extractors to use text-module
9 years ago