Mike Fährmann
bd08ee2859
remove most 'yield Message.Version' statements
...
only leave them in oauth.py as noop results
3 years ago
Nyasume
fa6af46756
Added ability to download GIFs instead of mp4 from Luscious and Reactor ( #1701 )
3 years ago
Mike Fährmann
21c2da454f
update extractor test results
3 years ago
Mike Fährmann
2c60c7d798
[reactor] skip deleted/empty posts
3 years ago
Mike Fährmann
bae874f370
replace 'wait-min/-max' with 'sleep-request'
...
on exhentai, idolcomplex, reactor
4 years ago
Mike Fährmann
3df527ee2c
update extractor test results
4 years ago
Mike Fährmann
65ca923b4e
fix 'whitelist' option for BaseExtractor instances
4 years ago
Mike Fährmann
912eea29bc
update extractor test results
4 years ago
Mike Fährmann
1e3dd7330e
merge SharedConfigMixin functionality into Extractor
4 years ago
Mike Fährmann
968d3e8465
remove '&' from URL patterns
...
'/?&#' -> '/?#' and '?&#' -> '?#'
According to https://www.ietf.org/rfc/rfc3986.txt , URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
4 years ago
dawidsowa
43b156fb40
[reactor] match URLs without subdomain ( #1053 )
4 years ago
Mike Fährmann
7619152988
[reactor] sort 'tags'
...
to ensure a consistent order for test results
4 years ago
Mike Fährmann
c50d60a53d
[reactor] fix image URLs
5 years ago
Mike Fährmann
b1db194c14
[reactor] update and improve
...
- split 'tags' into a list
- parse 'date' into a datetime object
- fix webm/mp4 URLs
5 years ago
Mike Fährmann
0f02e85961
[reactor] use "/full/" URLs ( closes #210 )
...
Putting a "/full/" in image URLs potentially gives higher resolution
and better quality.
6 years ago
Mike Fährmann
5530871b5a
change results of text.nameext_from_url()
...
Instead of getting a complete 'filename' from an URL and splitting that
into 'name' and 'extension', the new approach gets rid of the complete
version and renames 'name' to 'filename'. (Using anything other than
{extension} for a filename extension doesn't really work anyway)
Example: "https://example.org/path/filename.ext "
before:
- filename : filename.ext
- name : filename
- extension: ext
now:
- filename : filename
- extension: ext
6 years ago
Mike Fährmann
2e516a1e3e
store the full original URL in Extractor.url
6 years ago
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor
6 years ago
Mike Fährmann
6284731107
simplify extractor constants
...
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
6 years ago
Mike Fährmann
050bc1aa4a
[reactor] simplify tests
...
Some posts have, for whatever reason, a slightly different text
formatting the first time they are accessed that day
compared to any further time.
6 years ago
Mike Fährmann
4d656a81ca
replace SharedConfigExtractor class with a Mixin
6 years ago
Mike Fährmann
1734a6c879
[reactor] detect "circular" redirects ( #148 )
6 years ago
Mike Fährmann
e53cdfd6a8
update build_supportedsites.py
6 years ago
Mike Fährmann
e95b24f056
[reactor] add wait-min & -max options ( #148 )
6 years ago
Mike Fährmann
8e01cf0ef8
[reactor] generalize extractors ( #148 )
...
- support *.reactor.cc domains
- combine joyreactor and pornreactor modules
6 years ago