Mike Fährmann
2ff73873f0
[erolord] add gallery extractor ( closes #326 )
5 years ago
Mike Fährmann
b4da8c5a97
[sexcom] add extractor for related pins ( #325 )
5 years ago
Mike Fährmann
40da44b17f
Merge branch 'v1.9.0'
5 years ago
Mike Fährmann
a9c89085fb
[instagram] implement login support ( #195 )
5 years ago
Mike Fährmann
18a1f8c6cd
[vanillarock] add post and tag extractors ( closes #254 )
5 years ago
Mike Fährmann
f0c5093812
[nsfwalbum] add album extractor ( closes #287 )
5 years ago
Mike Fährmann
ec3e8601f1
[slickpic] add user extractor ( #249 )
5 years ago
Mike Fährmann
d997c10320
[8muses] add album extractor ( #305 )
5 years ago
Mike Fährmann
2184e3a86b
[slickpic] add album extractor ( #249 )
5 years ago
Mike Fährmann
c73c2cda50
[pornhub] add gallery & user extractor ( #282 )
5 years ago
Mike Fährmann
096009367b
[xhamster] add gallery & user extractor ( #281 )
5 years ago
Mike Fährmann
d33f5a7423
[wallhaven] rewrite
...
- use API
- remove login support, add 'api-key' option
- remove support for "alpha" subdomain - alpha.wallhaven.cc used numeric
IDs that can't be translated to the new ID system
- support direct links to wallpapers
5 years ago
Mike Fährmann
c849574def
[keenspot] add comic extractor ( #223 )
...
Doesn't work for
- http://brawlinthefamily.keenspot.com/
- http://flipside.keenspot.com/
- http://lastblood.keenspot.com/
- http://mysticrevolution.keenspot.com/
- http://porcelain.keenspot.com/
- http://twokinds.keenspot.com/
yet, because of custom layouts.
5 years ago
Mike Fährmann
bcd1801aa8
[sankakucomplex] add 'tag' extractor ( #258 )
5 years ago
Mike Fährmann
74c2415138
[sankakucomplex] move article extractor to its own module ( #258 )
5 years ago
Mike Fährmann
1e3e15c4f3
[sankaku] add article extractor ( #258 )
5 years ago
Mike Fährmann
e82cadac61
[patreon] add extractors ( #226 )
5 years ago
Mike Fährmann
4891f4a328
[hentainexus] add search extractor ( #256 )
5 years ago
Mike Fährmann
ba8eb1ffec
[hentainexus] add gallery extractor ( #256 )
5 years ago
Mike Fährmann
0318c610dc
[sexcom] add extractor for search results ( #147 )
5 years ago
Mike Fährmann
a247c94c34
[sexcom] add pin and board extractors ( #147 )
5 years ago
Mike Fährmann
0b2ff406f6
[plurk] add timeline- and post-extractors ( #212 )
6 years ago
Mike Fährmann
e7cd5510d5
[pixnet] add extractors ( closes #177 )
...
for:
- users/blogs: http://albertayu773.pixnet.net/
- folders: https://albertayu773.pixnet.net/album/folder/1405768
- sets : https://albertayu773.pixnet.net/album/set/15078995
- photos : https://albertayu773.pixnet.net/album/photo/159443828
6 years ago
Mike Fährmann
d9b94a585d
[mangoxo] add login support ( #184 )
...
A very recent change: It is now only possible to see more
than the first 5 images of an album if you are logged in.
6 years ago
Mike Fährmann
e730fc9045
[twitter] add login support ( #214 )
6 years ago
Mike Fährmann
35919a9bb8
[livedoor] add blog- and post-extractors ( #190 )
6 years ago
Mike Fährmann
2ff043edfa
[yaplog] add user- and post-extractors ( #190 )
6 years ago
Mike Fährmann
790f15a56f
[photobucket] use HTTPS
6 years ago
Mike Fährmann
6da665f32e
[mangoxo] add album- and channel-extractors ( closes #184 )
6 years ago
Mike Fährmann
c70b21248d
[wikiart] add extractors ( #179 )
...
for
- artists: https://www.wikiart.org/en/thomas-cole
- artist-listings: https://www.wikiart.org/en/artists-by-century/12
- artwork-listings: https://www.wikiart.org/en/paintings-by-media/grisaille
6 years ago
Mike Fährmann
bc2020e86c
release version 1.8.1
6 years ago
Mike Fährmann
8d96a8ce4c
[500px] add user-, gallery-, and image-extractors ( #185 )
6 years ago
Mike Fährmann
6277a739e4
[35photo] add user-, genre-, and image-extractors ( #162 )
6 years ago
Mike Fährmann
6f57d44ec2
[seaotterscans] remove extractor
...
http://seaotterscans.com/ now redirects to their MangaDex profile
6 years ago
Mike Fährmann
976ccb267f
[myportfolio] combine gallery and user extractors
...
An URL alone isn't good enough to distinguish between a gallery or a
gallery-listing, so the new extractor decides what to do based on the
page's content.
6 years ago
Mike Fährmann
9c0e2f294b
[shopify] add generic collection and product extractors ( #175 )
...
with fashionnova.com as a default domain
6 years ago
Mike Fährmann
13e0f2a78f
[deviantart] add 'scraps' extractor ( closes #168 )
6 years ago
Mike Fährmann
280531c8ff
[pururin] add gallery extractor ( closes #174 )
6 years ago
Mike Fährmann
3159dd79d5
[seiga] use HTTPS
6 years ago
Mike Fährmann
ea6fbfdd39
simplify build_supportedsites.py
6 years ago
Mike Fährmann
22d3a2fcc8
[artstation] add extractor for artwork listings ( #80 )
...
like https://www.artstation.com/artwork?sorting=latest
or https://www.artstation.com/artwork?sorting=picks
6 years ago
Mike Fährmann
937a802b49
[dynastyscans] add extractors for images and image searches
...
(closes #163 )
6 years ago
Mike Fährmann
19860655a3
[weibo] add 'user' and 'status' extractors
6 years ago
Mike Fährmann
53c2fd4664
add mastodon/foolslide/foolfuuka examples to example config
6 years ago
Mike Fährmann
d36ec51e5a
[tsumino] add extractor for search results ( #161 )
6 years ago
Mike Fährmann
bfbbac4495
[tsumino] add login capabilities ( #161 )
6 years ago
Mike Fährmann
6126615698
update URLs for supportedsites.rst
6 years ago
Mike Fährmann
80a75a1ecf
[tsumino] add gallery extractor ( #161 )
6 years ago
Mike Fährmann
0c32dc5858
[hentaifox] add extractor for search results ( #160 )
6 years ago
Mike Fährmann
0156189468
[hentaifox] add chapter extractor ( #160 )
6 years ago
Mike Fährmann
e4171d6baf
[luscious] add login capabilities ( closes #159 )
6 years ago
Mike Fährmann
4b441c162e
release version 1.6.3
6 years ago
Mike Fährmann
e53cdfd6a8
update build_supportedsites.py
6 years ago
Mike Fährmann
a36f52a730
[joyreactor] add extractor for search results ( #148 )
6 years ago
Mike Fährmann
6c71e9cf5d
[deviantart] add separate 'sta.sh' extractor ( #113 )
...
- supports multiple stashed deviations per page
- explicitly mentions sta.sh support on supportedsites.rst
6 years ago
Mike Fährmann
be52069cbc
update CHANGELOG and docs/supportedsites
6 years ago
Mike Fährmann
2cf3f53839
[yuki] add thread extractor ( closes #111 )
6 years ago
Mike Fährmann
17611bfec0
update build_supportedsites.py script
6 years ago
Mike Fährmann
c402cc4047
[hentaifoundry] add 'popular' and 'recent' extractors
...
for "Popular Pictures" and "Recent Pictures" listings
6 years ago
Mike Fährmann
a5fc311dfa
[hentaifoundry] add 'favorite' extractor
6 years ago
Mike Fährmann
006f75b538
[hentaifoundry] rewrite + more metadata
...
- extract width, height, artist per image
- improve pattern regex
- better extensibility for other listings
6 years ago
Mike Fährmann
6ea9a78588
[wallhaven] add login capabilities
...
Being logged in is required to access NSFW wallpapers.
6 years ago
Mike Fährmann
c9290d8212
[wallhaven] add wallpaper and search extractors
...
todo:
- login support to gain access to NSFW wallpapers
- extractors for tag-, similar-, latest-listings
- skip() support
6 years ago
Mike Fährmann
22ab509a70
[bobx] rename "model" to "idol" extractor
6 years ago
Mike Fährmann
69fd61ea86
[bobx] add gallery and model extractors
6 years ago
Mike Fährmann
3ecea4cf36
[hentaicafe] add chapter and manga extractors ( #101 )
6 years ago
Mike Fährmann
75862715ac
[behance] add user extractor
6 years ago
Mike Fährmann
ab2bfaeb46
[ngomik] add replacement for 'subapics'
...
http://subapics.com/ got discontinued and replaced by http://ngomik.in/ .
ngomik.in is still displaying a link to the "old site" showing a big
"Account Suspended" sign.
6 years ago
Mike Fährmann
e9dd2eff1d
[twitter] add extractor for media-tweet timelines ( #96 )
...
For example "https://twitter.com/PicturesEarth/media ".
They are different from normal timelines in that they do not contain
any (re)tweets from other users and feature all media the user ever
posted, including responses to other tweets.
6 years ago
Mike Fährmann
9b1c39032c
[twitter] changes and improvements
...
- rename User- to TimelineExtractor
- rename 'userid' to 'user_id' to conform to the other ..._id values
- adjust archive_fmt to deal with retweets
- emulate browser behavior for API calls
6 years ago
Mike Fährmann
10365394d7
[twitter] add support for user-timelines ( closes #96 )
...
also adds a 'retweets' option to filter retweeted content
6 years ago
Mike Fährmann
63fa0b2006
[pinterest] add extractors for related pins
...
Related pins can not be accessed by adding a "#related" fragment
to the end of a Pinterest URL, for example:
- https://www.pinterest.com/pin/858146903966145189/#related
- https://www.pinterest.com/g1952849/test-/#related
There are no explicit real URLs for related pins,
using an option to enable them results in "clunky" code,
and a custom "related:<URL>" scheme doesn't feel right either.
6 years ago
Mike Fährmann
3c90df6635
[piczel] add user, folder and image extractors
6 years ago
Mike Fährmann
df082e923c
[behance] add gallery extractor ( #95 )
6 years ago
Mike Fährmann
84854fcad7
[myportfolio] add user and gallery extractors ( #95 )
6 years ago
Mike Fährmann
c9f70e0a19
[paheal] use HTTPS
6 years ago
Mike Fährmann
c20c0a4820
[safebooru] add pool extractor
6 years ago
Mike Fährmann
188e956c4e
[imagefap] use HTTPS + update test results
6 years ago
Mike Fährmann
cdcc3427a0
[simplyhentai] add video extractor ( #89 )
...
All videos hosted on their own servers seem be to dead,
but myhentai.tv embeds, which are most of the videos, work fine.
6 years ago
Mike Fährmann
55b0913412
[simplyhentai] add gallery extractor ( #89 )
6 years ago
Mike Fährmann
ecdc3475b8
[pixhost] support .to TLDs
6 years ago
Mike Fährmann
909d105ae6
[pixiv] add extractor for illusts from followed users
6 years ago
Mike Fährmann
f3483a2b7c
[smugmug] add OAuth support
6 years ago
Mike Fährmann
82c50fa609
release version 1.3.5
6 years ago
Mike Fährmann
16e014baaa
[smugmug] added image and album extractor
...
just some initial code that still requires a lot of work ...
TODO:
- folders
- old-style albums (which are nearly all of them ...)
- images from users
- OAuth
It could also happen that the API credentials used will become invalid
whenever my 14 day trial period ends (7 days remaining), but that
would just require users to supply their own.
6 years ago
Mike Fährmann
d96b3474e5
[puremashiro] remove module
...
site has been unreachable for a couple of weeks
and now the DNS record is gone as well
6 years ago
Mike Fährmann
b44a296404
[gomanga] remove module
...
site has been unreachable for a couple of weeks
and the cloudflare status page shows host errors
6 years ago
Mike Fährmann
d10579edb5
[pinterest] improve PinterestAPI code; remove OAuth mentions
...
on another note: access_tokens have been set to only allow for
10 requests per hour (from 200 yesterday)
7 years ago
Mike Fährmann
4bd182c107
[pinterest] implement `oauth:pinterest` ( #83 )
...
Pinterest access tokens are rate limited at 200 requests per
hour (or maybe per 2 or 3 hours?) so having just one access token
for all users isn't going to work in the long run.
7 years ago
Mike Fährmann
dd49127408
[spectrumnexus] remove module
...
Site stopped hosting manga scans (http://view.thespectrum.net/ )
7 years ago
Mike Fährmann
564e12ca8f
replace 'imgyt' with 'imxto'
...
https://img.yt/ wasn't available for a couple of days, but has now
re-emerged as https://imx.to/ with a new web-interface.
Links to older images still work (see tests).
7 years ago
Mike Fährmann
7b562907c3
[nijie] add favorites extractor
...
adds support for 'https://nijie.info/user_like_illust_view.php?id= ...'
7 years ago
Mike Fährmann
f39153b6e9
[nhentai] add extractor for search results
7 years ago
Mike Fährmann
52d41c41e7
[exhentai] add extractor for favorited galleries
7 years ago
Mike Fährmann
a993d0ea90
release version 1.3.2
7 years ago
Mike Fährmann
e7525b1b0e
[artstation] add challenge extractor ( #80 )
7 years ago
Mike Fährmann
40ca562d7b
[artstation] add album extractor ( #80 )
7 years ago
HRXN
241eda8c3d
Update supportedsites.rst ( #81 )
...
Add ArtStation to Supported Sites.
7 years ago
Mike Fährmann
d6ef52897c
[imgchili] remove module
...
All previously hosted images yield a 404
and the main page is just a logo.
7 years ago
Mike Fährmann
7847ab1d5a
[imagehosts] remove even more dead sites
...
All removed sites either
- reject all incoming connections or
- display a message from their domain registrar
7 years ago
Mike Fährmann
1400868f53
[mangadex] general improvements
...
- support >100 chapter entries per manga
- custom archive ID format
- detect non-existing chapters
7 years ago
Mike Fährmann
749fbbfa6c
[mangadex] add chapter- and manga-extractor
7 years ago
Mike Fährmann
6e38cf5aab
[mangareader] use 'https://'
...
The site now redirects from http://mangareader.net/
to https://mangareader.net/
7 years ago
Mike Fährmann
8f338347b6
[imagehosts] cleanup
...
removed
- chronos.to - unable to resolve hostname
- coreimg.net - same
- imgmaid.net - same
- hosturimage.com - everything returns 404
- imageontime.org - redirects to some shady site
- imgupload.yt - cloudflare error 522, host down
- img4ever.net - read timeout
7 years ago
Mike Fährmann
54c36a8a34
[subapics] add chapter- and manga-extractor ( #70 )
7 years ago
Mike Fährmann
2dd3aeeeae
[komikcast] add chapter- and manga-extractor ( #70 )
7 years ago
Mike Fährmann
1e93955170
[batoto] remove module
...
Site officially shut down on 2018.01.18
7 years ago
Mike Fährmann
40d35c87bc
[paheal] add tag- and post-extractors ( closes #69 )
7 years ago
Mike Fährmann
cc0c2cca57
[reddit] add extractor for reddit-hosted images ( closes #68 )
7 years ago
Mike Fährmann
35e09869d1
[mangapark] fix image URLs and use HTTPS
7 years ago
Mike Fährmann
29d75fc3fa
[tumblr] add support for OAuth authentication ( #65 )
7 years ago
Mike Fährmann
b33efc99a4
[idolcomplex] add support for idol.sankakucomplex.com
7 years ago
Mike Fährmann
5b094328b5
[puremashiro] add chapter- and manga-extractor ( closes #66 )
...
Also adds support for region subtags in language codes (e.g. en-us)
7 years ago
Mike Fährmann
fc7d165c97
[deviantart] add support for OAuth2 authentication
...
Some user galleries [*] require you to be either logged in or
authenticated via OAuth2 to access their deviations.
[*] e.g. https://polinaegorussia.deviantart.com/gallery/
--------------
known issue:
A deviantart 'refresh_token' can only be used once and gets updated
whenever it is used to request a new 'access_token', so storing its
initial value in a config file and reusing it again and again is not
possible.
7 years ago
Mike Fährmann
0a9a07a6e1
[slideshare] improve metadata; flake8
...
- added 'views' and 'published' keywords
- fixed longer titles and descriptions
7 years ago
Mike Fährmann
19a6ae57b2
[sankaku] add pool extractor
7 years ago
Mike Fährmann
300346ecdf
[mangazuki] remove extractors
...
This site has been in "rebuild"-mode for a fairly long time and the
current extractor code isn't going to work for the new version either.
7 years ago
Mike Fährmann
214972bc9a
[gelbooru] use manual extraction
...
... to compensate for their disabled API.
(https://gelbooru.com/index.php?page=forum&s=view&id=3875 )
This also adds an extractor for image-pools.
7 years ago
Mike Fährmann
2d0cfb33e1
[xvideos] add user profile extractor ( #45 )
7 years ago
Mike Fährmann
6af921a952
[sankaku] rewrite/improve ( fixes #44 )
...
- add wait-time between HTTP requests similar to exhentai
- add 'wait-min' and 'wait-max' options
- increase retry-count for HTTP requests to 10
- implement user authentication (non-authenticated users can only view
images up to page 25)
- implement 'skip()' functionality (only works up to page 50)
- implement image-retrieval for pages >= 51
- fix issue with multiple tags
7 years ago
Mike Fährmann
cebf800a7f
[foolfuuka] add support for more sites ( #18 )
...
- https://arch.b4k.co
- https://archive.whatisthisimnotgoodwithcomputers.com
- https://archive.yeet.net
Notes:
- The name "whatisthisimnotgoodwithcomputers" is way too long ...
- archive.yeet.net is out of date and also blocked by 4chan servers
- newest threads are 2 weeks old
- using "https://archive.yeet.net " as Referer header results in
"403 Forbidden" when accessing 4chan
7 years ago
Mike Fährmann
49c7e70c10
[acidimg] add image extractor
7 years ago
Mike Fährmann
18e6ed1c7e
[booru] add extractors for "Popular" images
7 years ago
Mike Fährmann
4ea82ea556
[warosu] add thread extractor
7 years ago
Mike Fährmann
abd7c559cd
[yonkouprod] remove module
...
Every manga chapter on this site has been removed.
7 years ago
Mike Fährmann
d7cb3c668a
update supportedsites.rst
7 years ago
Mike Fährmann
4a60f6068a
[mangazuki] add manga extractor
7 years ago
Mike Fährmann
1cb1d2e0a3
[mangazuki] add chapter extractor
7 years ago
Mike Fährmann
74a2b5af06
update supportedsites.rst
7 years ago
Mike Fährmann
1dac76fd1c
update extractor docstrings
7 years ago
Mike Fährmann
fb1904dd59
add 'Authentication' column to supportedsites.rst
7 years ago
Mike Fährmann
4e80e0c884
[flickr] add user extractor ( #16 )
7 years ago
Mike Fährmann
c921b4f32a
code cleanup and fixing tests
7 years ago
Mike Fährmann
67791e1b36
[imgur] improve and add image extractor
7 years ago
Mike Fährmann
832a4a8ee9
[fallenangels] add manga extractor
7 years ago
Mike Fährmann
2974d782a3
[yomanga] remove module
...
site has been shut down
7 years ago
Mike Fährmann
677c8ced11
[deviantart] add "journal" extractor
...
(#14 )
7 years ago
Mike Fährmann
1e416e5227
update README
7 years ago