4df2cadf60
Updated README to include additional Windows installation method ( #763 )
4 years ago
0378d079a5
[webtoons] fixes and simplifications ( #593 , #761 )
...
- fix episode listings for french comics
- allow input URLs without explicit scheme
- add 'lang'/'language' metadata
- use str.format() instead of '+' to assemble URLs
4 years ago
ab11b1c896
[imagechest] simplify code ( #750 )
4 years ago
ece73b5b2a
make 'path' and 'keywords' available in logging messages
...
Wrap all loggers used by job, extractor, downloader, and postprocessor
objects into a (custom) LoggerAdapter that provides access to the
underlying job, extractor, pathfmt, and kwdict objects and their
properties.
__init__() signatures for all downloader and postprocessor classes have
been changed to take the current Job object as their first argument,
instead of the current extractor or pathfmt.
(#574 , #575 )
4 years ago
846d3a2466
[sexcom] replace 404ed test
4 years ago
9b4635917f
[gelbooru] simplify and fix pool extraction
...
use 'pool:<pool id>' as search tag to get pool posts
4 years ago
abbd8fbbd9
reset filenames on empty file extensions ( #733 )
4 years ago
d17e9628b3
fix/improve Cloudflare bypass code ( #728 , #757 )
...
- support changing values for 'k'
- use XML parser to get request parameters
(some input fields are now embedded in an HTML comment)
4 years ago
39cd389679
[webtoons] Add a new extractor for webtoons.com ( #761 )
...
The webtoons extractor can extract episode and entire comic (all
episodes) from webtoons.com.
All the logic of the extractors should be trivial except for a couple
of kludges needed:
- `ageGatePass' cookie is always set to avoid possible redirect and stop of
extraction, especially in the comic extractor
- The image URLs returned by the episode extractor could not be fetched
directly and the `Referer:' HTTP header needs to be passed to fetch them
Close #593 .
4 years ago
7b5711ee04
[imagechest] Add new extractor for ImageChest ( #750 )
...
* [imagechest] Add new extractor for ImageChest
* [imagechest] Fix flake8 compliance issues
4 years ago
b7ebf51229
Fix typo: defaut → default. ( #754 )
4 years ago
f8661c6578
[downloader:ytdl] fix file extensions when merging into mkv
4 years ago
a1e739b96c
reuse connection adapters from parent extractors
4 years ago
f8f95e68a7
improve '--write-pages' ( #737 )
...
- move code into its own function
- add enumeration index to filenames
- dump responses regardless of status code
4 years ago
dba87ca99e
[downloader:ytdl] change 'forward-cookies' default to 'false'
...
There are currently no situations where forwarding gallery-dl's cookies
to youtube-dl is necessary, and it only causes problems when forcing
youtube-dl for Twitter video downloads while logged in.
4 years ago
09cc9dbec0
prevent flake8 errors from comments looking like type annotations
4 years ago
2d6724180b
[hiperdex] update domain to hiperdex.info
4 years ago
c31e7dd573
add MANIFEST.in ( #730 )
...
include LICENSE, CHANGELOG.md, and example config files
in source tarballs created by 'python setup.py sdist'
4 years ago
be16d77073
simplify and improve documentation markup ( #742 )
...
- use | to get a more "compact" representation
- link directly to example config files
4 years ago
4cc761c730
Implement --write-pages option ( #736 )
...
* Implement --write-pages option
* Fix long lines
* Fix file mode to binary
* Fix pattern for Windows compatibility
4 years ago
fe224416bf
Some documentation improvements ( #742 )
...
* Link configuration examples in the intro, see #712
* Clarify overriding conf files by options, see #678
4 years ago
4b606b68e4
skip OAuth tests when server is unreachable
4 years ago
f557cac074
[redgifs] add image extractor ( #724 )
4 years ago
11d3247c85
use 'util.Formatter' when formatting logging output
4 years ago
8b60bd6a91
mock 'time()' in cache tests
...
instead of calling 'sleep()' to let time advance.
This shortens the time needed to run those tests,
and ensures consistent results.
(Tests would randomly fail when using 'sleep()')
4 years ago
42f29c3e11
improve and simplify attribute access in DownloadJob.initialize()
4 years ago
8f2c1da041
skip example config tests if files are not available ( #730 )
4 years ago
65b1cb7acd
[deviantart] use private access tokens for Journals ( fixes #738 )
4 years ago
0bf0146bfe
[reddit] don't send OAuth headers for file downloads ( fixes #729 )
4 years ago
ba42ec321c
release version 1.13.6
4 years ago
d6a480682f
update test results
4 years ago
5df8f2959b
insert local directory into PYTHONPATH when running tests
4 years ago
714566b6e3
fix/improve Cloudflare bypass code
4 years ago
b47cfc5ac9
[speakerdeck] Add a new extractor for speakerdeck.com ( #726 )
4 years ago
90491ab606
[artstation] improve embed extraction ( #720 )
4 years ago
999efec5cc
[deviantart] limit API wait times to 2**9=512 seconds ( #721 )
4 years ago
504de79d8b
[vsco] fix extraction
4 years ago
ff47641b13
test whether default/example config files contain valid JSON
4 years ago
5e2974d699
[weibo] add 'videos' option
4 years ago
9f638c2e01
[twitter] add 'replies' option ( closes #705 )
4 years ago
19a7afdd9b
[downloader:http] add MIME types for .psd files ( closes #714 )
4 years ago
fc3e54275b
[patreon] respect filters and sort order in query params ( #711 )
4 years ago
46b9a4d8ff
[patreon] improve hash extraction ( #693 , #713 )
...
Instead of accessing a specific part of a download URL, potentially
causing an exception if it doesn't exist, we're now searching through
all parts for a potential MD5 hash without ever raising an exception.
4 years ago
c56a751dae
[newgrounds] fix URLs produced by 'followng' extractors ( #684 )
4 years ago
baa877d853
release version 1.13.5
4 years ago
a4fd620a25
[hiperdex] revert domain back to hiperdex.com
4 years ago
233b6f93a2
[patreon] recognize URLs with creator IDs ( #711 )
...
e.g. https://www.patreon.com/user/posts?u= …
4 years ago
38b6bd66b0
[500px] match 'web.500px.com' subdomains
4 years ago
d3b3b30107
update test results
4 years ago
5d7ca76885
retry Cloudflare challenges
4 years ago
Starz0r