Mike Fährmann
dedea3b4db
[deviantart] fix journal creation ( #400 )
5 years ago
Mike Fährmann
c6c5cb1898
improve 'deviantart.quality' description
5 years ago
Mike Fährmann
8eaae58045
[downloader:http] change log message level to 'debug'
5 years ago
Mike Fährmann
efb64ad031
[deviantart] generate filenames ( #392 , #400 )
5 years ago
Mike Fährmann
0ce98169b8
improve path generation
...
- fix 'abspath()' results for Python <3.7 (closes #402 )
- 'abspath()' in Python 3.7+ removes trailing path separators
- in Python <3.7 it doesn't
- filter empty path segments
5 years ago
Mike Fährmann
b2151f3928
[seiga] support mobile URLs ( closes #401 )
5 years ago
Mike Fährmann
20fd2d8450
[flickr] skip unavailable images/videos ( fixes #398 )
5 years ago
Mike Fährmann
c9b97dbca3
extend post processor tests
5 years ago
Mike Fährmann
60c8e090da
[postprocessor:zip] fix archive names ( closes #397 )
...
Remove the trailing path separator introduced in 3284c62
before
adding the archive's filename extension.
[ci skip]
5 years ago
Mike Fährmann
7c09545f70
[downloader:ytdl] add 'outtmpl' option ( #395 )
5 years ago
Mike Fährmann
5cc7be2536
[piczel] update and improve
...
- use proper pagination (fixes #396 )
- update API host and endpoints
- "fix" double slash // in image URLs
5 years ago
Mike Fährmann
0c1c7abb4d
release version 1.10.2
5 years ago
Mike Fährmann
49f6d7176d
[deviantart] restore filenames ( #392 )
...
<title>_by_<user>_<id> --> <title>_by_<user>-<id>
5 years ago
Mike Fährmann
63daa68d67
[deviantart] improvements ( #392 )
...
- consistent 'filename' entries, at least as far as possible
- GIFs and SWFs don't have a <title>_by_<artist>_<id> anywhere in
their metadata
- Generating <id> (from 'deviationid'?) might be something that needs
to be figured out, so we can build those filenames ourselves
- better code structure etc.
- tests for videos, archives, and flash animations
5 years ago
Mike Fährmann
d1db5180ab
[simplyhentai] fix extraction; remove image+video extractors
5 years ago
Mike Fährmann
30d6e284b0
[deviantart] use NAPI for artworks and scraps ( #392 )
...
TODO:
- journal downloads
- test for all media types
5 years ago
Mike Fährmann
e528f3cb77
adjust postprocessor test results
...
see 2495b99
5 years ago
Mike Fährmann
7d6af936c5
[imgur] simplify gallery extraction
5 years ago
Mike Fährmann
3284c62f22
ensure PathFormat.directory ends with a path separator
...
... plus some other small optimizations
5 years ago
Mike Fährmann
ebabc5caf1
[downloader:http] treat 416 without downloaded data as error
...
Downloading https://pbs.twimg.com/media/EB2cGUYX4AI2Vuu.jpg:orig (NSFW)
sometimes returns a 416 status code, even though no 'Range' header was
sent and no data was downloaded prior.
This code usually means a file has already been downloaded completely
and the download method indicates success, but in this case it causes
an exception down the pipeline since no file was created.
5 years ago
Mike Fährmann
2495b99347
[postprocessor:classify] improve path generation ( fixes #138 )
...
It still doesn't work for converted ugoira animations thanks to how
those files are handled, but everything else, including files with
unknown or changing file extension, now works as it should.
5 years ago
Mike Fährmann
e77a656437
optimize directory path generation
...
- use str.join() instead of os.path.join()
(less "features", but 10x as fast)
- cache directory formatters
- detect and optimize field access for 1-element format strings
5 years ago
Mike Fährmann
51d10783fc
[patreon] include image info in API results ( #383 )
5 years ago
Mike Fährmann
7a5e78741c
[booru] build directory path for each file ( #385 )
5 years ago
Mike Fährmann
b1728f512d
[patreon] support multi image posts and post URLs ( #383 )
5 years ago
Mike Fährmann
454bf1ebf9
preserve enumeration index after 'set_extension()' ( #306 )
5 years ago
Mike Fährmann
f5039b897f
replace DownloadArchive.check() with __contains__()
...
Interestingly enough, 'a in obj' is slightly faster than
'obj.check(a)' and is also nicer to look at, I think.
5 years ago
Mike Fährmann
5a210991b6
Remove control characters from filesystem paths
...
- add 'path-remove' option to specify the set of characters that
should be removed
- rename 'restrict-filenames' to 'path-restrict'
- #348 , #380
5 years ago
Mike Fährmann
c50d60a53d
[reactor] fix image URLs
5 years ago
Mike Fährmann
32447d0d24
[pixiv] simplify default filename format
...
(#366 )
5 years ago
Mike Fährmann
5f8621b29d
improve output of active post processor modules
5 years ago
Mike Fährmann
2cbbc3dec4
add a 'whitelist' to '--ugoira-conv' ( #382 )
5 years ago
Mike Fährmann
e661a1ce19
update links to Request's documentation
5 years ago
Mike Fährmann
cb7f149974
fix mtime datetime test
...
datetime.timestamp() uses local time for a naive datetime object
5 years ago
Mike Fährmann
829b1ccf04
[imgur] distinguish album and gallery URLs ( #380 )
...
A gallery can be either an album or a single image.
5 years ago
Mike Fährmann
23251356cb
require 'extension' data for each URL ( #382 )
5 years ago
Mike Fährmann
a67413d64f
[xhamster] use input URL domain
...
Don't rewrite all URLs as 'https://xhamster.com/ ...'
5 years ago
Mike Fährmann
dd72ae7164
add postprocessor tests
5 years ago
Mike Fährmann
0bb873757a
update PathFormat class
...
- change 'has_extension' from a simple flag/bool to a field that
contains the original filename extension
- rename 'keywords' to 'kwdict' and some other stuff as well
- inline 'adjust_path()'
- put enumeration index before filename extension (#306 )
5 years ago
Mike Fährmann
423f68f585
[deviantart] fix scraps extraction ( closes #376 )
5 years ago
Mike Fährmann
748e37554c
update .travis.yml
...
- install pyOpenSSL before running tests
- simplify snap tests
5 years ago
leezhi0513
42bb0cdb9b
Update gallery-dl.conf ( #378 )
5 years ago
Mike Fährmann
3bf20ffb70
[instagram] add support for story highlights
5 years ago
Mike Fährmann
a732e9c430
[instagram] update query hashes and headers
5 years ago
Mike Fährmann
2ccf6a9e35
[instagram] make extractor tests happy ( #373 )
5 years ago
Mike Fährmann
8dc42bb178
implement 'enumerate' for 'extractor.skip' ( #306 )
...
[ci skip]
5 years ago
Leonardo Taccari
bc5eaf7746
[instagram] Add support for IGTV ( #373 )
...
Add support for IGTV profile (instagram.com/<username>/channel/)
and IGTV medias (instagram.com/tv/<short_id>).
5 years ago
Mike Fährmann
b7fb93e2b2
[downloader:http] add 'adjust-extensions' option
5 years ago
Mike Fährmann
eb7da159e2
[imagebam] update URL test results
...
Image URLs are now using https://, but the website itself is still
served as http://.
5 years ago
Mike Fährmann
189acbeac9
[imgbb] add extractor for individual images ( closes #363 )
5 years ago