8c11e81c9f
Merge commit '63e6993716db8d8bedfb7b0d445c7161493046b6'
5 years ago
63e6993716
merge 'bypost' functionality into metadata postprocessor
5 years ago
31a29835ff
[realbooru] simplify extractors and update tests ( #514 )
5 years ago
9a4ce20b8e
[realbooru] Add Realbooru extractor ( #514 )
5 years ago
f9e74320de
retain trailing zeroes in Cloudflare challenge answers
5 years ago
72b8fbfbad
[instagram] make post-page extraction nonfatal
5 years ago
922b8a9595
[weibo] raise NotFoundError for unavailable/deleted statuses
5 years ago
0cd157300e
[patreon] fix regex pattern for posts
...
The previous one would match the first number in the URL slug as
post ID, which would fail for posts with numbers in their title.
5 years ago
fe19e233f3
[xvideos] improve
...
- derive from GalleryExtractor
- match '…-channels' URLs
- "better" metadata structure
5 years ago
d3e44e899d
raise NotFoundErrors for 404 responses in GalleryExtractors
5 years ago
a4dd8b3dab
improve _check_cookies()
...
Only loop over all cookies once instead of calling
cookiejar._find() for each cookie name.
5 years ago
76e60d10a6
[patreon] raise proper exception if creator/post doesn't exist
5 years ago
9e63804347
[patreon] make retrieving user info nonfatal ( #508 )
...
… and fall back to the included data if an error occurs.
5 years ago
964dc57286
[vsco] improve image resolutions
...
https://im.vsco.co/ URLs redirect to the appropriate CDN server
and occasionally insert a '/1200x1600/' into the image path,
limiting image dimensions.
This commit constructs redirect targets out of the given
im,vsco.co URLs without sending extra HTTP requests
and without any "builtin" resolution restrictions.
5 years ago
0629fe8fa4
[vsco] fix user profile extraction … again
...
Given the pattern from last time, collections will also change
in due time and use cursor-based pagination.
5 years ago
ab17ea9632
[deviantart] only print warning if 'original' is enabled
5 years ago
2188db6284
[gelbooru] fix non-API tag extraction
5 years ago
c4702ec9b6
simplify some logging calls
5 years ago
c0b9ad678d
Separate metadata from handle_url into handle_metadata, commenting
5 years ago
c9ef1b21c3
[patreon] get partial user info without /api/user/<id> ( #507 )
...
It's a lot less data, but doesn't invoke any additional
HTTP requests with potential Cloudflare CAPTCHAs.
5 years ago
0ab9bb1721
[4chan] add extractor for entire boards ( closes #510 )
5 years ago
c59b98c81b
[downloader:http] improve rate limit handling
...
- Move the download "logic" with rate limit checks into its own
method that only gets used if a rate limit should be enforced
- Fix an issue where suspending gallery-dl during a download would
basically ignore the rate limit for the remaining download when
resuming its execution.
5 years ago
bbbafc1c24
[downloader:http] catch both possible SSLException instances
...
With pyOpenSSL installed, but disabled, the SSLError exception
would be set to the one from pyOpenSSL, which could never get raised.
This commit solves this problem by catching both, the native SSLError
exception as well as the one from pyOpenSSL (if available.1)
5 years ago
c20bb5c338
Naming convention, as per travis.
5 years ago
6ed4fc07ff
Don't print intentional metadata skips to the console.
5 years ago
cfc70a97ab
Added an additional channel for downloading the metadata of an entire post or gallery.
5 years ago
f451be48c3
release version 1.12.0
5 years ago
15f9bb3d14
add option to disable pyOpenSSL usage ( #508 )
...
(pyOpenSSL is now disabled by default)
5 years ago
c8e99e3b3b
[deviantart] fix crash on missing "token" field ( #505 )
5 years ago
6ed2c7823c
[deviantart] disable original downloads if no cookies set
...
For 'deviation' and 'scraps' extractors only, since original file
downloads for those two will always fail with a 404 Not Found
when not logged in.
5 years ago
50deab5265
[deviantart] fix URL generation from /extended_fetch results
...
(closes #505 )
5 years ago
1f209da4c0
[pixiv] match new search URLs ( closes #507 )
5 years ago
e17907ee2a
change default value of 'cookies-update' to 'true'
5 years ago
07dafad26d
[twitter] attempt to fix infinite loops ( #499 )
...
(Hopefully this doesn't break anything else)
5 years ago
71acbdabf4
[2chan] fix metadata extraction
5 years ago
c0a1241648
[livedoor] force https:// for image URLs
5 years ago
6e23c0da09
[imgur] add extractor for subreddit links ( closes #500 )
5 years ago
38c05df290
[oauth] add custom/default indicator to log messages ( #501 )
5 years ago
372ffe95ee
[oauth] adjust Flickr redirect URI ( fixes #503 )
...
Flickr now automatically forces https:// for all redirect URIs.
5 years ago
004812258d
[hentaifox] fix extraction
5 years ago
e2710702d4
fix Cloudflare bypss
5 years ago
8759403f37
[plurk] add delay between comment requests
5 years ago
a28552fd19
update test results
...
- hbrowse: one tag got removed
- mangoxo: gallery changed owner
- photobucket: ?, but photo still downloads
5 years ago
dcaa3d01bd
[imagefap] adapt to new image URL format
5 years ago
e62c209ca0
[nijie] fix 'date' parsing
5 years ago
3bba763ab9
[twitter] improve
...
- update metadata structure
- combine all user… entries into their own dict
- let 'user' always specify the Timeline owner
- add 'author' entry that specifies the original Tweet author
- create directories per post (closes #491 )
- fix username issues with /i/web/ URLs
5 years ago
26d2334550
[postprocessor:metadata] rename 'format' to 'content-format'
...
Just to be consistent with the other 'extension-format' option name,
and only 'format' is also still accepted.
5 years ago
a412531451
[postprocessor:metadata] implement 'extension-format' option
...
closes #477
5 years ago
0f1538af78
split filename formatting into its own function
5 years ago
db35c3b581
[directlink] separate filenames from paths
...
With this, all default filename formats specify an '{extension}'
and PathFormat.set_extension() reliably works for all files.
5 years ago
Mike Fährmann