18 Commits (c83c812a1e2c279fb14ec2020c7b616e7d9ff908)

Author	SHA1	Message	Date
Mike Fährmann	9c65db2a92	consistent 'with open(…) as fp:' syntax	3 months ago
Mike Fährmann	179d112083	[downloader] overhaul http and text modules ... Get rid of the modular structure and simplify/specialize those modules.	5 years ago
Mike Fährmann	b17a5d6f3b	give downloader classes proper names	6 years ago
Mike Fährmann	4a348990f4	adjust value resolution for retries/timeout/verify options ... This change introduces 'extractor.*.retries/timeout/verify' options as a general way to set these values for all HTTP requests. 'downloader.http.retries/timeout/verify' is a way to override these options for file downloads only and will fall back to 'extractor.*.…* values if they haven't been explicitly set. Also: downloader classes now take an extractor object as first argument instead of a requests.session.	6 years ago
Mike Fährmann	707b15b586	create missing directories for 'part-directory' ... also some code improvements regarding downloader config values	7 years ago
Mike Fährmann	caf26412dd	add option to set alternate location of .part files (#29) ... Note: The path set for 'downloader.*.part-directory' needs to point to an already existing directory.	7 years ago
Mike Fährmann	9a41002b77	fix partial downloads for 'text:' URLs ... Using a filesize in bytes as offset into a Python string is not a good idea if said file contains non-ASCII characters.	7 years ago
Mike Fährmann	963670d73b	add options to control usage of .part files (#29) ... - '--no-part' command line option to disable them - 'downloader.http.part' and 'downloader.text.part' config options Disabling .part files restores the behaviour of the old downloader implementation.	7 years ago
Mike Fährmann	b0353aa02d	rewrite download modules (#29) ... - use '.part' files during file-download - implement continuation of incomplete downloads - check if file size matches the one reported by server	7 years ago
Mike Fährmann	d70c66c516	fix "text:" downloader	7 years ago
Mike Fährmann	107d29ad8a	improve handling of text:... URLs ... - don't require // after the colon - open output files in text mode	7 years ago
Mike Fährmann	4f123b8513	code adjustments according to pep8	8 years ago
Mike Fährmann	3c1daef839	don't delete downloaded files in certain edge cases	8 years ago
Mike Fährmann	29692c5784	get extension from Content-Type header if not provided	8 years ago
Mike Fährmann	4b377ccc09	use output-module during downloads	9 years ago
Mike Fährmann	28fa7c53b4	docstrings and other small fixes for downloaders	10 years ago
Mike Fährmann	5545624da1	use seperate session in http downloader	10 years ago
Mike Fährmann	deef91eddc	initial commit	10 years ago