[deviantart] also search journals for sta.sh links (#712)

when 'extra' is enabled
pull/866/head
Mike Fährmann 4 years ago
parent 0138e9c1d8
commit 41d03160ff
No known key found for this signature in database
GPG Key ID: 5680CA389D365A88

@ -576,7 +576,8 @@ extractor.deviantart.extra
=========== =====
Type ``bool``
Default ``false``
Description Download extra Sta.sh resources from description texts.
Description Download extra Sta.sh resources from
description texts and journals.
Note: Enabling this option also enables deviantart.metadata_.
=========== =====

@ -123,6 +123,10 @@ class DeviantartExtractor(Extractor):
journal = self.api.deviation_content(deviation["deviationid"])
yield self.commit_journal(deviation, journal)
if self.extra:
deviation["description"] = \
deviation.get("description", "") + journal["html"]
if self.extra:
for match in DeviantartStashExtractor.pattern.finditer(
deviation.get("description", "")):

Loading…
Cancel
Save