Add support for hashtags (TagPage-s), i.e. explore/tags/<tag> URLs.
This also introduce a get_metadata() method in order to append
possible further metadata per-(sub)extractor.
Refactor and generalize _extract_profilepage() to _extract_page()
in order to be reused by _extract_profilepage() and _extract_tagpage()
simply by passing the type of page (`ProfilePage' or `TagPage') and picking up
the respective fields in shared data.
* [instagram] Add support for GraphSidecar media types
Refactor _extract_postpage() to always return a list of medias.
Fetch common keywords and gracefully handle GraphSidecar media type
by extracting each single media and adding `sidecar_media_id' and
`sidecar_shortcode' keywords to indicate the parent of sidecar
childrens.
While here join the copyright comment lines in a single one.
Closes#178.
* [instagram] Use `yield from' instead of `for ... yield' (thanks @mikf)!
* [instagram] Adjust filename for GraphSidecar medias
Add a possible leading `media_id' of the sidecar for GraphSidecar
media.
Thanks to @mikf for the suggestion!
* [instagram] Add extra metadata for youtube-dl in GraphSidecar childrens
GraphSidecar children ytdl: URLs when consumed by youtube-dl
redirects to the URL of their parent. In GraphSidecar-s with
multiple GraphVideo-s this leads to downloading the same video
multiple times.
Add a `_ytdl_index' field to indicate the index of the youtube-dl
playlist corresponding the children of the sidecar.
This will be used by the `ytdl' downloader.
This patch adds a copy of the youtube-dl package to the snap to enable the video downloading feature.
Tested with the Twitter extractor.
Signed-off-by: 林博仁(Buo-ren Lin) <Buo.Ren.Lin@gmail.com>
This patch re-implements the pull step that will only build
development snapshots snaps if the latest tagged release has been
promoted to the stable channel. This ensures that there's always
a revision of the stable release snap available in the edge channel
for the publisher to promote to stable as currently the build
infrastructure only supports build on code push (but not new tagged
releases) at this time.
The implementation is inspired from the snapcrafters adopted snaps workflow,
documented in the following forum topic:
The automatic build and pubish process of snaps owned by the Snapcrafters - doc - snapcraft.io
https://forum.snapcraft.io/t/autopublishing-of-snapcrafters-organizations-snaps-how/7954/2
Refer the following forum topic for a possible solution proposal of this problem:
Proposal: Allow overriding the source-tag property for an one-time build in the build infrastructure - snapcraft - snapcraft.io
https://forum.snapcraft.io/t/proposal-allow-overriding-the-source-tag-property-for-an-one-time-build-in-the-build-infrastructure/10303
Signed-off-by: 林博仁(Buo-ren Lin) <Buo.Ren.Lin@gmail.com>
- use original image if available
- support video formats
- remove user info for ImageExtractor (it is no longer possible to get
image owner information for a single image)
An URL alone isn't good enough to distinguish between a gallery or a
gallery-listing, so the new extractor decides what to do based on the
page's content.
Mike Fährmann