Sean Quah
633c48eda4
Fix a CI flake ( #94 )
...
We need to consume the `Failure` in `test_error`, otherwise subsequent
tests will fail if the garbage collector runs during them.
Signed-off-by: Sean Quah <seanq@matrix.org>
2023-03-30 14:01:01 +01:00
Victor Freire
6f8b3821aa
Add support to read database configuration from homeserver.yaml ( #91 )
2023-03-14 15:38:51 +00:00
David Robertson
fa27fa1a92
Fix CI again ( #90 )
...
* Fix CI again
* Try invoking isort etc directly
* Don't bother nuking pyc files
* Remove usedevelop and commented-out 'find' invocation
2023-01-09 18:11:45 +00:00
David Robertson
2ec5ec5a79
Bump version to 1.2.0
2023-01-09 16:57:15 +00:00
Cody Neiman
c65f9e5ba5
Fix SSE-C extra_arg checks ( #87 )
...
Fixes https://github.com/matrix-org/synapse-s3-storage-provider/issues/86
2022-11-15 12:19:19 +00:00
reivilibre
52be2ef613
Merge pull request #84 from xangelix/sse-c
2022-11-08 12:55:08 +00:00
Olivier Wilkinson (reivilibre)
0f162da9ed
fixup! (Copy flake8 config from Synapse to dismiss spurious error)
2022-11-08 12:52:40 +00:00
Olivier Wilkinson (reivilibre)
966a0cf28a
(Copy flake8 config from Synapse to dismiss spurious error)
2022-11-08 12:44:50 +00:00
Cody Wyatt Neiman
139e44673c
Add extra_args comment and link to docs
2022-11-04 13:18:53 -04:00
Cody Wyatt Neiman
5957773cf7
Refactor eargs to extra_args
2022-11-04 13:17:00 -04:00
Cody Wyatt Neiman
ff28d0a02f
Remove nargs=? from --sse-customer-algo arg
2022-11-04 13:10:40 -04:00
Cody Wyatt Neiman
095f8ebe86
Add sse_customer_algo default note
2022-11-04 13:09:01 -04:00
Kat Gerasimova
e531d0afea
Add automation to move X-Needs-Info issues ( #85 )
2022-11-02 15:17:17 +00:00
Cody Wyatt Neiman
8383d2e346
Merge branch 'main' into sse-c
2022-10-30 17:57:17 -04:00
Cody Wyatt Neiman
f39ce16a97
Fix set whitespace
2022-10-30 16:58:08 -04:00
Cody Wyatt Neiman
d08c2dcf22
Resolve linting errors
2022-10-30 16:52:01 -04:00
David Robertson
40c6a5599b
Fix CI and bump actions dependencies to deal with GHA deprecations. ( #83 )
2022-10-25 12:37:48 +01:00
Cody Wyatt Neiman
1e2cec3e56
Fix s3_download_task args order
2022-10-24 21:53:28 -04:00
Cody Wyatt Neiman
c118826d14
Fix remaining eargs run_upload reference
2022-10-24 20:28:11 -04:00
Cody Wyatt Neiman
4ab0624023
Fix run_upload eargs
2022-10-24 20:00:23 -04:00
Cody Wyatt Neiman
3e3f059f6c
Match SSE-C docs closer to other optional configs
2022-10-24 14:58:26 -04:00
Cody Wyatt Neiman
93b41de6b4
Add S3 SSE-C support
...
https://docs.aws.amazon.com/AmazonS3/latest/userguide/ServerSideEncryptionCustomerKeys.html
2022-10-24 14:20:20 -04:00
David Robertson
b0fa9bb140
Bump minimum required version of PyYAML ( #81 )
2022-10-12 12:27:06 +01:00
David Robertson
6d7c60a04a
Correct name of secret in release job
2022-10-12 12:00:17 +01:00
David Robertson
3a312edebd
Bump version to 1.1.1
2022-10-12 11:55:09 +01:00
David Robertson
e762c859ac
Fix dependencies syntax ( #80 )
2022-10-12 11:51:55 +01:00
Erik Johnston
5d734dee80
Merge pull request #77 from kittykat/patch-1
...
Add issue automation for triage
2022-09-02 17:52:31 +02:00
Kat Gerasimova
5a70b9ac8f
Add issue automation for triage
...
Move new issues to https://github.com/orgs/matrix-org/projects/67 for triage
2022-09-02 10:18:45 +01:00
Michael Kaye
ffd3fa4773
Update to v1.0.1 of setup-matrix-synapse ( #73 )
...
Required to be compatible with recently released synapse versions v1.56.0 or higher.
2022-04-06 11:59:54 +01:00
Michael Kaye
10558c8edc
Integration test s3 storage provider against latest synapse release ( #72 )
...
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-03-25 18:20:02 +01:00
Alex Auvolat
8926b4e417
Add support for Synapse running on sqlite3 database in scripts/s3_media_upload ( #71 )
...
Signed-off-by: Alex Auvolat <alex@adnab.me>
2022-01-13 18:14:52 +00:00
David Robertson
f821cff55e
Make the release action upload to real PyPI ( #70 )
...
* Run `build` and `twine check` in regular CI
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-12-01 17:28:02 +00:00
David Robertson
acd45b4aec
Bump version to 1.1.1-test
...
I have already uploaded a v1.1 to testpypi (and proper pypi for that
matter) and cannot reupload it.
2021-11-24 13:23:30 +00:00
David Robertson
ab561b53bd
Correct CI's invocation of twine
2021-11-24 13:23:30 +00:00
David Robertson
af3d6c8ab7
Test uploading to PyPI on release ( #69 )
2021-11-23 15:13:39 +00:00
David Robertson
84858af0ca
Update version number and give setup.py some love ( #68 )
...
* Mark version as 1.1
The previous owner of the PyPI package updated to a version 1.1 two days
ago, so this ought to be bumped to 1.1 too. See https://pypi.org/project/synapse-s3-storage-provider/1.1/#description
* Extra options in setup.py
* Packaging instructions
2021-11-22 15:33:10 +00:00
Sean Quah
04d5fb7fe9
Configure @matrix-org/synapse-core to be the code owner for the repo ( #67 )
...
Signed-off-by: Sean Quah <seanq@element.io>
2021-10-25 14:30:03 +01:00
Brendan Abolivier
fd1e12385d
Use the right config section ( #62 )
2021-10-12 12:14:27 +02:00
Sean Quah
04e3d31b40
Add support for thumbnail offload to scripts/s3_media_upload ( #61 )
...
Media are now considered deleted only if both the original file and all
thumbnails have been deleted.
`cache.db`s built before this change may incorrectly count media as
deleted while their thumbnails still exist in the local cache. This can
be resolved by either:
a) deleting `cache.db` and running an `update` to crawl through the
entire local cache again. This may take an extremely long time for
large Synapse deployments.
b) uploading the contents of local_thumbnails/ and remote_thumbnail/
manually, then deleting the uploaded files. Note that a running
Synapse instance may write new thumbnails during the process.
If the S3 storage provider has been installed since the very start
and configured to store both local and remote media synchronously,
all thumbnails should already be in S3 and the upload step can be
skipped.
This commit changes the behavior of the `write` command. Previously,
`write` would only output undeleted file paths. Now the output contains
a mix of file and thumbnail directory paths, which may sometimes already
be deleted / not exist.
2021-09-15 10:18:26 +01:00
Sean
a5b15d644d
Misc improvements to scripts/s3_media_upload and fix CI ( #59 )
...
scripts/s3_media_upload:
* Mark `scripts/s3_media_upload` as executable
* Fix `scripts/s3_media_upload` shebang to respect virtual environments
* Format `scripts/s3_media_upload`
* Remove unused imports from `scripts/s3_media_upload`
* Include `scripts/s3_media_upload` in CI checks
* Refactor `s3_media_upload`'s `run_check_delete` to use `to_path` instead of duplicating code
CI:
* Fix branch names in CI config
2021-09-10 11:39:50 +01:00
rkfg
3c3fafd6a2
Support intelligent tiering in s3 upload script ( #52 )
2021-02-18 18:35:38 +00:00
Patrick Cloke
c0bc3ce95a
Switch CI to GitHub Actions. ( #53 )
2021-02-17 11:22:05 -05:00
Richard van der Hoff
236e0cddb8
Improve the efficiency of the S3 storage provider ( #50 )
...
there are a few separate things here, which I hope will mean that (a) we use less CPU, and (b) what CPU we do use gets traced to the requests that cause it rather than getting lost down the sofa.
2021-01-21 12:22:43 +00:00
Erik Johnston
887ee24d76
Merge pull request #48 from matrix-org/michaelkaye/allow_option_not_to_use_upload
...
Option to not show progressbar.
2021-01-18 10:36:31 +00:00
Erik Johnston
3c8ea6076f
Fix up formatting
...
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-01-18 10:08:42 +00:00
Erik Johnston
3d4cc94a0f
Merge pull request #49 from ShadowJonathan/master
...
Add S3 endpoint url
2020-11-19 17:40:00 +00:00
Jonathan de Jong
954e221396
add endpoint url
2020-11-07 14:12:09 +01:00
Michael Kaye
52f2a5bbef
Make writes to progress be in global scope.
2020-10-29 14:39:14 +00:00
Michael Kaye
03d21a79df
Option to not show progressbar.
2020-10-28 15:59:49 +00:00
Erik Johnston
d58c93d7ca
Merge pull request #47 from matrix-org/erikj/fallback_to_created_ts
...
Fix upload script to handle media that has never been accessed
2020-10-28 10:02:05 +00:00
