Commit Graph

135 Commits

Author SHA1 Message Date
Slavi Pantaleev
ad082b3b1b Fix self-building for Synapse v1.51.0 (requires BuildKit)
Synapse v1.51.0 requires to be built with BuildKit since
https://github.com/matrix-org/synapse/pull/11691

The `docker_image` Ansible module does not support BuildKit
(https://github.com/ansible-collections/community.general/issues/514),
so we had to switch to a `docker build` call.
2022-01-26 08:41:34 +02:00
Aaron Raimist
ff94d815e1
Upgrade synapse-simple-antispam (0.0.3 -> 0.0.7) 2022-01-19 14:35:55 +00:00
Slavi Pantaleev
4e4fb98a65 Do not install fuse unless necessary
Discussed here: https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1510
2022-01-08 14:14:46 +02:00
Slavi Pantaleev
2bd33e5cf2 Make --tags=register-user work for both Dendrite/Synapse
Also get rid of `--tags=update-user-password` in the
`matrix-dendrite` role, as what we had doesn't work.

We may be able to do it with some Ansible helper or something else.
For now, we'll omit this feature.
2022-01-07 15:59:35 +02:00
Slavi Pantaleev
4625b34acc Fix failure trying to stop orphaned systemd services
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1461
2021-12-17 17:18:35 +02:00
Slavi Pantaleev
e1a6d1e4b2 Upgrade Synapse (1.46.0 -> 1.47.0)
We had to remove UID/GID environment variables that we used to pass
to the Synapse container, because it was causing a problem after
https://github.com/matrix-org/synapse/pull/11209

We were using both `--user` and UID/GID environment variables until now.
2021-11-17 17:21:15 +02:00
Slavi Pantaleev
735c966ab6 Disable systemd services when stopping to uninstall them
Until now, we were leaving services "enabled"
(symlinks in /etc/systemd/system/multi-user.target.wants/).

We clean these up now. Broken symlinks may still exist in older
installations that enabled/disabled services. We're not taking care
to fix these up. It's just a cosmetic defect anyway.
2021-11-10 17:39:21 +02:00
Neutron
1ad9bba88f Skip importing validate_config task when Synapse is disabled 2021-07-02 20:58:57 +03:00
Slavi Pantaleev
fa76c1ee5b Do not run self-build Ansible version-check, if component not enabled
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1108
2021-06-08 08:59:02 +03:00
pushytoxin
bee14550ab Fix local/bin scripts autocompletion by adding rx perms to everyone
It's mildly annoying when trying to execute these scripts while logged
in as a regular user, as the missing execute permissions will hinder
autocompletion even when trying to use with sudo.

These shell scripts don't contain secrets, but may fail when ran by a
regular user. The failure is due to the lack of access to the /matrix
directory, and does not result in any damage.
2021-05-28 10:39:27 +02:00
Slavi Pantaleev
d61fe94bae
Fix incorrect path in Mjolnir uninstallation tasks 2021-05-26 10:52:15 +03:00
BG
763952395b Adding mjolnir antispam synapse modul. 2021-05-25 16:43:30 +02:00
Michael-GMH
85777e8f96 merge with upstream 2021-05-25 21:08:00 +08:00
Michael-GMH
4e6f6e179b GMH 0.4.6 update 2021-05-10 18:50:10 +08:00
Slavi Pantaleev
e3fa3e12bc Upgrade Synapse (1.31 -> 1.32.2) 2021-04-22 14:22:07 +03:00
Slavi Pantaleev
378fabf177 Revert "Upgrade Synapse (1.31 -> 1.32.1)"
This reverts commit 1fb54a37cb.

Seems like it's been pulled or something. It used to exist, but not
anymore. Not sure what's going on.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1017

Related to
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1010
2021-04-21 23:36:58 +03:00
Slavi Pantaleev
1fb54a37cb Upgrade Synapse (1.31 -> 1.32.1)
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1010
2021-04-21 18:47:15 +03:00
Slavi Pantaleev
ca786cc343 Revert "Upgrade Synapse (1.31 -> 1.32)"
This reverts commit f825c7c263.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1010
2021-04-20 23:40:55 +03:00
Slavi Pantaleev
f825c7c263 Upgrade Synapse (1.31 -> 1.32) 2021-04-20 17:47:34 +03:00
Slavi Pantaleev
9dc87bb948 Add Synapse worker presets for easier configuration
Adding more presets in the future would be nice.
2021-02-19 11:38:47 +02:00
Slavi Pantaleev
d6c4d41c2b Define instanceId property on workers
This give us the possibility to run multiple instances of
workers that that don't expose a port.

Right now, we don't support that, but in the future we could
run multiple `federation_sender` or `pusher` workers, without
them fighting over naming (previously, they'd all be named
something like `matrix-synapse-worker-pusher-0`, because
they'd all define `port` as `0`).
2021-02-18 18:19:51 +02:00
Slavi Pantaleev
61e427d690 Do not let people enable more than 1 federation_sender worker 2021-02-15 11:37:03 +02:00
Slavi Pantaleev
85a05f38e8 Allow Synapse worker list to be generated dynamically
This leads to much easier management and potential safety
features (validation). In the future, we could try to avoid port
conflicts as well, but it didn't seem worth the effort to do it now.
Our port ranges seem large enough.

This can also pave the way for a "presets" feature
(similar to `matrix_nginx_proxy_ssl_presets`) which makes it even easier
for people to configure worker counts.
2021-02-15 11:25:35 +02:00
Slavi Pantaleev
453a4ec2d8 Relocate tasks related to Synapse workers 2021-02-15 11:18:47 +02:00
Slavi Pantaleev
5cfeae806b Merge branch 'master' into synapse-workers 2021-02-14 13:00:57 +02:00
Slavi Pantaleev
7e8e95a09a Make S3-mounting path configurable
This will make data migration easier.
2021-02-09 22:05:07 +02:00
Béla Becker
2edc9cb83c Name the Synapse database on state compression import
Fixes:
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/833
2021-01-28 17:54:02 +01:00
Slavi Pantaleev
a535226210 Stop/disable unnecessary worker services before deleting them 2021-01-25 15:20:37 +02:00
Slavi Pantaleev
70796703d3 Run Synapse workers in their own containers
This switches the `docker exec` method of spawning
Synapse workers inside the `matrix-synapse` container with
dedicated containers for each worker.

We also have dedicated systemd services for each worker,
so this are now:
- more consistent with everything else (we don't use systemd
instantiated services anywhere)
- we don't need the "parse systemd instance name into worker name +
port" part
- we don't need to keep track of PIDs manually
- we don't need jq (less depenendencies)
- workers dying would be restarted by systemd correctly, like any other
service
- `docker ps` shows each worker separately and we can observe resource
usage
2021-01-25 12:14:46 +02:00
Slavi Pantaleev
cc5cf0d725 Load roles/matrix-synapse/vars/workers.yml earlier to not break --tags=setup-nginx-proxy
If we load it at runtime, during matrix-synapse role execution,
it's good enough for matrix-synapse and all roles after that,
but.. it breaks when someone uses `--tags=setup-nginx-proxy` alone.

The downside of including this vars file like this in `setup.yml`
is that the variables contained in it cannot be overriden by the user
(in their inventory's `vars.yml`).
... but it's not like overriding these variables was possible anyway
when including them at runtime.
2021-01-24 20:19:55 +02:00
Marcel Partap
183adec3d8 Merge remote-tracking branch 'origin/master' into synapse-workers 2021-01-23 15:04:11 +01:00
Marcel Partap
c8f051a42d Track workers endpoint list in repo instead of regenerating on user side 2021-01-23 14:44:36 +01:00
Slavi Pantaleev
95346f3117 Reorganize Postgres access (breaking change)
In short, this makes Synapse a 2nd class citizen,
preparing for a future where it's just one-of-many homeserver software
options.

We also no longer have a default Postgres superuser password,
which improves security.

The changelog explains more as to why this was done
and how to proceed from here.
2021-01-22 13:26:12 +02:00
Slavi Pantaleev
05ca9357a8 Add .service suffix to systemd units list
We'll be adding `.timer` units later on, so it's good to be
more explicit.
2021-01-14 23:02:10 +02:00
Slavi Pantaleev
24100342e1 Tell people that federation_ip_range_blacklist is gone
Related to d5945c6e78
2021-01-13 13:47:51 +02:00
Marcel Partap
cd8100544b Merge remote-tracking branch 'origin/master' into synapse-workers
Sync with upstream
2021-01-08 20:58:50 +01:00
Slavi Pantaleev
6e1dfb62f0 Rename some doc files and commands related to importing
Since we'll likely have generic SQLite database importing
via [pgloader](https://pgloader.io/) for migrating bridge
databases from SQLite to Postgres, we'd rather avoid
calling the "import Synapse SQLite database" command
as just `--tags=import-sqlite-db`.

Similarly, for the media store, we'd like to mention that it's
related to Synapse as well.

We'd like to be more explicit, so as to be less confusing,
especially in light of other homeserver implementations
coming in the future.
2020-12-14 01:51:00 +02:00
Marcel Partap
414b812a29 synapse role workers setup: make configs clean action remote compatible
Many people probably didn't even know this - that ansible can be
quite a bit picky about what it will be willing to work with remotely.

Thanks @maxklenk !
2020-12-01 22:20:27 +01:00
Marcel Partap
d5932ca393 synapse role workers setup: execute the endpoint extraction locally
Thanks @maxklenk !
2020-12-01 22:18:42 +01:00
Marcel Partap
b73ac965ac Merge remote-tracking branch 'origin/master' into synapse-workers 2020-12-01 21:24:26 +01:00
Slavi Pantaleev
be5263f397 Move self-building git repository URLs to variables (stop hardcoding) 2020-11-28 21:34:14 +02:00
Slavi Pantaleev
75f9fde7a4 Remove some more -v usage
Continuation of 1fca917ad1.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/722
2020-11-25 10:49:59 +02:00
Slavi Pantaleev
1fca917ad1 Replace some -v instances with --mount
`-v` magically creates the source destination as a directory,
if it doesn't exist already. We'd like to avoid this magic
and the potential breakage that it might cause.

We'd rather fail while Docker tries to find things to `--mount`
than have it automatically create directories and fail anyway,
while having contaminated the filesystem.

There's a lot more `-v` instances remaining to be fixed later on.
This is just some start.

Things like `matrix_synapse_container_additional_volumes` and
`matrix_nginx_proxy_container_additional_volumes` were not changed to
use `--mount`, as options for each one are passed differently
(`ro` is `ro`, but `rw` doesn't exist and `slave` is `bind-propagation=slave`).
To avoid breaking people's custom volume mounts, we keep it as it is for now.

A deficiency with `--mount` is that it lacks the `z` option (SELinux
ownership changes), and some of our `-v` instances use that. I'm not
sure how supported SELinux is for us right now, but it might be,
and breaking that would not be a good idea.
2020-11-24 10:26:05 +02:00
Slavi Pantaleev
5eed874199 Improve self-building experience (avoid conflict with pullable images)
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/716

This patch makes us use more fully-qualified container image names
(either prefixed with docker.io/ or with localhost/).

The latter happens when self-building is enabled.

We've recently had issues where if an image was removed manually
and the service was restarted (making `docker run` fetch it from Docker Hub, etc.),
we'd end up with a pulled image, even though we're aiming for a self-built one.
Re-running the playbook would then not do a rebuild, because:
- the image with that name already exists (even though it's something
else)
- we sometimes had conditional logic where we'd build only if the git
repo changed

By explicitly changing the name of the images (prefixing with localhost/),
we avoid such confusion and the possibility that we'd automatically pul something
which is not what we expect.

Also, I've removed that condition where building would happen on git
changes only. We now always build (unless an image with that name
already exists). We just force-build when the git repo changes.
2020-11-14 23:00:49 +02:00
Marcel Partap
f3d2797d9c synapse workers: make awk script invocation handle paths with spaces
(quoting ftw)
2020-11-10 22:40:48 +01:00
Marcel Partap
cce90b187a synapse workers: fix undefined variable cases when removing workers 2020-10-28 23:09:21 +01:00
Marcel Partap
2d1b9f2dbf synapse workers: reworkings + get endpoints from upstream docs via awk
(yes, a bit awkward and brittle… xD)
2020-10-28 07:13:19 +01:00
Marcel Partap
a4125d5446 synapse workers: polishing, cleansing and installation of jq dependency 2020-10-23 20:49:53 +02:00
Marcel Partap
501efee07e synapse workers: supply systemd with actual worker PIDs (requires jq)
also, worker.yaml.j2:
  - hone worker_name
  - remove worker_pid_file entry (would only be used if worker_daemonize
    set to true; also, synapse only knows about the container namespace
    and thus can not provide the required host-view PID)
2020-10-22 20:53:41 +02:00
Max Klenk
567d0318b0
Merge branch 'synapse-workers' into feature/add-worker-support 2020-08-27 15:22:12 +02:00