Add support for not taking over a server (no matrix-nginx-proxy) and disabling Riot

This commit is contained in:
Slavi Pantaleev 2017-09-12 12:41:44 +03:00
parent b3a8698734
commit 6962bfcc42
12 changed files with 178 additions and 24 deletions

View File

@ -16,10 +16,12 @@ Using this playbook, you can get the following services configured on your serve
- a [STUN/TURN server](https://github.com/coturn/coturn) for WebRTC audio/video calls - a [STUN/TURN server](https://github.com/coturn/coturn) for WebRTC audio/video calls
- a [Riot](https://riot.im/) web UI, which is configured to connect to your own Matrix Synapse server by default - (optional default) a [Riot](https://riot.im/) web UI, which is configured to connect to your own Matrix Synapse server by default
- free [Let's Encrypt](https://letsencrypt.org/) SSL certificate, which secures the connection to the Synapse server and the Riot web UI - free [Let's Encrypt](https://letsencrypt.org/) SSL certificate, which secures the connection to the Synapse server and the Riot web UI
- (optional default) an [nginx](http://nginx.org/) web server, listening on ports 80 and 443 - standing in front of all the other services. Using your own webserver [is possible](#using-your-own-webserver-instead-of-this-playbooks-nginx-proxy-optional)
Basically, this playbook aims to get you up-and-running with all the basic necessities around Matrix, without you having to do anything else. Basically, this playbook aims to get you up-and-running with all the basic necessities around Matrix, without you having to do anything else.
@ -33,13 +35,17 @@ This is similar to the [EMnify/matrix-synapse-auto-deploy](https://github.com/EM
- works on both **CentOS** (7.0+) and Debian-based distributions (**Debian** 9/Stretch+, **Ubuntu** 16.04+) - works on both **CentOS** (7.0+) and Debian-based distributions (**Debian** 9/Stretch+, **Ubuntu** 16.04+)
- this one keeps mostly everything in a single directory (`/matrix` by default) and **doesn't "contaminate" your server** with files all over the place
- this one **doesn't necessarily take over** ports 80 and 443. By default, it sets up nginx for you there, but you can disable that and configure your own webserver (proxy)
- this one **runs everything in Docker containers** (like [silviof/docker-matrix](https://hub.docker.com/r/silviof/docker-matrix/) and [silviof/matrix-riot-docker](https://hub.docker.com/r/silviof/matrix-riot-docker/)), so it's likely more predictable - this one **runs everything in Docker containers** (like [silviof/docker-matrix](https://hub.docker.com/r/silviof/docker-matrix/) and [silviof/matrix-riot-docker](https://hub.docker.com/r/silviof/matrix-riot-docker/)), so it's likely more predictable
- this one retrieves and automatically renews free [Let's Encrypt](https://letsencrypt.org/) **SSL certificates** for you - this one retrieves and automatically renews free [Let's Encrypt](https://letsencrypt.org/) **SSL certificates** for you
- this one optionally can store the `media_store` content repository files on [Amazon S3](https://aws.amazon.com/s3/) (but defaults to storing files on the server's filesystem) - this one optionally can store the `media_store` content repository files on [Amazon S3](https://aws.amazon.com/s3/) (but defaults to storing files on the server's filesystem)
- this one optionally allows you to use an external PostgreSQL server for Matrix Synapse's database (but defaults to running one in a container) - this one optionally **allows you to use an external PostgreSQL server** for Matrix Synapse's database (but defaults to running one in a container)
Special thanks goes to: Special thanks goes to:
@ -52,7 +58,7 @@ Special thanks goes to:
## Prerequisites ## Prerequisites
- **CentOS** (7.0+), **Debian** (9/Stretch+) or **Ubuntu** (16.04+) server with no services running on port 80/443 - **CentOS** (7.0+), **Debian** (9/Stretch+) or **Ubuntu** (16.04+) server. This playbook can take over your whole server or co-exist with other services that you have there.
- the [Ansible](http://ansible.com/) program, which is used to run this playbook and configures everything for you - the [Ansible](http://ansible.com/) program, which is used to run this playbook and configures everything for you
@ -158,6 +164,33 @@ The database (as specified in `matrix_postgres_db_name`) must exist and be acces
It must be empty or contain a valid Matrix Synapse database. If empty, Matrix Synapse would populate it the first time it runs. It must be empty or contain a valid Matrix Synapse database. If empty, Matrix Synapse would populate it the first time it runs.
## Using your own webserver, instead of this playbook's nginx proxy (optional)
By default, this playbook installs its own nginx webserver (in a Docker container) which listens on ports 80 and 443.
If that's alright, you can skip ahead.
If you don't want this playbook's nginx webserver to take over your server's 80/443 ports like that,
and you'd like to use your own webserver (be it nginx, Apache, Varnish Cache, etc.), you can.
All it takes is editing your configuration file (`inventory/matrix.<your-domain>/vars.yml`):
```
matrix_nginx_proxy_enabled: false
```
**Note**: even if you do this, in order [to install](#installing), this playbook still expects port 80 to be available. **Please manually stop your other webserver while installing**. You can start it back again afterwards.
**If your own webserver is nginx**, you can most likely directly use the config files installed by this playbook at: `/matrix/nginx-proxy/conf.d`. Just include them in your `nginx.conf` like this: `include /matrix/nginx-proxy/conf.d/*.conf;`
**If your own webserver is not nginx**, you can still take a look at the sample files in `/matrix/nginx-proxy/conf.d`, and:
- ensure you set up (separate) vhosts that proxy for both Riot (`localhost:8765`) and Matrix Synapse (`localhost:8008`)
- ensure that the `/.well-known/acme-challenge` location for each "port=80 vhost" is an alias to the `/matrix/ssl/run/acme-challenge` directory (for automated SSL renewal to work)
- ensure that you restart/reload your webserver once in a while, so that renewed SSL certificates would take effect (once a month should be enough)
## Installing ## Installing
Once you have your server and you have [configured your DNS records](#configuring-dns), you can proceed with installing. Once you have your server and you have [configured your DNS records](#configuring-dns), you can proceed with installing.
@ -226,12 +259,31 @@ You can do it via this Ansible playbook (make sure to edit the `<your-username>`
**Note**: `<your-username>` is just a plain username (like `john`), not your full `@<username>:<your-domain>` identifier. **Note**: `<your-username>` is just a plain username (like `john`), not your full `@<username>:<your-domain>` identifier.
## Uninstalling
**Note**: If you have some trouble with your installation configuration, you can just re-run the playbook and it will try to set things up again. You don't need to uninstall and install fresh.
However, if you've installed this on some server where you have other stuff you wish to preserve, and now want get rid of Matrix, it's enough to do these:
- ensure all Matrix services are stopped (`systemctl stop 'matrix*'`)
- delete the Matrix-related systemd .service files (`rm -f /etc/systemd/system/matrix*`) and reload systemd (`systemctl daemon-reload`)
- delete all Matrix-related cronjobs (`rm -f /etc/cron.d/matrix*'`)
- delete some helper scripts (`rm -f /usr/local/bin/matrix*`)
- delete some cached Docker images (or just delete them all: `docker rmi $(docker images -aq)`)
- uninstall Docker itself, if necessary
- delete the `/matrix` directory (`rm -rf /matrix`)
## Deficiencies ## Deficiencies
This Ansible playbook can be improved in the following ways: This Ansible playbook can be improved in the following ways:
- not expecting to run its own nginx server overtaking port 80/443, thus allowing own/custom proxying to be configured
- setting up automatic backups to one or more storage providers - setting up automatic backups to one or more storage providers
- [importing an old SQLite database](#Restoring-an-existing-SQLite=database-from-another-installation) likely works because of a patch, but may be fragile until [this](https://github.com/matrix-org/synapse/issues/2287) is fixed - [importing an old SQLite database](#Restoring-an-existing-SQLite=database-from-another-installation) likely works because of a patch, but may be fragile until [this](https://github.com/matrix-org/synapse/issues/2287) is fixed

View File

@ -61,3 +61,14 @@ matrix_s3_media_store_enabled: false
matrix_s3_media_store_bucket_name: "your-bucket-name" matrix_s3_media_store_bucket_name: "your-bucket-name"
matrix_s3_media_store_aws_access_key: "your-aws-access-key" matrix_s3_media_store_aws_access_key: "your-aws-access-key"
matrix_s3_media_store_aws_secret_key: "your-aws-secret-key" matrix_s3_media_store_aws_secret_key: "your-aws-secret-key"
# By default, this playbook installs the Riot.IM web UI on the `hostname_riot` domain.
# If you wish to connect to your Matrix server by other means,
# you may wish to disable this.
matrix_riot_web_enabled: true
# By default, this playbook sets up its own nginx proxy server on port 80/443.
# This is fine if you're dedicating the whole server to Matrix.
# But in case that's not the case, you may wish to prevent that
# and take care of proxying by yourself.
matrix_nginx_proxy_enabled: true

View File

@ -1,5 +1,15 @@
--- ---
#
# Generic tasks that we always want to happen, regardless
# if the user wants matrix-nginx-proxy or not.
#
# If the user would set up their own nginx proxy server,
# the config files from matrix-nginx-proxy can be reused.
#
# It doesn't hurt to put them in place, even if they turn out
# to be unnecessary.
#
- name: Ensure Matrix nginx-proxy paths exists - name: Ensure Matrix nginx-proxy paths exists
file: file:
path: "{{ item }}" path: "{{ item }}"
@ -11,10 +21,6 @@
- "{{ matrix_nginx_proxy_data_path }}" - "{{ matrix_nginx_proxy_data_path }}"
- "{{ matrix_nginx_proxy_confd_path }}" - "{{ matrix_nginx_proxy_confd_path }}"
- name: Ensure nginx Docker image is pulled
docker_image:
name: "{{ docker_nginx_image }}"
- name: Ensure Matrix Synapse proxy vhost configured - name: Ensure Matrix Synapse proxy vhost configured
template: template:
src: "{{ role_path }}/templates/nginx-conf.d/{{ item }}.j2" src: "{{ role_path }}/templates/nginx-conf.d/{{ item }}.j2"
@ -24,6 +30,14 @@
- "matrix-synapse.conf" - "matrix-synapse.conf"
- "matrix-riot-web.conf" - "matrix-riot-web.conf"
#
# Tasks related to setting up matrix-nginx-proxy
#
- name: Ensure nginx Docker image is pulled
docker_image:
name: "{{ docker_nginx_image }}"
when: matrix_nginx_proxy_enabled
- name: Allow access to nginx proxy ports in firewalld - name: Allow access to nginx proxy ports in firewalld
firewalld: firewalld:
service: "{{ item }}" service: "{{ item }}"
@ -33,10 +47,30 @@
with_items: with_items:
- "http" - "http"
- "https" - "https"
when: ansible_os_family == 'RedHat' when: "ansible_os_family == 'RedHat' and matrix_nginx_proxy_enabled"
- name: Ensure matrix-nginx-proxy.service installed - name: Ensure matrix-nginx-proxy.service installed
template: template:
src: "{{ role_path }}/templates/systemd/matrix-nginx-proxy.service.j2" src: "{{ role_path }}/templates/systemd/matrix-nginx-proxy.service.j2"
dest: "/etc/systemd/system/matrix-nginx-proxy.service" dest: "/etc/systemd/system/matrix-nginx-proxy.service"
mode: 0644 mode: 0644
when: matrix_nginx_proxy_enabled
#
# Tasks related to getting rid of matrix-nginx-proxy (if it was previously enabled)
#
- name: Check existence of matrix-nginx-proxy service
stat: path="/etc/systemd/system/matrix-nginx-proxy.service"
register: matrix_nginx_proxy_service_stat
- name: Ensure matrix-nginx-proxy is stopped
service: name=matrix-nginx-proxy state=stopped daemon_reload=yes
register: stopping_result
when: "not matrix_nginx_proxy_enabled and matrix_nginx_proxy_service_stat.stat.exists"
- name: Ensure matrix-nginx-proxy.service doesn't exist
file:
path: "/etc/systemd/system/matrix-nginx-proxy.service"
state: absent
when: "not matrix_nginx_proxy_enabled and matrix_nginx_proxy_service_stat.stat.exists"

View File

@ -1,5 +1,9 @@
--- ---
#
# Tasks related to setting up riot-web
#
- name: Ensure Matrix riot-web paths exists - name: Ensure Matrix riot-web paths exists
file: file:
path: "{{ matrix_nginx_riot_web_data_path }}" path: "{{ matrix_nginx_riot_web_data_path }}"
@ -7,10 +11,12 @@
mode: 0750 mode: 0750
owner: "{{ matrix_user_username }}" owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_username }}" group: "{{ matrix_user_username }}"
when: matrix_riot_web_enabled
- name: Ensure riot-web Docker image is pulled - name: Ensure riot-web Docker image is pulled
docker_image: docker_image:
name: "{{ docker_riot_image }}" name: "{{ docker_riot_image }}"
when: matrix_riot_web_enabled
- name: Ensure Matrix riot-web configured - name: Ensure Matrix riot-web configured
template: template:
@ -22,9 +28,42 @@
with_items: with_items:
- "riot.im.conf" - "riot.im.conf"
- "config.json" - "config.json"
when: matrix_riot_web_enabled
- name: Ensure matrix-riot-web.service installed - name: Ensure matrix-riot-web.service installed
template: template:
src: "{{ role_path }}/templates/systemd/matrix-riot-web.service.j2" src: "{{ role_path }}/templates/systemd/matrix-riot-web.service.j2"
dest: "/etc/systemd/system/matrix-riot-web.service" dest: "/etc/systemd/system/matrix-riot-web.service"
mode: 0644 mode: 0644
when: matrix_riot_web_enabled
#
# Tasks related to getting rid of riot-web (if it was previously enabled)
#
- name: Check existence of matrix-riot-web service
stat: path="/etc/systemd/system/matrix-riot-web.service"
register: matrix_riot_web_service_stat
- name: Ensure matrix-riot-web is stopped
service: name=matrix-riot-web state=stopped daemon_reload=yes
register: stopping_result
when: "not matrix_riot_web_enabled and matrix_riot_web_service_stat.stat.exists"
- name: Ensure matrix-riot-web.service doesn't exist
file:
path: "/etc/systemd/system/matrix-riot-web.service"
state: absent
when: "not matrix_riot_web_enabled and matrix_riot_web_service_stat.stat.exists"
- name: Ensure Matrix riot-web paths doesn't exist
file:
path: "{{ matrix_nginx_riot_web_data_path }}"
state: absent
when: "not matrix_riot_web_enabled"
- name: Ensure riot-web Docker image doesn't exist
docker_image:
name: "{{ docker_riot_image }}"
state: absent
when: "not matrix_riot_web_enabled"

View File

@ -1,5 +1,14 @@
--- ---
- name: Determine domains to obtain certificates for (Matrix)
set_fact:
domains_to_obtain_certificate_for: "['{{ hostname_matrix }}']"
- name: Determine domains to obtain certificates for (Riot)
set_fact:
domains_to_obtain_certificate_for: "{{ domains_to_obtain_certificate_for + [hostname_riot] }}"
when: matrix_riot_web_enabled
- name: Allow access to HTTP/HTTPS in firewalld - name: Allow access to HTTP/HTTPS in firewalld
firewalld: firewalld:
service: "{{ item }}" service: "{{ item }}"
@ -38,9 +47,7 @@
-e ACME_EMAIL={{ matrix_ssl_support_email }} -e ACME_EMAIL={{ matrix_ssl_support_email }}
willwill/acme-docker willwill/acme-docker
acmetool want {{ item }} --xlog.severity=debug acmetool want {{ item }} --xlog.severity=debug
with_items: with_items: "{{ domains_to_obtain_certificate_for }}"
- "{{ hostname_matrix }}"
- "{{ hostname_riot }}"
- name: Ensure matrix-nginx-proxy is started (if previously installed & started) - name: Ensure matrix-nginx-proxy is started (if previously installed & started)
service: name=matrix-nginx-proxy state=started service: name=matrix-nginx-proxy state=started

View File

@ -13,6 +13,8 @@
- name: Ensure matrix-riot-web autoruns and is restarted - name: Ensure matrix-riot-web autoruns and is restarted
service: name=matrix-riot-web enabled=yes state=restarted daemon_reload=yes service: name=matrix-riot-web enabled=yes state=restarted daemon_reload=yes
when: matrix_riot_web_enabled
- name: Ensure matrix-nginx-proxy autoruns and is restarted - name: Ensure matrix-nginx-proxy autoruns and is restarted
service: name=matrix-nginx-proxy enabled=yes state=restarted daemon_reload=yes service: name=matrix-nginx-proxy enabled=yes state=restarted daemon_reload=yes
when: matrix_nginx_proxy_enabled

View File

@ -11,11 +11,14 @@ MAILTO="{{ matrix_ssl_support_email }}"
# #
# How renewal works? # How renewal works?
# #
# acmetool will fail to bind to port :80 (because matrix-nginx-proxy is running there), # acmetool will fail to bind to port :80 (because matrix-nginx-proxy or some other server is running there),
# and will fall back to its "webroot" validation method. # and will fall back to its "webroot" validation method.
# #
# Thus, it would put validation files in `/var/run/acme/acme-challenge`. # Thus, it would put validation files in `/var/run/acme/acme-challenge`.
# These files can be retrieved via any vhost on port 80 of matrix-nginx-proxy, # These files can be retrieved via any vhost on port 80 of matrix-nginx-proxy,
# because it aliases `/.well-known/acme-challenge` to that same directory. # because it aliases `/.well-known/acme-challenge` to that same directory.
#
# When a custom proxy server (not matrix-nginx-proxy provided by this playbook),
# you'd need to make sure you alias these files corretly or SSL renewal would not work.
15 4 */5 * * root /usr/bin/docker run --rm --name acmetool-host-grab --net=host -v {{ matrix_ssl_certs_path }}:/certs -v {{ matrix_ssl_certs_path }}/run:/var/run/acme -e ACME_EMAIL={{ matrix_ssl_support_email }} willwill/acme-docker acmetool --batch reconcile # --xlog.severity=debug 15 4 */5 * * root /usr/bin/docker run --rm --name acmetool-host-grab --net=host -v {{ matrix_ssl_certs_path }}:/certs -v {{ matrix_ssl_certs_path }}/run:/var/run/acme -e ACME_EMAIL={{ matrix_ssl_support_email }} willwill/acme-docker acmetool --batch reconcile # --xlog.severity=debug

View File

@ -6,7 +6,7 @@ server {
location /.well-known/acme-challenge { location /.well-known/acme-challenge {
default_type "text/plain"; default_type "text/plain";
alias /acmetool-certs/run/acme-challenge; alias {{ matrix_ssl_certs_path }}/run/acme-challenge;
} }
location / { location / {
@ -24,14 +24,14 @@ server {
root /dev/null; root /dev/null;
ssl on; ssl on;
ssl_certificate /acmetool-certs/live/{{ hostname_riot }}/fullchain; ssl_certificate {{ matrix_ssl_certs_path }}/live/{{ hostname_riot }}/fullchain;
ssl_certificate_key /acmetool-certs/live/{{ hostname_riot }}/privkey; ssl_certificate_key {{ matrix_ssl_certs_path }}/live/{{ hostname_riot }}/privkey;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on; ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
location / { location / {
proxy_pass http://riot:8765; proxy_pass http://{{ 'riot' if matrix_nginx_proxy_enabled else 'localhost' }}:8765;
proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-For $remote_addr;
} }
} }

View File

@ -6,7 +6,7 @@ server {
location /.well-known/acme-challenge { location /.well-known/acme-challenge {
default_type "text/plain"; default_type "text/plain";
alias /acmetool-certs/run/acme-challenge; alias {{ matrix_ssl_certs_path }}/run/acme-challenge;
} }
location / { location / {
@ -24,14 +24,14 @@ server {
root /dev/null; root /dev/null;
ssl on; ssl on;
ssl_certificate /acmetool-certs/live/{{ hostname_matrix }}/fullchain; ssl_certificate {{ matrix_ssl_certs_path }}/live/{{ hostname_matrix }}/fullchain;
ssl_certificate_key /acmetool-certs/live/{{ hostname_matrix }}/privkey; ssl_certificate_key {{ matrix_ssl_certs_path }}/live/{{ hostname_matrix }}/privkey;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on; ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
location /_matrix { location /_matrix {
proxy_pass http://synapse:8008; proxy_pass http://{{ 'synapse' if matrix_nginx_proxy_enabled else 'localhost' }}:8008;
proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-For $remote_addr;
client_body_buffer_size 25M; client_body_buffer_size 25M;

View File

@ -17,7 +17,7 @@ ExecStart=/usr/bin/docker run --rm --name matrix-nginx-proxy \
--link matrix-synapse:synapse \ --link matrix-synapse:synapse \
--link matrix-riot-web:riot \ --link matrix-riot-web:riot \
-v {{ matrix_nginx_proxy_confd_path }}:/etc/nginx/conf.d \ -v {{ matrix_nginx_proxy_confd_path }}:/etc/nginx/conf.d \
-v {{ matrix_ssl_certs_path }}:/acmetool-certs \ -v {{ matrix_ssl_certs_path }}:{{ matrix_ssl_certs_path }} \
{{ docker_nginx_image }} {{ docker_nginx_image }}
ExecStop=-/usr/bin/docker kill matrix-nginx-proxy ExecStop=-/usr/bin/docker kill matrix-nginx-proxy
ExecStop=-/usr/bin/docker rm matrix-nginx-proxy ExecStop=-/usr/bin/docker rm matrix-nginx-proxy

View File

@ -9,6 +9,9 @@ ExecStartPre=-/usr/bin/docker kill matrix-riot-web
ExecStartPre=-/usr/bin/docker rm matrix-riot-web ExecStartPre=-/usr/bin/docker rm matrix-riot-web
ExecStart=/usr/bin/docker run --rm --name matrix-riot-web \ ExecStart=/usr/bin/docker run --rm --name matrix-riot-web \
-v {{ matrix_nginx_riot_web_data_path }}:/data \ -v {{ matrix_nginx_riot_web_data_path }}:/data \
{% if not matrix_nginx_proxy_enabled %}
-p 127.0.0.1:8765:8765 \
{% endif %}
{{ docker_riot_image }} {{ docker_riot_image }}
ExecStop=-/usr/bin/docker kill matrix-riot-web ExecStop=-/usr/bin/docker kill matrix-riot-web
ExecStop=-/usr/bin/docker rm matrix-riot-web ExecStop=-/usr/bin/docker rm matrix-riot-web

View File

@ -21,6 +21,9 @@ ExecStart=/usr/bin/docker run --rm --name matrix-synapse \
--link matrix-postgres:{{ matrix_postgres_connection_hostname }} \ --link matrix-postgres:{{ matrix_postgres_connection_hostname }} \
{% endif %} {% endif %}
-p 8448:8448 \ -p 8448:8448 \
{% if not matrix_nginx_proxy_enabled %}
-p 127.0.0.1:8008:8008 \
{% endif %}
-p 3478:3478 \ -p 3478:3478 \
-p 3478:3478/udp \ -p 3478:3478/udp \
-p {{ matrix_coturn_turn_udp_min_port }}-{{ matrix_coturn_turn_udp_max_port }}:{{ matrix_coturn_turn_udp_min_port }}-{{ matrix_coturn_turn_udp_max_port }}/udp \ -p {{ matrix_coturn_turn_udp_min_port }}-{{ matrix_coturn_turn_udp_max_port }}:{{ matrix_coturn_turn_udp_min_port }}-{{ matrix_coturn_turn_udp_max_port }}/udp \