add prometheus-nginxlog-exporter role (#2315)

* add prometheus-nginxlog-exporter role

* Rename matrix_prometheus_nginxlog_exporter_container_url to matrix_prometheus_nginxlog_exporter_container_hostname

* avoid referencing variables from other roles, handover info using group_vars/matrix_servers

* fix: stop service when uninstalling

fix: typo

move available arch's into a var

fix: text

* fix: prometheus enabled condition

Co-authored-by: ikkemaniac <ikkemaniac@localhost>

docs/configuring-playbook-prometheus-grafana.md

@@ -15,6 +15,9 @@ matrix_prometheus_node_exporter_enabled: true
 # You can remove this, if unnecessary.
 matrix_prometheus_postgres_exporter_enabled: true
 
+# You can remove this, if unnecessary.
+matrix_prometheus_nginxlog_exporter_enabled: true
+
 matrix_grafana_enabled: true
 
 matrix_grafana_anonymous_access: false
@@ -39,6 +42,7 @@ Name | Description
 `matrix_prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about.
 `matrix_prometheus_node_exporter_enabled`|[Node Exporter](https://prometheus.io/docs/guides/node-exporter/) is an addon of sorts to Prometheus that collects generic system information such as CPU, memory, filesystem, and even system temperatures
 `matrix_prometheus_postgres_exporter_enabled`|[Postgres Exporter](configuring-playbook-prometheus-postgres.md) is an addon of sorts to expose Postgres database metrics to Prometheus.
+`matrix_prometheus_nginxlog_exporter_enabled`|[NGINX Log Exporter](configuring-playbook-prometheus-nginxlog.md) is an addon of sorts to expose NGINX logs to Prometheus.
 `matrix_grafana_enabled`|[Grafana](https://grafana.com/) is the visual component. It shows (on the `stats.<your-domain>` subdomain) the dashboards with the graphs that we're interested in
 `matrix_grafana_anonymous_access`|By default you need to log in to see graphs. If you want to publicly share your graphs (e.g. when asking for help in [`#synapse:matrix.org`](https://matrix.to/#/#synapse:matrix.org?via=matrix.org&via=privacytools.io&via=mozilla.org)) you'll want to enable this option.
 `matrix_grafana_default_admin_user`<br>`matrix_grafana_default_admin_password`|By default Grafana creates a user with `admin` as the username and password. If you feel this is insecure and you want to change it beforehand, you can do that here
@@ -73,6 +77,7 @@ Name | Description
 `matrix_prometheus_node_exporter_enabled`|Set this to `true` to enable the node (general system stats) exporter (locally, on the container network)
 `matrix_prometheus_node_exporter_metrics_proxying_enabled`|Set this to `true` to expose the node (general system stats) metrics on `https://matrix.DOMAIN/metrics/node-exporter` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
 `matrix_prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](configuring-playbook-prometheus-postgres.md) (locally, on the container network)
+`matrix_prometheus_nginxlog_exporter_enabled`|Set this to `true` to enable the [NGINX Log exporter](configuring-playbook-prometheus-nginxlog.md) (locally, on the container network)
 `matrix_prometheus_postgres_exporter_metrics_proxying_enabled`|Set this to `true` to expose the [Postgres exporter](configuring-playbook-prometheus-postgres.md) metrics on `https://matrix.DOMAIN/metrics/postgres-exporter` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
 `matrix_bridge_hookshot_metrics_enabled`|Set this to `true` to make [Hookshot](configuring-playbook-bridge-hookshot.md) expose metrics (locally, on the container network)
 `matrix_bridge_hookshot_metrics_proxying_enabled`|Set this to `true` to expose the [Hookshot](configuring-playbook-bridge-hookshot.md) metrics on `https://matrix.DOMAIN/metrics/hookshot` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)

docs/configuring-playbook-prometheus-nginxlog.md

@@ -0,0 +1,59 @@
+# Enabling metrics and graphs for NginX logs (optional)
+
+It can be useful to have some (visual) insight into NignX logs.
+
+This adds [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) to your matrix deployment.
+It will provide a prometheus 'metrics' endpoint exposing data from both the `matrix-nginx-proxy` and `matrix-synapse-reverse-proxy-companion` logs and automatically aggregates the data with prometheus.
+Optionally it visualizes the data, if [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) is enabled, by means of a dedicated Grafana dashboard named `NGINX PROXY`
+
+You can enable this role by adding the following settings in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
+
+```yaml
+matrix_prometheus_nginxlog_exporter_enabled: true
+
+# required depency
+matrix_prometheus_enabled: true
+
+# optional for visualization
+matrix_grafana_enabled: true
+```
+
+x | Prerequisites | Variable | Description
+|:--:|:--:|:--:|:--|
+**REQUIRED** | `matrix-prometheus`| `matrix_prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about.
+_Optional_ | [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) | [`matrix_grafana_enabled`](configuring-playbook-prometheus-grafana.md)|[Grafana](https://grafana.com) is the visual component. It shows (on the `stats.<your-domain>` subdomain) graphs that we're interested in. When enabled the `NGINX PROXY` dashboard is automatically added.
+
+## Docker Image Compatibility
+
+At the moment of writing only images for `amd64` and `arm64` architectures are available
+
+The playbook currently does not support building an image.
+You can however use a custom-build image by setting
+```yaml
+matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false
+matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag
+```
+
+## Security and privacy
+
+Metrics and resulting graphs can contain a lot of information. NginX logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access.
+Please make sure you change the default Grafana password.
+
+## Save metrics on an external Prometheus server
+
+The playbook will automatically integrate the metrics into the Prometheus server provided with this playbook. You can choose to save data on an external Prometheus instance.
+
+The metrics of this role will be exposed on `https://matrix.DOMAIN/metrics/nginxlog` when setting
+```yaml
+matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled: true
+
+# required dependency
+matrix_nginx_proxy_proxy_matrix_metrics_enabled: true
+```
+The playbook can provide a single endpoint (`https://matrix.DOMAIN/metrics/*`), under which various services may expose their metrics (e.g. `/metrics/node-exporter`, `/metrics/postgres-exporter`, `/metrics/nginxlog`, etc). To enable this `/metrics/*` feature, use `matrix_nginx_proxy_proxy_matrix_metrics_enabled`. To protect access using [Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication), see `matrix_nginx_proxy_proxy_matrix_metrics_basic_auth_enabled`.
+
+The following variables may be of interest:
+
+Name | Description
+-----|----------
+`matrix_nginx_proxy_proxy_matrix_metrics_enabled`|Set this to `true` to enable metrics exposure for various services on `https://matrix.DOMAIN/metrics/*`. Refer to the individual `matrix_SERVICE_metrics_proxying_enabled` variables below for exposing metrics for each individual service.

group_vars/matrix_servers

@@ -297,6 +297,8 @@ devture_systemd_service_manager_services_list_auto: |
     +
     ([{'name': 'matrix-prometheus-postgres-exporter.service', 'priority': 3900, 'groups': ['matrix', 'monitoring', 'prometheus', 'prometheus-node-exporters']}] if matrix_prometheus_postgres_exporter_enabled else [])
     +
+    ([{'name': (matrix_prometheus_nginxlog_exporter_container_hostname + '.service'), 'priority': 3900, 'groups': ['matrix', 'monitoring', 'prometheus', 'prometheus-node-exporters']}] if matrix_prometheus_postgres_exporter_enabled else [])
+    +
     ([{'name': 'matrix-redis', 'priority': 750, 'groups': ['matrix', 'redis']}] if matrix_redis_enabled else [])
     +
     ([{'name': 'matrix-registration.service', 'priority': 4000, 'groups': ['matrix', 'registration']}] if matrix_registration_enabled else [])
@@ -2235,6 +2237,9 @@ matrix_ssl_architecture: "{{
 
 matrix_ssl_pre_obtaining_required_service_name: "{{ 'matrix-dynamic-dns' if matrix_dynamic_dns_enabled else '' }}"
 
+matrix_nginx_proxy_access_log_syslog_integration_enabled: "{{ matrix_prometheus_nginxlog_exporter_enabled }}"
+matrix_nginx_proxy_access_log_syslog_integration_server_port: "{{ (matrix_prometheus_nginxlog_exporter_container_hostname | string +':'+ matrix_prometheus_nginxlog_exporter_container_syslog_port | string) | default('') }}"
+
 ######################################################################
 #
 # /matrix-nginx-proxy
@@ -2820,6 +2825,9 @@ matrix_synapse_reverse_proxy_companion_synapse_stream_writer_presence_stream_wor
 matrix_synapse_reverse_proxy_companion_synapse_media_repository_locations: "{{matrix_synapse_workers_media_repository_endpoints|default([]) }}"
 matrix_synapse_reverse_proxy_companion_synapse_user_dir_locations: "{{ matrix_synapse_workers_user_dir_worker_client_server_endpoints|default([]) }}"
 
+matrix_synapse_reverse_proxy_companion_access_log_syslog_integration_enabled: "{{ matrix_prometheus_nginxlog_exporter_enabled }}"
+matrix_synapse_reverse_proxy_companion_access_log_syslog_integration_server_port: "{{ (matrix_prometheus_nginxlog_exporter_container_hostname | string +':'+ matrix_prometheus_nginxlog_exporter_container_syslog_port | string) | default('') }}"
+
 ######################################################################
 #
 # /matrix-synapse-reverse-proxy-companion
@@ -2861,6 +2869,20 @@ matrix_prometheus_node_exporter_enabled: false
 #
 ######################################################################
 
+######################################################################
+#
+# matrix-prometheus-nginxlog-exporter
+#
+######################################################################
+
+matrix_prometheus_nginxlog_exporter_enabled: false
+
+######################################################################
+#
+# /matrix-prometheus-nginxlog-exporter
+#
+######################################################################
+
 ######################################################################
 #
 # matrix-prometheus
@@ -2888,6 +2910,10 @@ matrix_prometheus_scraper_postgres_targets: "{{ ['matrix-prometheus-postgres-exp
 matrix_prometheus_scraper_hookshot_enabled: "{{ matrix_hookshot_metrics_enabled|default(false) }}"
 matrix_prometheus_scraper_hookshot_targets: "{{ [matrix_hookshot_container_url | string +':'+ matrix_hookshot_metrics_port | string] if matrix_hookshot_metrics_enabled else [] }}"
 
+matrix_prometheus_scraper_nginxlog_enabled: "{{ matrix_prometheus_nginxlog_exporter_enabled }}"
+matrix_prometheus_scraper_nginxlog_server_port: "{{ (matrix_prometheus_nginxlog_exporter_container_hostname | string +':'+ matrix_prometheus_nginxlog_exporter_container_syslog_port | string)
+| default('') }}"
+
 ######################################################################
 #
 # /matrix-prometheus
@@ -2937,6 +2963,8 @@ matrix_grafana_dashboard_download_urls: |
     (matrix_prometheus_node_exporter_dashboard_urls if matrix_prometheus_node_exporter_enabled else [])
     +
     (matrix_prometheus_postgres_exporter_dashboard_urls if matrix_prometheus_postgres_exporter_enabled else [])
+    +
+    (matrix_prometheus_nginxlog_exporter_dashboard_urls if matrix_prometheus_nginxlog_exporter_enabled else [])
   }}
 
 matrix_grafana_default_home_dashboard_path: |-

playbooks/matrix.yml

@@ -80,6 +80,7 @@
     - custom/matrix-synapse-admin
     - custom/matrix-prometheus-node-exporter
     - custom/matrix-prometheus-postgres-exporter
+    - custom/matrix-prometheus-nginxlog-exporter
     - custom/matrix-prometheus
     - custom/matrix-grafana
     - custom/matrix-registration

roles/custom/matrix-nginx-proxy/templates/nginx/nginx.conf.j2

@@ -39,7 +39,18 @@ http {
 
 	{% if matrix_nginx_proxy_access_log_enabled %}
 	access_log /var/log/nginx/access.log main;
-	{% else %}
+	{% endif %}
+
+	{% if matrix_nginx_proxy_access_log_syslog_integration_enabled %}
+	log_format prometheus_fmt 'matrix-nginx-proxy $server_name - $upstream_addr - $remote_addr - $remote_user [$time_local] '
+								'$host "$request" '
+								'$status "$http_referer" '
+								'"$http_user_agent" "$http_x_forwarded_for"';
+
+	access_log syslog:server={{ matrix_nginx_proxy_access_log_syslog_integration_server_port }},tag=matrix_nginx_proxy prometheus_fmt;
+	{% endif %}
+
+	{% if not matrix_nginx_proxy_access_log_enabled and not matrix_nginx_proxy_access_log_syslog_integration_enabled %}
 	access_log off;
 	{% endif %}
 

roles/custom/matrix-prometheus-nginxlog-exporter/defaults/main.yml

@@ -0,0 +1,49 @@
+---
+# prometheus-nginxlog-exporter exports nginx logs in a prometheus usable format on a `/metrics/ endpoint
+# See: https://github.com/martin-helmich/prometheus-nginxlog-exporter/
+
+matrix_prometheus_nginxlog_exporter_enabled: true
+matrix_prometheus_nginxlog_exporter_version: v1.10.0
+
+matrix_prometheus_nginxlog_exporter_container_hostname: 'matrix-prometheus-nginxlog-exporter'
+matrix_prometheus_nginxlog_exporter_container_metrics_port: '4040'
+matrix_prometheus_nginxlog_exporter_container_syslog_port: '6514'
+
+matrix_prometheus_nginxlog_exporter_docker_image_arch: ['amd64', 'arm64']
+matrix_prometheus_nginxlog_exporter_docker_image_name_prefix: "{{ 'ghcr.io/martin-helmich/' }}"
+matrix_prometheus_nginxlog_exporter_docker_image_tag: "{{ 'latest' if matrix_prometheus_nginxlog_exporter_version == 'master' else matrix_prometheus_nginxlog_exporter_version }}-{{ matrix_architecture }}"
+matrix_prometheus_nginxlog_exporter_docker_image: "{{ matrix_prometheus_nginxlog_exporter_docker_image_name_prefix }}prometheus-nginxlog-exporter/exporter:{{ matrix_prometheus_nginxlog_exporter_docker_image_tag }}"
+matrix_prometheus_nginxlog_exporter_docker_image_force_pull: "{{ matrix_prometheus_nginxlog_exporter_docker_image.endswith(':latest') }}"
+matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: true
+
+# Controls whether prometheus-nginxlog-exporter metrics should be proxied (exposed) on `matrix.DOMAIN/metrics/nginxlog`.
+# This will only take effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`.
+# See the `matrix-nginx-proxy` role for details about enabling `matrix_nginx_proxy_proxy_matrix_metrics_enabled`.
+matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled: false
+
+# matrix_prometheus_nginxlog_exporter_dashboard_urls contains a list of URLs with Grafana dashboard definitions.
+# If the Grafana role is enabled, these dashboards will be downloaded.
+matrix_prometheus_nginxlog_exporter_dashboard_urls: []
+
+matrix_prometheus_nginxlog_exporter_base_path: "{{ matrix_base_data_path }}/prometheus-nginxlog-exporter"
+matrix_prometheus_nginxlog_exporter_config_path: "{{ matrix_prometheus_nginxlog_exporter_base_path }}/config"
+
+# A list of extra docker arguments to pass to the container
+matrix_prometheus_nginxlog_exporter_container_extra_arguments: []
+
+# List of systemd services that matrix-prometheus_nginxlog_exporter.service depends on
+matrix_prometheus_nginxlog_exporter_systemd_required_services_list: ['docker.service']
+
+# List of systemd services that matrix-prometheus_nginxlog_exporter.service wants
+matrix_prometheus_nginxlog_exporter_systemd_wanted_services_list: []
+
+# Controls whether the matrix-prometheus_nginxlog_exporter container exposes its HTTP port (tcp/4040 in the container).
+#
+# Takes an "<ip>:<port>" or "<port>" value (e.g. "127.0.0.1:4040"), or empty string to not expose.
+matrix_prometheus_nginxlog_exporter_container_http_host_bind_port: ''
+
+# Controls whether the matrix-prometheus_nginxlog_exporter container exposes its SYSLOG port (udp/6514 in the container).
+#
+# Takes an "<ip>:<port>" or "<port>" value (e.g. "127.0.0.1:6514"), or empty string to not expose.
+# NOTE: is an UDP connection!
+matrix_prometheus_nginxlog_exporter_container_syslog_host_bind_port: ''

roles/custom/matrix-prometheus-nginxlog-exporter/tasks/inject_into_nginx_proxy.yml

@@ -0,0 +1,31 @@
+---
+
+- name: Fail if matrix-nginx-proxy role already executed
+  ansible.builtin.fail:
+    msg: >-
+      Trying to append prometheus-nginxlog-exporter's configuration to matrix-nginx-proxy,
+      but it's pointless since the matrix-nginx-proxy role is already executed.
+      To fix this, please change the order of roles in your playbook to make sure that
+      the matrix-nginx-proxy role is run after the prometheus-nginxlog-exporter role.
+  when: matrix_nginx_proxy_role_executed | default(False) | bool
+
+- name: Generate prometheus-nginxlog-exporter's metrics proxying configuration for matrix-nginx-proxy (matrix.DOMAIN/metrics/nginxlog)
+  ansible.builtin.set_fact:
+    matrix_prometheus_nginxlog_exporter_matrix_nginx_proxy_metrics_configuration_matrix_domain: |
+      location /metrics/nginxlog {
+          {# Use the embedded DNS resolver in Docker containers to discover the service #}
+          resolver 127.0.0.11 valid=5s;
+          set $backend "{{ matrix_prometheus_nginxlog_exporter_container_hostname }}:{{ matrix_prometheus_nginxlog_exporter_container_metrics_port }}";
+          proxy_pass http://$backend/metrics;
+      }
+  when: matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled | bool and matrix_nginx_proxy_enabled | bool
+
+- name: Register prometheus-nginxlog-exporter metrics proxying configuration with matrix-nginx-proxy (matrix.DOMAIN/metrics/nginxlog)
+  ansible.builtin.set_fact:
+    matrix_nginx_proxy_proxy_matrix_metrics_additional_system_location_configuration_blocks: |
+      {{
+        matrix_nginx_proxy_proxy_matrix_metrics_additional_system_location_configuration_blocks | default([])
+        +
+        [matrix_prometheus_nginxlog_exporter_matrix_nginx_proxy_metrics_configuration_matrix_domain]
+      }}
+  when: matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled | bool

roles/custom/matrix-prometheus-nginxlog-exporter/tasks/main.yml

@@ -0,0 +1,29 @@
+---
+
+- block:
+    - when: matrix_prometheus_nginxlog_exporter_enabled | bool
+      ansible.builtin.include_tasks: "{{ role_path }}/tasks/inject_into_nginx_proxy.yml"
+  tags:
+    - setup-all
+    - setup-prometheus-nginxlog-exporter
+    - install-all
+    - install-prometheus-nginxlog-exporter
+
+- block:
+    - when: matrix_prometheus_nginxlog_exporter_enabled | bool
+      ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_config.yml"
+
+    - when: matrix_prometheus_nginxlog_exporter_enabled | bool
+      ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_install.yml"
+  tags:
+    - setup-all
+    - setup-prometheus-nginxlog-exporter
+    - install-all
+    - install-prometheus-nginxlog-exporter
+
+- block:
+    - when: not matrix_prometheus_nginxlog_exporter_enabled | bool
+      ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_uninstall.yml"
+  tags:
+    - setup-all
+    - setup-prometheus-nginxlog-exporter

roles/custom/matrix-prometheus-nginxlog-exporter/tasks/setup_install.yml

@@ -0,0 +1,47 @@
+---
+
+- name: Ensure matrix-prometheus-nginxlog-exporter image is pulled
+  community.docker.docker_image:
+    name: "{{ matrix_prometheus_nginxlog_exporter_docker_image }}"
+    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
+    force_source: "{{ matrix_prometheus_nginxlog_exporter_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
+    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_prometheus_nginxlog_exporter_docker_image_force_pull }}"
+  register: result
+  retries: "{{ devture_playbook_help_container_retries_count }}"
+  delay: "{{ devture_playbook_help_container_retries_delay }}"
+  until: result is not failed
+
+- name: Ensure prometheus-nginxlog-exporter paths exists
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: directory
+    mode: 0750
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+  with_items:
+    - "{{ matrix_prometheus_nginxlog_exporter_base_path }}"
+    - "{{ matrix_prometheus_nginxlog_exporter_config_path }}"
+
+- name: Ensure prometheus-nginxlog-exporter.yaml present
+  ansible.builtin.template:
+    src: "{{ role_path }}/templates/prometheus-nginxlog-exporter.yaml.j2"
+    dest: "{{ matrix_prometheus_nginxlog_exporter_config_path }}/prometheus-nginxlog-exporter.yaml"
+    mode: 0644
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+
+- name: Ensure grafana dashboard is installed
+  ansible.builtin.template:
+    src: "{{ role_path }}/templates/nginx-proxy.json"
+    dest: "{{ matrix_grafana_config_path }}/dashboards/"
+    mode: 0644
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+  when: matrix_grafana_enabled
+
+- name: Ensure matrix-prometheus-nginxlog-exporter.service installed
+  ansible.builtin.template:
+    src: "{{ role_path }}/templates/systemd/matrix-prometheus-nginxlog-exporter.service.j2"
+    dest: "{{ devture_systemd_docker_base_systemd_path }}/matrix-prometheus-nginxlog-exporter.service"
+    mode: 0644
+  register: matrix_prometheus_nginxlog_exporter_systemd_service_result

roles/custom/matrix-prometheus-nginxlog-exporter/tasks/setup_uninstall.yml

@@ -0,0 +1,39 @@
+---
+
+- name: Check existence of matrix-prometheus-nginxlog-exporter service
+  ansible.builtin.stat:
+    path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-prometheus-nginxlog-exporter.service"
+  register: matrix_prometheus_nginxlog_exporter_service_stat
+
+- when: matrix_prometheus_nginxlog_exporter_service_stat.stat.exists | bool
+  block:
+    - name: Fail when not cleaning up nginx and prometheus configs
+      ansible.builtin.fail:
+        msg: >
+          This role has added to configs in 'matrix-nginx-proxy', 'matrix-synapse-reverse-proxy-companion' and 'matrix-prometheus'.
+          Running 'setup-synapse-reverse-proxy-companion' WILL NOT remove those settings from those roles.
+          Run the playbook again with the `setup-all` tag or all three 'setup-nginx-proxy,setup-synapse-reverse-proxy-companion,setup-prometheus' tags while
+          'matrix_prometheus_enabled: false' to rebuild their configs.
+      when: not ('setup-all' in ansible_run_tags or ('setup-nginx-proxy' in ansible_run_tags and 'setup-synapse-reverse-proxy-companion' in ansible_run_tags and 'setup-prometheus' in ansible_run_tags))
+
+    - name: Ensure matrix-prometheus-nginxlog-exporter is stopped
+      ansible.builtin.service:
+        name: matrix-prometheus-nginxlog-exporter
+        state: stopped
+        enabled: false
+        daemon_reload: true
+
+    - name: Ensure matrix-prometheus-nginxlog-exporter.service doesn't exist
+      ansible.builtin.file:
+        path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-prometheus-nginxlog-exporter.service"
+        state: absent
+
+    - name: Ensure matrix-prometheus-nginxlog-exporter paths don't exist
+      ansible.builtin.file:
+        path: "{{ matrix_prometheus_nginxlog_exporter_base_path }}"
+        state: absent
+
+    - name: Ensure matrix-prometheus-nginxlog-exporter grafana dashboard doesn't exist
+      ansible.builtin.file:
+        path: "{{ matrix_grafana_config_path }}/dashboards/nginx-proxy.json"
+        state: absent

roles/custom/matrix-prometheus-nginxlog-exporter/tasks/validate_config.yml

@@ -0,0 +1,32 @@
+---
+
+- name: Fail if Prometheus not enabled
+  ansible.builtin.fail:
+    msg: >
+      You need to set `matrix_prometheus_enabled: true` or configure an external Prometheus database
+      as this roles requires Prometheus for data storage.
+  when: not matrix_prometheus_enabled and not matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled
+
+- name: Fail if docker image not availble for arch
+  ansible.builtin.fail:
+    msg: >
+      'prometheus-nginxlog-exporter' docker image is not available for your arch '{{ matrix_architecture }}'.
+
+      We currently do not support building an image using this playbook.
+
+      You can use a custom-build image by setting
+          'matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false'
+          'matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag'
+
+      or disable 'prometheus-nginxlog-exporter' by setting
+          'matrix_prometheus_nginxlog_exporter: false'
+
+      in vars.yml
+  when: matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled and matrix_architecture not in matrix_prometheus_nginxlog_exporter_docker_image_arch
+
+
+- name: Fail if nginx-proxy is not set to proxy metrics while prometheus-nginxlog-exporter is
+  ansible.builtin.fail:
+    msg: >
+      'matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled' is set but 'matrix_nginx_proxy_proxy_matrix_metrics_enabled' is not
+  when: matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled | bool and not matrix_nginx_proxy_proxy_matrix_metrics_enabled | bool

roles/custom/matrix-prometheus-nginxlog-exporter/templates/prometheus-nginxlog-exporter.yaml.j2

@@ -0,0 +1,37 @@
+listen:
+  port: {{ matrix_prometheus_nginxlog_exporter_container_metrics_port }}
+  address: "0.0.0.0"
+  metrics_endpoint: "/metrics"
+
+namespaces:
+  - name: matrix
+    metrics_override:
+      preffix: "myprefix"
+    namespace_label: "namespace"
+    format: "$log_source $server_name - $upstream_addr - $remote_addr - $remote_user [$time_local] $host \"$request\" $status \"$http_referer\" \"$http_user_agent\" \"$http_x_forwarded_for\""
+    # enable to print to console
+    # print_log: true
+    source:
+      syslog:
+        listen_address: "udp://0.0.0.0:{{ matrix_prometheus_nginxlog_exporter_container_syslog_port }}"
+        # format options: "rfc3164" ,"rfc5424", "rfc6587", "auto"
+        format: auto
+        tags: ["matrix_nginx_proxy", "matrix_nginx_proxy_companion"]
+    relabel_configs:
+      - target_label: request_uri
+        from: request
+        split: 2
+        separator: ' '
+        matches:
+        - regexp: "^([^\\?]+)(.*)"
+          replacement: "$1"
+      - target_label: remote_addr
+        from: remote_addr
+      - target_label: server_name
+        from: server_name
+      - target_label: http_x_forwarded_for
+        from: http_x_forwarded_for
+      - target_label: request_host
+        from: host
+      - target_label: log_source
+        from: log_source

roles/custom/matrix-prometheus-nginxlog-exporter/templates/systemd/matrix-prometheus-nginxlog-exporter.service.j2

@@ -0,0 +1,46 @@
+#jinja2: lstrip_blocks: "True"
+[Unit]
+Description={{ matrix_prometheus_nginxlog_exporter_container_hostname }}
+{% for service in matrix_prometheus_nginxlog_exporter_systemd_required_services_list %}
+Requires={{ service }}
+After={{ service }}
+{% endfor %}
+{% for service in matrix_prometheus_nginxlog_exporter_systemd_wanted_services_list %}
+Wants={{ service }}
+{% endfor %}
+DefaultDependencies=no
+
+[Service]
+Type=simple
+Environment="HOME={{ devture_systemd_docker_base_systemd_unit_home_path }}"
+ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} kill {{ matrix_prometheus_nginxlog_exporter_container_hostname }} 2>/dev/null || true'
+ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm {{ matrix_prometheus_nginxlog_exporter_container_hostname }} 2>/dev/null || true'
+
+
+ExecStart={{ devture_systemd_docker_base_host_command_docker }} run --rm --name {{ matrix_prometheus_nginxlog_exporter_container_hostname }} \
+			--log-driver=none \
+			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+			--cap-drop=ALL \
+			--read-only \
+			--network={{ matrix_docker_network }} \
+			{% if matrix_prometheus_nginxlog_exporter_container_http_host_bind_port %}
+			-p {{ matrix_prometheus_nginxlog_exporter_container_http_host_bind_port }}:{{ matrix_prometheus_nginxlog_exporter_container_metrics_port }} \
+			{% endif %}
+			{% if matrix_prometheus_nginxlog_exporter_container_syslog_host_bind_port %}
+			-p {{ matrix_prometheus_nginxlog_exporter_container_syslog_host_bind_port }}:{{ matrix_prometheus_nginxlog_exporter_container_syslog_port }}/udp \
+			{% endif %}
+			-v {{ matrix_prometheus_nginxlog_exporter_config_path }}:/etc/prometheus-nginxlog-exporter:z \
+			{% for arg in matrix_prometheus_nginxlog_exporter_container_extra_arguments %}
+			{{ arg }} \
+			{% endfor %}
+			{{ matrix_prometheus_nginxlog_exporter_docker_image }} \
+			-config-file /etc/prometheus-nginxlog-exporter/prometheus-nginxlog-exporter.yaml
+
+ExecStop=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} kill {{ matrix_prometheus_nginxlog_exporter_container_hostname }} 2>/dev/null || true'
+ExecStop=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm {{ matrix_prometheus_nginxlog_exporter_container_hostname }} 2>/dev/null || true'
+Restart=always
+RestartSec=30
+SyslogIdentifier={{ matrix_prometheus_nginxlog_exporter_container_hostname }}
+
+[Install]
+WantedBy=multi-user.target

roles/custom/matrix-prometheus/templates/prometheus.yml.j2

@@ -64,3 +64,10 @@ scrape_configs:
     static_configs:
       - targets: {{ matrix_prometheus_scraper_hookshot_targets|to_json }}
   {% endif %}
+
+  {% if matrix_prometheus_scraper_nginxlog_enabled %}
+  - job_name: nginxlog
+    static_configs:
+      - targets:
+        - {{ matrix_prometheus_scraper_nginxlog_server_port}}
+  {% endif %}

roles/custom/matrix-synapse-reverse-proxy-companion/templates/nginx/nginx.conf.j2

@@ -39,7 +39,18 @@ http {
 
 	{% if matrix_synapse_reverse_proxy_companion_access_log_enabled %}
 	access_log /var/log/nginx/access.log main;
-	{% else %}
+	{% endif %}
+
+	{% if matrix_synapse_reverse_proxy_companion_access_log_syslog_integration_enabled %}
+	log_format prometheus_fmt 'matrix-synapse-reverse-proxy-companion $server_name - $upstream_addr - $remote_addr - $remote_user [$time_local] '
+								'$host "$request" '
+								'$status "$http_referer" '
+								'"$http_user_agent" "$http_x_forwarded_for"';
+
+	access_log syslog:server={{ matrix_synapse_reverse_proxy_companion_access_log_syslog_integration_server_port }},tag=matrix_nginx_proxy_companion prometheus_fmt;
+	{% endif %}
+
+	{% if not matrix_synapse_reverse_proxy_companion_access_log_enabled and not matrix_synapse_reverse_proxy_companion_access_log_syslog_integration_enabled %}
 	access_log off;
 	{% endif %}