matrix-docker-ansible-deploy/docs/configuring-playbook-bot-baibot.md

Setting up baibot (optional)

baibot

🤖 baibot (pronounced bye-bot) is a Matrix bot developed by etke.cc that exposes the power of AI / Large Language Models to you. 🤖

It supports OpenAI's ChatGPT models, as many well as other ☁️ providers.

It's designed as a more private and ✨ featureful alternative to matrix-chatgpt-bot. See the baibot project and its documentation for more information.

Prerequisites

API access to one or more LLM ☁️ providers.

Adjusting the playbook configuration

There are a lot of configuration options (some required, some possibly required, some optional), so they're split into multiple sections below:

• Base configuration
• 👮‍♂️ Administrator configuration
• 👥 Initial users configuration
• 🤖 Configuring agents via Ansible
• 🤝 Configuring initial default handlers

Depending on your current vars.yml file and desired configuration, you may require more than just the base configuration.

Base configuration

Add the following configuration to your inventory/host_vars/matrix.example.com/vars.yml file:

matrix_bot_baibot_enabled: true

# Uncomment and adjust this part if you'd like to use a username different than the default
# matrix_bot_baibot_config_user_mxid_localpart: baibot

# Generate a strong password here. Consider generating it with `pwgen -s 64 1`.
# If you'd like to change this password subsequently, see the details below.
matrix_bot_baibot_config_user_password: 'PASSWORD_FOR_THE_BOT'

# An optional passphrase to use for backing up and recovering the bot's encryption keys.
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
#
# If set to null, the recovery module will not be used and losing your session/database
# will mean you lose access to old messages in encrypted room.
# It's highly recommended that you configure this to avoid losing access to encrypted messages.
#
# Changing this subsequently will also cause you to lose access to old messages in encrypted rooms.
# For details about changing this subsequently or resetting, see `defaults/main.yml` in the baibot role.
matrix_bot_baibot_config_user_encryption_recovery_passphrase: 'ANY_LONG_AND_SECURE_PASSPHRASE_STRING_HERE'

# An optional secret for encrypting the bot's session data (see `matrix_bot_baibot_data_path`).
# This must be 32-bytes (64 characters when HEX-encoded).
# Generate it with: `openssl rand -hex 32`
# Set to null or empty to avoid using encryption.
# Changing this subsequently requires that you also throw away all data (see `matrix_bot_baibot_data_path`)
matrix_bot_baibot_config_persistence_session_encryption_key: 'A_HEX_STRING_OF_64_CHARACTERS_HERE'

# An optional secret for encrypting bot configuration stored in Matrix's account data.
# This must be 32-bytes (64 characters when HEX-encoded).
# Generate it with: `openssl rand -hex 32`
# Set to null or empty to avoid using encryption.
# Changing this subsequently will make you lose your configuration.
matrix_bot_baibot_config_persistence_config_encryption_key: 'A_HEX_STRING_OF_64_CHARACTERS_HERE'

As mentioned above, this may not be enough. Continue with the configuration sections below.

👮‍♂️ Administrator configuration

This is an addition to the base configuration.

To specify who is considered a bot 👮‍♂️ Administrator, you either need to specify matrix_bot_baibot_config_access_admin_patterns or matrix_admin. The latter is a single variable which affects all bridges and bots.

If matrix_admin is already configured in your vars.yml configuration, you can skip this section.

If necessary, add the following configuration to your inventory/host_vars/matrix.example.com/vars.yml file:

# Uncomment to add one or more admins to this bridge:
#
# matrix_bot_baibot_config_access_admin_patterns:
#   - "@*:example.com"
#   - "@admin:example.net"
#
# .. unless you've made yourself an admin of all bots/bridges like this:
#
# matrix_admin: '@yourAdminAccount:{{ matrix_domain }}'

👥 Initial users configuration

By default, all users on your homeserver are considered allowed users. If that's OK, you can skip this section.

This is an addition to the base configuration.

To specify who is considered a bot 👥 User, you may:

• define an initial value for matrix_bot_baibot_config_initial_global_config_user_patterns Ansible variable, as shown below
• configure the list at runtime via the bot's !bai access set-users SPACE_SEPARATED_PATTERNS command

Configuring matrix_bot_baibot_config_initial_global_config_user_patterns is optional, but it can be useful to pre-configure the bot with a list of users who should have access to the bot's features.

Note: Once initially configured, the allowed users list cannot be managed via Ansible anymore. It can only be managed subsequently via bot commands.

If necessary, add the following configuration to your inventory/host_vars/matrix.example.com/vars.yml file:

# Uncomment and adjust the bot users if necessary:
#
# Subsequent changes to `matrix_bot_baibot_config_initial_global_config_user_patterns` do not affect the bot's behavior.
# Once initially configured, the allowed users list is managed via bot commands, not via Ansible.
#
# matrix_bot_baibot_config_initial_global_config_user_patterns:
#   - "@*:{{ matrix_bot_baibot_config_homeserver_server_name }}"

🤖 Configuring agents via Ansible

You are not required to define agents statically via Ansible. To get started quickly, you can skip this section and define agents at runtime via chat commands (following the bot's guidance).

Privileged users (like the 👮‍♂️ Administrator, but potentially others too - see the upstream 🔒 access documentation) can define agents dynamically at any time via chat commands.

The Ansible role includes preset variables for easily enabling some 🤖 agents on various ☁️ providers (e.g. OpenAI, etc).

Besides the presets, the Ansible role also includes support for configuring additional statically-defined agents via the matrix_bot_baibot_config_agents_static_definitions_custom Ansible variable.

Agents defined statically and those created dynamically (via chat) are named differently, so conflict cannot arise.

Depending on your propensity for GitOps, you may prefer to define agents statically via Ansible, or you may wish to do it dynamically via chat.

Before proceeding, we recommend reading the upstream documentation on How to choose a provider. In short, it's probably best to go with OpenAI.

Anthropic

You can statically-define a single 🤖 agent instance powered by the Anthropic provider with the help of the playbook's preset variables.

Here's an example addition to your vars.yml file:

matrix_bot_baibot_config_agents_static_definitions_anthropic_enabled: true

matrix_bot_baibot_config_agents_static_definitions_anthropic_config_api_key: "YOUR_API_KEY_HERE"

# If you'd like to use another text-generation agent, uncomment and adjust:
# matrix_bot_baibot_config_agents_static_definitions_anthropic_config_text_generation_model_id: claude-3-5-sonnet-20240620

# The playbook defines a default prompt for all statically-defined agents.
# You can adjust it in the `matrix_bot_baibot_config_agents_static_definitions_prompt` variable,
# or you can adjust it below only for the Anthropic agent.
# matrix_bot_baibot_config_agents_static_definitions_anthropic_config_text_generation_prompt: "{{ matrix_bot_baibot_config_agents_static_definitions_prompt }}"

# See `defaults/main.yml` in the baibot role for more configuration options.

If you'd like to use more than one model, take a look at the Configuring additional agents (without a preset) section below.

💡 You may also wish to use this new agent for 🤝 Configuring initial default handlers.

Groq

You can statically-define a single 🤖 agent instance powered by the Groq provider with the help of the playbook's preset variables.

Here's an example addition to your vars.yml file:

matrix_bot_baibot_config_agents_static_definitions_groq_enabled: true

matrix_bot_baibot_config_agents_static_definitions_groq_config_api_key: "YOUR_API_KEY_HERE"

# Specify the text-generation agent you'd like to use
matrix_bot_baibot_config_agents_static_definitions_groq_config_text_generation_model_id: "llama3-70b-8192"

# The playbook defines a default prompt for all statically-defined agents.
# You can adjust it in the `matrix_bot_baibot_config_agents_static_definitions_prompt` variable,
# or you can adjust it below only for the Groq agent.
# matrix_bot_baibot_config_agents_static_definitions_groq_config_text_generation_prompt: "{{ matrix_bot_baibot_config_agents_static_definitions_prompt }}"

# Uncomment and adjust this part if you're not happy with these speech-to-text defaults:
#
# matrix_bot_baibot_config_agents_static_definitions_groq_config_speech_to_text_enabled: true
# matrix_bot_baibot_config_agents_static_definitions_groq_config_speech_to_text_model_id: whisper-large-v3

# See `defaults/main.yml` in the baibot role for more configuration options.

Because this is a statically-defined agent, it will be given a static/ ID prefix and will be named static/groq.

If you'd like to use more than one model, take a look at the Configuring additional agents (without a preset) section below.

💡 You may also wish to use this new agent for 🤝 Configuring initial default handlers.

Mistral

You can statically-define a single 🤖 agent instance powered by the 🇫🇷 Mistral provider with the help of the playbook's preset variables.

Here's an example addition to your vars.yml file:

matrix_bot_baibot_config_agents_static_definitions_mistral_enabled: true

matrix_bot_baibot_config_agents_static_definitions_mistral_config_api_key: "YOUR_API_KEY_HERE"

# The playbook defines a default prompt for all statically-defined agents.
# You can adjust it in the `matrix_bot_baibot_config_agents_static_definitions_prompt` variable,
# or you can adjust it below only for the Mistral agent.
# matrix_bot_baibot_config_agents_static_definitions_mistral_config_text_generation_prompt: "{{ matrix_bot_baibot_config_agents_static_definitions_prompt }}"

# Uncomment and adjust this part if you're not happy with these defaults:
# matrix_bot_baibot_config_agents_static_definitions_mistral_config_text_generation_model_id: mistral-large-latest

# See `defaults/main.yml` in the baibot role for more configuration options.

Because this is a statically-defined agent, it will be given a static/ ID prefix and will be named static/mistral.

If you'd like to use more than one model, take a look at the Configuring additional agents (without a preset) section below.

💡 You may also wish to use this new agent for 🤝 Configuring initial default handlers.

OpenAI

You can statically-define a single 🤖 agent instance powered by the OpenAI provider with the help of the playbook's preset variables.

The OpenAI provider is only meant to be used with OpenAI's official API and compatibility with other services (which do not fully adhere to the OpenAI API spec completely) is limited. If you're targeting an OpenAI-compatible service, use the OpenAI Compatible provider instead.

Here's an example addition to your vars.yml file:

matrix_bot_baibot_config_agents_static_definitions_openai_enabled: true

matrix_bot_baibot_config_agents_static_definitions_openai_config_api_key: "YOUR_API_KEY_HERE"

# The playbook defines a default prompt for all statically-defined agents.
# You can adjust it in the `matrix_bot_baibot_config_agents_static_definitions_prompt` variable,
# or you can adjust it below only for the OpenAI agent.
# matrix_bot_baibot_config_agents_static_definitions_openai_config_text_generation_prompt: "{{ matrix_bot_baibot_config_agents_static_definitions_prompt }}"

# If you'd like to use another text-generation agent, uncomment and adjust:
# matrix_bot_baibot_config_agents_static_definitions_openai_config_text_generation_model_id: gpt-4o

# See `defaults/main.yml` in the baibot role for more configuration options.

Because this is a statically-defined agent, it will be given a static/ ID prefix and will be named static/openai.

If you'd like to use more than one model, take a look at the Configuring additional agents (without a preset) section below.

💡 You may also wish to use this new agent for 🤝 Configuring initial default handlers.

OpenAI Compatible

You can statically-define a single 🤖 agent instance powered by the OpenAI Compatible provider with the help of the playbook's preset variables.

This provider allows you to use OpenAI-compatible API services like OpenRouter, Together AI, etc.

Some of these popular services already have shortcut providers (see supported providers leading to this one behind the scenes - this make it easier to get started.

As of this moment, the playbook does not include presets for any of these services, so you'll need to Configuring additional agents (without a preset).

Configuring additional agents (without a preset)

The Ansible role may be lacking preset variables for some ☁️ provider, or you may wish to statically-define an agent on the same provider twice (or more) with different configuration.

It's possible to inject your own agent configuration using the matrix_bot_baibot_config_agents_static_definitions_custom Ansible variable.

You can also define providers at runtime, by chatting with the bot, so using Ansible is not a requirement.

Below is an an example demonstrating statically-defining agents via Ansible without using presets:

matrix_bot_baibot_config_agents_static_definitions_custom:
  # This agent will use the GPT 3.5 model and will only support text-generation,
  # even though the `openai` provider could support other features (e.g. image-generation).
  - id: my-openai-gpt-3.5-turbo-agent
    provider: openai
    config:
        base_url: https://api.openai.com/v1
        api_key: "YOUR_API_KEY_HERE"
        text_generation:
          model_id: gpt-3.5-turbo-0125
          prompt: "{{ matrix_bot_baibot_config_agents_static_definitions_prompt }}"
          temperature: 1.0
          max_response_tokens: 4096
          max_context_tokens: 16385
        speech_to_text: null
        text_to_speech: null
        image_generation: null

  # This agent uses the `openai` provider, but adjusts the base URL, so that it points to some Ollama instance
  # (which supports an OpenAI-compatible API).
  - id: my-ollama-agent
    provider: openai
    config:
      base_url: http://ollama-service:1234/v1
      api_key: ""
      text_generation:
        model_id: "llama3.1:8b"
        prompt: "{{ matrix_bot_baibot_config_agents_static_definitions_prompt }}"
        temperature: 1.0
        max_response_tokens: 4096
        max_context_tokens: 128000
          speech_to_text: null
          text_to_speech: null
          image_generation: null

Because these are statically-defined agents, they will be given a static/ ID prefix and will be named static/my-openai-gpt-3.5-turbo-agent and static/my-ollama-agent, respectively.

💡 To figure out what to put in the config section, refer to the ☁️ provider page, which contains sample configuration YAML for each provider.

As with any 🤖 agent, defining them means they exist. To actually make use of them, they need to be configured as handlers globally or in a specific room - see Mixing & matching models.

💡 You may also wish to use these new agents for 🤝 Configuring initial default handlers.

🤝 Configuring initial default handlers

This section is only useful if you're 🤖 Configuring agents via Ansible, as it lets you put these agents to use as soon as the bot starts (by adjusting the bot's initial global configuration).

If you're not configuring agents via Ansible, you can skip this section.

This section is only useful the first time around. Once initially configured the global configuration cannot be managed Ansible, but only via bot commands.

baibot supports various purposes:

• 💬 text-generation: communicating with you via text

• 🦻 speech-to-text: turning your voice messages into text

• 🗣️ text-to-speech: turning bot or users text messages into voice messages

• 🖌️ image-generation: generating images based on instructions

• ❓ catch-all: special purposes, indicating use as a fallback (when no specific handler is configured)

Mixing & matching models is made possible by the bot's ability to have different 🤝 handlers configured for different purposes.

This configuration can be done as a global fallback, or per-room. Both of these 🛠️ configurations are managed at runtime (viat chat), but the global configuration can have some initial defaults configured via Ansible.

You can configure the initial values for these via Ansible, via the matrix_bot_baibot_config_initial_global_config_handler_* variables.

Example additional vars.yml configuration:

# Note: these are initial defaults for the bot's global configuration.
# As such, changing any of these values subsequently has no effect on the bot's behavior.
# Once initially configured, the global configuration is managed via bot commands, not via Ansible.

matrix_bot_baibot_config_initial_global_config_handler_catch_all: static/openai

# In this example, there's no need to define any of these below.
# Configuring the catch-all purpose handler is enough.
matrix_bot_baibot_config_initial_global_config_handler_text_generation: null
matrix_bot_baibot_config_initial_global_config_handler_text_to_speech: null
matrix_bot_baibot_config_initial_global_config_handler_speech_to_text: null
matrix_bot_baibot_config_initial_global_config_handler_image_generation: null

Note: these are initial defaults for the bot's global configuration. As such, changing any of these values subsequently has no effect on the bot's behavior. Once initially configured the global configuration cannot be managed Ansible, but only via bot commands.

Installing

After configuring the playbook, run it with playbook tags as below:

ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start

Notes:

• The ensure-matrix-users-created playbook tag makes the playbook automatically create the bot's user account.

• The shortcut commands with the just program are also available: just install-all or just setup-all

  just install-all is useful for maintaining your setup quickly when its components remain unchanged. If you adjust your vars.yml to remove other components, you'd need to run just setup-all, or these components will still remain installed.

• If you change the bot password (matrix_bot_baibot_config_user_password in your vars.yml file) subsequently, the bot user's credentials on the homeserver won't be updated automatically. If you'd like to change the bot user's password, use a tool like synapse-admin to change it, and then update matrix_bot_baibot_config_user_password to let the bot know its new password.

Usage

To use the bot, invite the @baibot:example.com bot user into a room.

If you're an allowed bot 👥 user (see 👥 Initial users configuration), the bot will accept your invitation and join the room.

After joining, the bot will introduce itself and show information about the ✨ features that are enabled for it.

If you've 🤖 configured one or more agents via Ansible and have 🤝 configured initial default handlers, the bot will immediately be able to make use of these agents for this new room. Otherwise, you will need to configure agents and/or handlers via chat commands.

Send !bai help to the room at any time to see the bot's help menu for additional commands.

You can also refer to the upstream baibot project's documentation.

Debugging

As with all other services, you can find service logs in systemd-journald by running something like journalctl -fu matrix-bot-baibot

The default logging level for this service is info, but you can increase it to debug (or even trace) with the following additional configuration:

# Adjust the bot's own logging level.
matrix_bot_baibot_config_logging_level_baibot: debug

# Adjust the logging level for the mxlink bot library used by the bot.
matrix_bot_baibot_config_logging_level_mxlink: debug

# Adjust the logging level for other libraries used by the bot.
# Having this set to a value other than "warn" can be very noisy.
matrix_bot_baibot_config_logging_level_other_libs: debug

Alternatively, you can use a single variable to set the logging level for all of the above (bot + all libraries):

matrix_bot_baibot_config_logging: debug