# Clustering

> For the complete documentation index, see [llms.txt](https://doc.ibexa.co/en/5.0/llms.txt).

Clustering enables you to host one installation of Ibexa DXP on multiple servers.

Clustering in Ibexa DXP refers to setting up your installation with several web servers for handling more load and/or for failover.

## Server setup overview

This diagram illustrates how clustering in Ibexa DXP is typically set up. The parts illustrate the different roles needed for a successful cluster setup.

![Server setup for clustering](https://doc.ibexa.co/en/5.0/infrastructure_and_maintenance/img/server_setup.png)

The number of web servers, Redis/Valkey, Solr, Varnish, Database, and NFS servers, but also whether some servers play several of these roles (typically running Redis/Valkey across the web server), is up to you and your performance needs.

The minimal requirements are:

- [Shared HTTP cache (using Varnish)](https://doc.ibexa.co/en/5.0/infrastructure_and_maintenance/cache/http_cache/reverse_proxy/#using-varnish-or-fastly)
- [Shared persistence cache](#shared-persistence-cache) and [sessions](#shared-sessions) (using Redis/Valkey)
- Shared database (using MySQL/MariaDB)
- [Shared binary files](#shared-binary-files) (using NFS, or S3)

For more information on requirements, see [Requirements page](https://doc.ibexa.co/en/5.0/getting_started/requirements/index.md).

It's also recommended to use:

- [Solr](https://doc.ibexa.co/en/5.0/search/search_engines/solr_search_engine/solr_overview/index.md) or [Elasticsearch](https://doc.ibexa.co/en/5.0/search/search_engines/elasticsearch/elasticsearch_overview/index.md) for better search and performance
- a CDN for improved performance and faster ping time worldwide
  - you can use Fastly, which has native support as HTTP cache and CDN.
- active/passive database for failover
- more recent versions of PHP and MySQL/MariaDB within [what is supported](https://doc.ibexa.co/en/5.0/getting_started/requirements/index.md) for your Ibexa DXP version to get more performance out of each server. Numbers might vary so make sure to test this when upgrading.

### Shared persistence cache

Redis and Valkey are the recommended cache solutions for clustering.

See [persistence cache documentation](https://doc.ibexa.co/en/5.0/infrastructure_and_maintenance/cache/persistence_cache/#persistence-cache-configuration) on information on how to configure them.

### Shared sessions

For a [cluster](https://doc.ibexa.co/en/5.0/infrastructure_and_maintenance/clustering/clustering/index.md) setup you need to configure sessions to use a back end that is shared between web servers. The main option out of the box in Symfony is the PHP Redis session handler (also compatible with Valkey). Alternatively, there is Symfony session handler for PDO (database).

To avoid concurrent access to session data from front-end nodes, if possible you should either:

- Enable [Session locking](https://www.php.net/manual/en/features.session.security.management.php#features.session.security.management.session-locking)
- Use "Sticky Session", aka [Load Balancer Persistence](https://en.wikipedia.org/wiki/Load_balancing_%28computing%29#Persistence)

Session locking is available with `php-redis` (v4.2.0 and higher).

On Ibexa Cloud (and Upsun) Redis and Valkey are preferred and supported.

### Shared binary files

Ibexa DXP supports multi-server setups by means of [custom IO handlers](https://doc.ibexa.co/en/5.0/content_management/file_management/file_management/#dfs-cluster-handler). They make sure that files are correctly synchronized among the multiple clients using the data.

## DFS IO handler

The DFS IO handler (`legacy_dfs_cluster`) can be used to store binary files on an NFS server. It uses a database to manipulate metadata, making up for the potential inconsistency of network-based filesystems.

### Configuring the DFS IO handler

You need to configure both metadata and binarydata handlers.

Ibexa DXP ships with a custom local adapter (`ibexa.io.nfs.adapter.site_access_aware`), which decorates the Flysystem v2 local adapter to enable support for SiteAccess-aware settings. If an NFS path relies on SiteAccess-aware dynamic parameters, you must use the custom local adapter instead of the Flysystem v2 local adapter. Configure the custom local adapter to read/write to the NFS mount point on each local server. As metadata handler, create a DFS one, configured with a Doctrine connection.

> **Tip: Tip**
>
> The default database install now includes the dfs table *in the same database*

First, define DFS folder path as a variable in `.env` file:

`DFS_NFS_PATH=<absolute_directory_path>`

Next, if you're using a separate DFS database, configure it via the `DATABASE_URL` variable in the `.env` file. Depending on which database you're using:

`DFS_DATABASE_URL=mysql://root:rootpassword@127.0.0.1:3306/ibexa_dfs?serverVersion=8.0`

or

`DATABASE_URL=postgresql://root:rootpassword@127.0.0.1:5432/ibexa_dfs?serverVersion=14.18`

For production, it's recommended to create the DFS table in its own database, manually importing its schema definition:

> **Note: dfs_schema.sql (MySQL)**
>
> ```sql
>     CREATE TABLE ibexa_dfs_file (
>       name text NOT NULL,
>       name_trunk text NOT NULL,
>       name_hash varchar(34) NOT NULL DEFAULT '',
>       datatype varchar(255) NOT NULL DEFAULT 'application/octet-stream',
>       scope varchar(25) NOT NULL DEFAULT '',
>       size bigint(20) unsigned NOT NULL DEFAULT '0',
>       mtime int(11) NOT NULL DEFAULT '0',
>       expired tinyint(1) NOT NULL DEFAULT '0',
>       status tinyint(1) NOT NULL DEFAULT '0',
>       PRIMARY KEY (name_hash),
>       KEY ibexa_dfs_file_name (name (191)),
>       KEY ibexa_dfs_file_name_trunk (name_trunk (191)),
>       KEY ibexa_dfs_file_mtime (mtime),
>       KEY ibexa_dfs_file_expired_name (expired,name (191))
>     ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
> ```

> **Note: dfs_schema.sql (PostgreSQL)**
>
> ```sql
> CREATE TABLE ibexa_dfs_file (
>   name_hash varchar(34) DEFAULT '' NOT NULL,
>   name text NOT NULL,
>   name_trunk text NOT NULL,
>   datatype varchar(255) DEFAULT 'application/octet-stream' NOT NULL,
>   scope character varying(25) DEFAULT '' NOT NULL,
>   size bigint DEFAULT 0 NOT NULL,
>   mtime integer DEFAULT 0 NOT NULL,
>   expired boolean DEFAULT false NOT NULL,
>   status boolean DEFAULT false NOT NULL
> );
>
> ALTER TABLE ONLY ibexa_dfs_file
>   ADD CONSTRAINT ibexa_dfs_file_pkey PRIMARY KEY (name_hash);
>
> CREATE INDEX ibexa_dfs_file_expired_name ON ibexa_dfs_file USING btree (expired, name);
> CREATE INDEX ibexa_dfs_file_mtime ON ibexa_dfs_file USING btree (mtime);
> CREATE INDEX ibexa_dfs_file_name ON ibexa_dfs_file USING btree (name);
> CREATE INDEX ibexa_dfs_file_name_trunk ON ibexa_dfs_file USING btree (name_trunk);
> ```

This example uses Doctrine connection named `dfs`:

```yaml
parameters:
    env(DFS_DATABASE_URL): '%env(resolve:DATABASE_URL)%'
    dfs_database_url: '%env(resolve:DFS_DATABASE_URL)%'
    ibexa.io.nfs.adapter.config:
        root: '%kernel.project_dir%/%env(string:DFS_NFS_PATH)%'
        path: '$var_dir$/$storage_dir$/'
        writeFlags: ~
        linkHandling: ~
        permissions: [ ]

# new Doctrine connection for the DFS legacy_dfs_cluster metadata handler.
doctrine:
    dbal:
        connections:
            dfs:
                # configure these for your database server
                driver: '%env(string:DFS_DATABASE_DRIVER)%'
                charset: '%env(string:DFS_DATABASE_CHARSET)%'
                default_table_options:
                    charset: '%env(string:DFS_DATABASE_CHARSET)%'
                    collate: '%env(string:DFS_DATABASE_COLLATION)%'
                url: '%env(string:DFS_DATABASE_URL)%'

# define the Flysystem handler
oneup_flysystem:
    adapters:
        nfs_adapter:
            custom:
                service:  ibexa.io.nfs.adapter.site_access_aware

# define the Ibexa handlers
ibexa_io:
    binarydata_handlers:
        nfs:
            flysystem:
                adapter: nfs_adapter
    metadata_handlers:
        dfs:
            legacy_dfs_cluster:
                connection: doctrine.dbal.dfs_connection

# set the application handlers
ibexa:
    system:
        default:
            io:
                metadata_handler: dfs
                binarydata_handler: nfs
```

> **Tip: Tip**
>
> If you're looking to [set up S3](https://doc.ibexa.co/en/5.0/infrastructure_and_maintenance/clustering/clustering_with_aws_s3/index.md) or other [Flysystem](https://flysystem.thephpleague.com/docs/)/third-party adapters like Google Cloud Storage, this needs to be configured as binary handler. The rest here still stays the same, the DFS metadata handler takes care of caching the lookups to avoid slow IO lookups.

#### Customizing the storage directory

Earlier versions required the NFS adapter directory to be set to `$var_dir$/$storage_dir$` part for the NFS path. It's no longer required, but the default prefix used to serve binary files still matches this expectation.

If you decide to change this setting, make sure you also set `io.url_prefix` to a matching value. If you set the NFS adapter's directory to `/path/to/nfs/storage`, use this configuration so that the files can be served by Symfony:

```yaml
ibexa:
    system:
        default:
            io:
                url_prefix: storage
```

As an alternative, you may serve images from NFS by using a dedicated web server. If in the example above, this server listens on `http://static.example.com/` and uses `/path/to/nfs/storage` as the document root, configure `io.url_prefix` as follows:

```yaml
ibexa:
    system:
        default:
            io:
                url_prefix: 'http://static.example.com/'
```

You can read more about that on [Binary files URL handling](https://doc.ibexa.co/en/5.0/content_management/file_management/file_url_handling/#file-url-handling).

### Web server rewrite rules

The default Ibexa DXP rewrite rules let image requests be served directly from disk. In a cluster setup, files matching `^/var/([^/]+/)?storage/images(-versioned)?/.*` have to be passed through `/public/index.php` instead.

In any case, this specific rewrite rule must be placed before the ones that "ignore" image files and let the web server serve the files directly.

#### Apache

```apacheconf
RewriteRule ^/var/([^/]+/)?storage/images(-versioned)?/.* /index.php [L]
```

Place this before the standard image rewrite rule in your vhost config (or uncomment if already there).

#### nginx

```nginx
rewrite "^/var/([^/]+/)?storage/images(-versioned)?/(.*)" "/index.php" break;
```

Place this before the include of `ibexa_params.d`/`ibexa_rewrite_params` in your vhost config (or uncomment if already there).

## Migrating to a cluster setup

If you're migrating an existing single-server site to a cluster setup, and not setting up clustering from scratch, you need to migrate your files. Once you have configured your binarydata and metadata handlers, you can run the `ibexa:io:migrate-files` command. You can also use it when you're migrating from one data handler to another, for example, from NFS to Amazon S3.

This command shows which handlers are configured:

```bash
> php bin/console ibexa:io:migrate-files --list-io-handlers
Configured meta data handlers: default, dfs, aws_s3
Configured binary data handlers: default, nfs, aws_s3
```

You can do the actual migration like this:

```bash
php bin/console ibexa:io:migrate-files --from=default,default --to=dfs,nfs --env=prod
```

The `--from` and `--to` values must be specified as `<metadata_handler>,<binarydata_handler>`. If `--from` is omitted, the default IO configuration is used. If `--to` is omitted, the first non-default IO configuration is used.

> **Tip: Tip**
>
> The command must be executed with the same permissions as the web server.

While the command is running, the files should not be modified. To avoid surprises you should create a [backup](https://doc.ibexa.co/en/5.0/infrastructure_and_maintenance/backup/index.md) and/or execute a dry run before doing the actual update, using the `--dry-run` switch.

Since this command can run for a long time, to avoid memory exhaustion, use the `--env=prod` switch when you run it in the production environment.