Skip to content
  • Documentation >
  • Guide >
  • Repository >
  • Data migration >
  • Exporting and importing data

Exporting data

To see an example of migrations in action, export data already present in your installation.

To export Repository content, use the ibexa:migrations:generate command. This command generates a YAML file with the requested part of the Repository. The file is located by default in the src/Migrations/Ibexa/migrations folder or in a custom folder that you configure. You can later use this file to import the data.

1
php bin/console ibexa:migrations:generate --type=content --mode=create --siteaccess=admin

This generates a file containing all Content items. Below you can see part of the output of the default Ibexa DXP installation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
-
    type: content
    mode: create
    metadata:
        contentType: user_group
        mainTranslation: eng-GB
        creatorId: 14
        modificationDate: '2002-10-06T17:19:56+02:00'
        publicationDate: '2002-10-06T17:19:56+02:00'
        remoteId: f5c88a2209584891056f987fd965b0ba
        alwaysAvailable: true
        section:
            id: 2
            identifier: users
    location:
        parentLocationId: 1
        parentLocationRemoteId: null
        locationRemoteId: 3f6d92f8044aed134f32153517850f5a
        hidden: false
        sortField: 1
        sortOrder: 1
        priority: 0
    fields:
        -
            fieldDefIdentifier: name
            languageCode: eng-GB
            value: Users
        -
            fieldDefIdentifier: description
            languageCode: eng-GB
            value: 'Main group'
    references:
        -
            name: ref__content__user_group__users
            type: content_id
        -
            name: ref_location__user_group__users
            type: location_id
        -
            name: ref_path__user_group__users
            type: path

The output contains all the possible information for a future migration command. Parts of it can be removed or modified. You can treat it as a template for another Content item for user group. For example, you could:

  • Remove references if you don't intend to store IDs for future use (see migration references)
  • Remove publicationDate, modificationDate, locationRemoteId, as those are generated if not passed (just like in PHP API)
  • Add actions
  • Add fields for other languages present in the system.

Similarly, you can create update and delete operations. They are particularly useful combined with match-property. This option is automatically added as part of match expression in the update/delete migration:

1
php bin/console ibexa:migrations:generate --type=content_type --mode=update --match-property=content_type_identifier --value=article

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
-
    type: content_type
    mode: update
    match:
        field: content_type_identifier
        value: article
    metadata:
        identifier: article
        mainTranslation: eng-GB
        modifierId: 14
        modificationDate: '2012-07-24T14:35:34+00:00'
        remoteId: c15b600eb9198b1924063b5a68758232
        urlAliasSchema: ''
        nameSchema: '<short_title|title>'
        container: true
        defaultAlwaysAvailable: false
        defaultSortField: 1
        defaultSortOrder: 1
        translations:
            eng-GB:
                name: Article
    fields:
        -
            identifier: title
            type: ezstring
            position: 1
            translations:
                eng-GB:
                    name: Title
            required: true
            searchable: true
            infoCollector: false
            translatable: true
            category: ''
            defaultValue: 'New article'
            fieldSettings: {  }
            validatorConfiguration:
                StringLengthValidator:
                    maxStringLength: 255
                    minStringLength: null
        # - ...

Note that you should test your migrations. See Executing migrations.

Tip

Migration command can be executed with database rollback at the end with the --dry-run option.

Caution

--siteaccess option usage can be relevant when multiple languages or multiple repositories are used. To prevent translation loss, it is recommended that you use the SiteAccess that has all the languages used in your implementation, most likely the Back Office one.

type

The mandatory --type option defines the type of Repository data to export. The following types are available:

  • content
  • content_type
  • role
  • content_type_group
  • user
  • user_group
  • language
  • object_state_group
  • object_state
  • section
  • location

If you do not provide the --type option, the command asks you to select a type of data.

mode

The mandatory --mode option defines the action that importing the file performs. The following modes are available:

  • create - creates new items
  • update - updates an existing item. Only covers specified fields and properties. If the item does not exist, causes an error.
  • delete - deletes an existing item. If the item does not exist, causes an error.

If you do not provide the --mode option, the command asks you to select the mode.

The following data migration steps are available:

type create update delete swap
content_type ✔ ✔
content_type_group ✔ ✔
content ✔ ✔ ✔
language ✔
location ✔ ✔
object_state ✔
object_state_group ✔
role ✔ ✔ ✔
section ✔ ✔
user ✔ ✔
user_group ✔ ✔

siteaccess

The optional --siteaccess option enables you to export (or import) data in a SiteAccess configuration's context. If not provided, the default SiteAccess is used.

It is recommended that you use the SiteAccess of the target repository's Back Office.

Specifying the SiteAccess can be mandatory, for example, when you use several SiteAccesses to handle several languages. Export and import commands only work with languages supported by the context SiteAccess. You must export and import with the SiteAccess supporting all the languages to preserve translations.

This option is also important if you use several repositories with their own dabases.

match-property

The optional --match-property option, together with value, enables you to select which data from the Repository to export. match-property defines what property should be used as a criterion for selecting data. The following properties are available (per type):

  • content
    • content_id
    • content_type_id
    • content_type_group_id
    • content_type_identifier
    • content_remote_id
    • location_id
    • location_remote_id
    • parent_location_id
    • user_id
  • content_type
    • content_type_identifier
  • content_type_group
    • content_type_group_id
    • content_type_group_identifier
  • language
    • language_code
  • location
    • location_remote_id
    • location_id
  • object_state
    • object_state_id
    • object_state_identifier
  • object_state_group
    • object_state_group_id
    • object_state_group_identifier
  • role
    • identifier
    • id
  • section
    • section_id
    • section_identifier
  • user
  • login
  • email
  • user_group
  • id
  • remoteId

You can extend the list of available matchers by creating a custom one.

value

The optional --value option, together with match-property, filters the Repository content that the command exports. value defines which values of the match-property should be included in the export.

For example, to export only Article Content items, use the content_type_identifier match property with article as the value:

1
php bin/console ibexa:migrations:generate --type=content --mode=create --match-property=content_type_identifier --value=article

Note

The same match-property and value is added to generated update and delete type migration files.

file

The optional --file option defines the name of the YAML file to export to.

1
php bin/console ibexa:migrations:generate --type=content --mode=create --file=my_data_export.yaml

Note

When migrating multiple files at once (for example when calling ibexa:migrations:migrate without options), they are executed in alphabetical order.

user-context

The optional --user-context option enables you to run the export command as a specified User. The command only exports Repository data that the selected User has access to. By default the admin account is used, unless specifically overridden by this option or in bundle configuration (ibexa_migrations.default_user_login).

1
php bin/console ibexa:migrations:generate --type=content --mode=create --user-context=jessica_andaya

Executing migrations

To import Repository data from YAML files, run the ibexa:migrations:migrate command.

Place your import file in the src/Migrations/Ibexa/migrations folder or in a custom folder that you configure. The command takes the file name within this folder as parameter. If file is not specified, all files within this directory are used.

1
php bin/console ibexa:migrations:migrate --file=my_data_export.yaml --siteaccess=admin

Ibexa Migrations store execution metadata in ibexa_migrations database table. This allows incremental upgrades: the ibexa:migrations:migrate command ignores files that it had previously executed.

The --siteaccess option usage can be relevant when multiple languages or multiple repositories are used.

Repeatable steps

You can run a set of one or more similar migration steps multiple times by using the special repeatable migration type.

A repeatable migration performs the defined migration steps as many times as the iterations setting declares.

1
2
3
4
5
-
    type: repeatable
    mode: create
    iterations: 5
    steps:

Tip

You can use repeatable migration steps, for example, to quickly generate large numbers of Content items for testing purposes.

You can vary the operations using the iteration counter.

For example, to create five Folders, with names ranging from "Folder 0" to "Folder 4", you can run the following migration using the iteration counter i:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
-
    type: repeatable
    mode: create
    iterations: 5
    steps:
        -   type: content
            mode: create
            metadata:
                contentType: folder
                mainTranslation: eng-GB
            location:
                parentLocationId: 2
            fields:
                -   fieldDefIdentifier: name
                    languageCode: eng-GB
                    value: 'Folder ###SSS i SSS###'

To vary the content name, the migration above uses Symfony expression syntax.

In the example above, the expression is enclosed in ### and the repeated string SSS.

Note

Iteration counter is assigned to i by default, but you can modify it in the iteration_counter_name setting.

Generating fake data

You can also generate fake data with the help of FakerPHP.

To use it, first install Faker on your system:

1
composer require fakerphp/faker

Then, you can use faker() in expressions, for example:

1
2
3
                -   fieldDefIdentifier: short_name
                    languageCode: eng-GB
                    value: '### faker().name() ###'

This step generates Field values with fake personal names.

Migration examples

The following examples show what data you can import using data migrations.

Content Types

The following example shows how to create a Content Type with two Field definitions.

The required metadata keys are: identifier, mainTranslation, contentTypeGroups and translations.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
-
    type: content_type
    mode: create
    metadata:
        identifier: blog_post
        mainTranslation: eng-GB
        contentTypeGroups:
            - Content
        translations:
            eng-GB:
                name: Blog Post
    fields:
        -   identifier: title
            type: ezstring
            required: true
            translations:
                eng-GB:
                    name: 'Title'
        -   identifier: body
            type: ezrichtext
            required: false
            translations:
                eng-GB:
                    name: 'Body'

Locations

The following example shows how to swap content items assigned to given locations.

1
2
3
4
5
6
7
8
9
-
    type: location
    mode: swap
    match1:
        field: location_remote_id
        value: f3e90596361e31d496d4026eb624c983
    match2:
        field: location_id
        value: 5

Images

The following example shows how to migrate an example-image.png located in public/var/site/storage/images/3/8/3/0/383-1-eng-GB without manually placing it in the appropriate path.

To prevent the manual addition of images to specific DFS or local locations, such as public/var/site/storage/images/ you can move image files to, for example src/Migrations/images. Adjust the migration file and configure the image field data as follows:

1
2
3
4
5
6
        -   fieldDefIdentifier: image
            languageCode: eng-GB
            value:
                alternativeText: ''
                fileName: example-image.png
                path: src/Migrations/images/example-image.png

This migration copies the image to the appropriate directory, in this case public/var/site/storage/images/3/8/3/0/254-1-eng-GB/example-image.png, enabling swift file migration regardless of storage (local, DFS).

Roles

The following example shows how to create a Role. A Role requires the identifier metadata key.

For each Policy assigned to the Role, you select the module and function, with optional Limitations.

The following example shows the creation of a Contributor Role:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
-
    type: role
    mode: create
    metadata:
        identifier: Contributor
    policies:
        -
            module: content
            function: read
        -
            module: content
            function: create
            limitations:
                -
                    identifier: Class
                    values: [folder, article, blog_post]
                -
                    identifier: Section
                    values: [standard, media]
        -
            module: content
            function: edit
            limitations:
                -
                    identifier: Owner
                    values: ['1']

To update an existing Role, two policies' modes are available:

  • replace: (default) All existing policies are replaced by the ones from the migration.
  • append: Migration policies are added while already existing ones are kept.

The following example shows how to replace the policies of the existing Editor Role:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
-
    type: role
    mode: update
    match:
        field: identifier
        value: Editor
    policies:
        -   module: content
            function: '*'
        -   module: user
            function: login
            limitations:
                -   identifier: SiteAccess
                    values: [ admin ]
        -   module: url
            function: '*'

The following example shows the addition of a policy to the Anonymous Role:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    type: role
    mode: update
    match:
        field: identifier
        value: Anonymous
    policies:
        mode: append
        list:
            -
                module: user
                function: login
                limitations:
                    -   identifier: SiteAccess
                        values: [ new_siteaccess ]

The following example shows how to delete the Contributor Role:

1
2
3
4
5
6
-
    type: role
    mode: delete
    match:
        field: identifier
        value: Contributor

Expression syntax

You can use Symfony expression syntax in data migrations. It is especially useful in repeatable steps, where you can use it to generate varied content in migration steps.

The expression syntax uses the following structure: ###<IDENTIFIER> <EXPRESSION> <IDENTIFIER>###

The IDENTIFIER can be any repeated string that encloses the actual expression.