Skip to content
  • Documentation >
  • Content management >
  • Data migration >
  • Importing data

Importing data

To import data from YAML migration files into Repository, you run the ibexa:migrations:migrate command.

The ibexa:migrations:import command automatically places migration files in the correct folder.

Alternatively, you can place the files manually in the src/Migrations/Ibexa/migrations folder or in a custom folder that you configure, and specify the file name within this folder as parameter. If you don't specify the file, all files within this directory are used.

1
php bin/console ibexa:migrations:migrate --file=my_data_export.yaml --siteaccess=admin

Migrations store execution metadata in the ibexa_migrations database table. This allows incremental upgrades: the ibexa:migrations:migrate command ignores files that it had previously executed.

The --siteaccess option usage can be relevant when multiple languages or multiple repositories are used.

Migration step

A data migration step is a single operation in data migration process that combines a mode (for example: create, update, delete) and a type (for example: content, section, currency), with optional additional information depending on the specific step.

In a migration file, a step is an array item starting with the mandatory properties type and mode, for example:

1
2
3
-
    type: content
    mode: create

Then, the step is described by additional properties depending on its type and mode.

Available migrations

The following data migration step modes are available:

type create update delete
attribute ✔ ✔ ✔
attribute_group ✔ ✔ ✔
content_type ✔ ✔ ✔
content_type_group ✔ ✔ ✔
content ✔ ✔ ✔
currency ✔ ✔ ✔
customer_group ✔ ✔ ✔
language ✔
location ✔
object_state ✔
object_state_group ✔
payment_method ✔
product_asset ✔
product_availability ✔
product_price ✔
product_variant ✔
role ✔ ✔ ✔
section ✔ ✔
segment ✔ ✔ ✔
segment_group ✔ ✔ ✔
setting ✔ ✔ ✔
user ✔ ✔
user_group ✔ ✔ ✔

Repeatable steps

You can run a set of one or more similar migration steps multiple times by using the special repeatable migration type.

A repeatable migration performs the defined migration steps as many times as the iterations setting declares.

1
2
3
4
5
-
    type: repeatable
    mode: create
    iterations: 5
    steps:

Tip

You can use repeatable migration steps, for example, to quickly generate large numbers of content items for testing purposes.

You can vary the operations using the iteration counter.

For example, to create five Folders, with names ranging from "Folder 0" to "Folder 4", you can run the following migration using the iteration counter i:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
-
    type: repeatable
    mode: create
    iterations: 5
    steps:
        -   type: content
            mode: create
            metadata:
                contentType: folder
                mainTranslation: eng-GB
            location:
                parentLocationId: 2
            fields:
                -   fieldDefIdentifier: name
                    languageCode: eng-GB
                    value: 'Folder ###SSS i SSS###'

To vary the content name, the migration above uses Symfony expression syntax.

In the example above, the expression is enclosed in ### and the repeated string SSS.

Note

Iteration counter is assigned to i by default, but you can modify it in the iteration_counter_name setting.

Generating fake data

You can also generate fake data with the help of FakerPHP.

To use it, first install Faker on your system:

1
composer require fakerphp/faker

Then, you can use faker() in expressions, for example:

1
2
3
                -   fieldDefIdentifier: short_name
                    languageCode: eng-GB
                    value: '### faker().name() ###'

This step generates Field values with fake personal names.

Expression syntax

You can use Symfony expression syntax in data migrations, like in repeatable steps, where you can use it to generate varied content in migration steps.

The expression syntax uses the following structure: ###<IDENTIFIER> <EXPRESSION> <IDENTIFIER>###

The IDENTIFIER can be any repeated string that encloses the actual expression.

Built-in functions

Built-in expression language functions that are tagged with ibexa.migrations.template.expression_language.function:

  • to_bool, to_int, to_float, to_string - convert various data types by passing them into PHP casting functions (like floatval, intval, and others).
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
                -   fieldDefIdentifier: show_children
                    languageCode: eng-US
                    value: '###XXX to_bool(i % 3) XXX###'

                -   fieldDefIdentifier: quantity
                    languageCode: eng-US
                    value: '###XXX to_int("42") XXX###'

                -   fieldDefIdentifier: price
                    languageCode: eng-US
                    value: '###XXX to_float("19.99") XXX###'

                -   fieldDefIdentifier: description
                    languageCode: eng-US
                    value: '###XXX to_string(123) XXX###'
  • ibexa.migrations.template.reference - references a specific object or resource within your application or configuration. Learn more about migration references.
1
2
3
                -   fieldDefIdentifier: some_field
                    languageCode: eng-US
                    value: '###XXX reference("example_reference") XXX###'
  • ibexa.migrations.template.project_dir - retrieves the project's root directory path, for example to construct file paths or access project-specific resources.
1
2
3
                -   fieldDefIdentifier: project_directory
                    languageCode: eng-US
                    value: '###XXX project_dir() XXX###'

Custom functions

To add custom functionality into Migration's expression language declare it as a service and tag it with ibexa.migrations.template.expression_language.function.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
ibexa.migrations.template.to_bool:
    class: Closure
    factory: [ Closure, fromCallable ]
    arguments:
        - 'boolval'
    tags:
        -   name: 'ibexa.migrations.template.expression_language.function'
            function: to_bool

ibexa.migrations.template.faker:
    class: Closure
    factory: [ Closure, fromCallable ]
    arguments:
        - 'Faker\Factory::create'
    tags:
        -   name: 'ibexa.migrations.template.expression_language.function'
            function: faker

Service-based functions can be also added, but they must be callable, requiring either an __invoke function or a wrapping service with one.

Migration examples

The following examples show what data you can import using data migrations.

Content types

The following example shows how to create a content type with two Field definitions.

The required metadata keys are: identifier, mainTranslation, contentTypeGroups and translations.

The default values of Field definition properties mirror the underlying PHP API, for example:

  • translatable defaults to true
  • required defaults to false
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
-
    type: content_type
    mode: create
    metadata:
        identifier: blog_post
        mainTranslation: eng-GB
        contentTypeGroups:
            - Content
        translations:
            eng-GB:
                name: Blog Post
    fields:
        -   identifier: title
            type: ezstring
            required: true
            translations:
                eng-GB:
                    name: 'Title'
        -   identifier: body
            type: ezrichtext
            required: false
            translations:
                eng-GB:
                    name: 'Body'

Content items

The following example shows how to create two content items: a folder and an article inside it.

When creating a content item, three metadata keys are required: contentType, mainTranslation, and parentLocationId.

To use the Location ID of the folder, which is created automatically by the system, you can use a reference. In this case you assign the parent_folder_location_id reference name to the Location ID, and then use it when creating the article.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
-
    type: content
    mode: create
    metadata:
        contentType: folder
        mainTranslation: eng-GB
    location:
        parentLocationId: 2
    fields:
        -   fieldDefIdentifier: name
            languageCode: eng-GB
            value: 'Parent folder'
    references:
        -
            name: parent_folder_location_id
            type: location_id

-
    type: content
    mode: create
    metadata:
        contentType: article
        mainTranslation: eng-GB
    location:
        parentLocationId: 'reference:parent_folder_location_id'
    fields:
        -   fieldDefIdentifier: title
            languageCode: eng-GB
            value: 'Child article'
        -   fieldDefIdentifier: intro
            languageCode: eng-GB
            value:
                xml: |
                    <?xml version="1.0" encoding="UTF-8"?>
                    <section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ezxhtml="http://ibexa.co/xmlns/dxp/docbook/xhtml" xmlns:ezcustom="http://ibexa.co/xmlns/dxp/docbook/custom" version="5.0-variant ezpublish-1.0"><para>This is <emphasis role="strong">article into</emphasis>.</para></section>

Images

The following example shows how to migrate an example-image.png located in public/var/site/storage/images/3/8/3/0/383-1-eng-GB without manually placing it in the appropriate path.

To prevent the manual addition of images to specific DFS or local locations, such as public/var/site/storage/images/ you can move image files to, for example src/Migrations/images. Adjust the migration file and configure the image field data as follows:

1
2
3
4
5
6
        -   fieldDefIdentifier: image
            languageCode: eng-GB
            value:
                alternativeText: ''
                fileName: example-image.png
                path: src/Migrations/images/example-image.png

This migration copies the image to the appropriate directory, in this case public/var/site/storage/images/3/8/3/0/254-1-eng-GB/example-image.png, enabling swift file migration regardless of storage (local, DFS).

Roles

The following example shows how to create a Role. A Role requires the identifier metadata key.

For each Policy assigned to the Role, you select the module and function, with optional Limitations.

The following example shows the creation of a Contributor Role:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
-
    type: role
    mode: create
    metadata:
        identifier: Contributor
    policies:
        -
            module: content
            function: read
        -
            module: content
            function: create
            limitations:
                -
                    identifier: Class
                    values: [folder, article, blog_post]
                -
                    identifier: Section
                    values: [standard, media]
        -
            module: content
            function: edit
            limitations:
                -
                    identifier: Owner
                    values: ['1']

To update an existing Role, two policies' modes are available:

  • replace: (default) All existing policies are replaced by the ones from the migration.
  • append: Migration policies are added while already existing ones are kept.

The following example shows how to replace the policies of the existing Editor Role:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
-
    type: role
    mode: update
    match:
        field: identifier
        value: Editor
    policies:
        -   module: content
            function: '*'
        -   module: user
            function: login
            limitations:
                -   identifier: SiteAccess
                    values: [ admin ]
        -   module: url
            function: '*'

The following example shows the addition of a policy to the Anonymous Role:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    type: role
    mode: update
    match:
        field: identifier
        value: Anonymous
    policies:
        mode: append
        list:
            -
                module: user
                function: login
                limitations:
                    -   identifier: SiteAccess
                        values: [ new_siteaccess ]

The following example shows how to delete the Contributor Role:

1
2
3
4
5
6
-
    type: role
    mode: delete
    match:
        field: identifier
        value: Contributor

Users

The following example shows how to create a user.

The required metadata keys are: login, email, password, enabled, mainLanguage, and contentType. You also need to provide the user group's remote content ID.

You can use an action to assign a Role to the user.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
-
    type: user
    mode: create
    metadata:
        login: janedoe
        email: [email protected]
        password: Password123Password
        enabled: true
        mainLanguage: eng-GB
        contentType: user
    groups:
        - 3c160cca19fb135f83bd02d911f04db2
    fields:
        -
            fieldDefIdentifier: first_name
            languageCode: eng-GB
            value: John
        -
            fieldDefIdentifier: last_name
            languageCode: eng-GB
            value: Doe
    actions:
        - { action: assign_user_to_role, identifier: 'Member'}

Languages

The following example shows how to create a language.

The required metadata keys are: languageCode, name, and enabled.

1
2
3
4
5
6
7
-
    type: language
    mode: create
    metadata:
        languageCode: ger-DE
        name: German
        enabled: true

Product catalog

Attributes and attribute groups

The following example shows how to create an attribute group with two attributes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
-   type: attribute_group
    mode: create
    identifier: hat
    names:
        eng-GB: Hat

-   type: attribute
    mode: create
    identifier: size
    attribute_type_identifier: integer
    attribute_group_identifier: hat
    names:
        eng-GB: Size

-   type: attribute
    mode: create
    identifier: color
    attribute_type_identifier: selection
    attribute_group_identifier: hat
    names:
        eng-GB: Color
    options:
        choices:
            -    value: red
                 label:
                     "eng-GB": "Red"
            -    value: white
                 label:
                     "eng-GB": "White"
            -    value: black
                 label:
                     "eng-GB": "Black"

You can also update attributes, including changing which attribute group they belong to:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
-
    type: attribute
    mode: update
    criteria:
        type: field_value
        field: identifier
        value: width
        operator: '='
    identifier: new_width
    attribute_group_identifier: size
    names:
        eng-GB: New Width

You can't change the attribute type of an existing attribute.

Product types

The following example shows how to create a product type.

The main part of the migration file is the same as when creating a regular content type.

A product type must also contain the definition for an ibexa_product_specification Field. fieldSettings contains information about the product attributes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
-   type: content_type
    mode: create
    metadata:
        identifier: hat
        mainTranslation: eng-GB
        contentTypeGroups:
            - product
        translations:
            eng-GB:
                name: Hat
    fields:
        -   identifier: name
            type: ezstring
            required: true
            translations:
                eng-GB:
                    name: Name
        -   identifier: specification
            type: ibexa_product_specification
            required: true
            translatable: false
            translations:
                eng-GB:
                    name: Specification
            fieldSettings:
                attributes_definitions:
                    dimensions:
                        - { attributeDefinition: size, required: true, discriminator: false }
                        - { attributeDefinition: color, required: true, discriminator: true }

Products

The following example shows how to create a product:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
-   type: content
    mode: create
    metadata:
        contentType: hat
        mainTranslation: eng-GB
    location:
        parentLocationId: 60
    fields:
        -   fieldDefIdentifier: name
            languageCode: eng-GB
            value: 'Top hat 58cm'
        -   fieldDefIdentifier: specification
            languageCode: eng-GB
            value:
                code: top_hat__58
                attributes:
                    size: 58
                is_virtual: false

Product variants

The following example shows how to create variants for a product identified by its code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
-   type: product_variant
    mode: create
    base_product_code: top_hat__58
    variants:
        -   code: top_hat__58__white
            attributes:
                color: white
        -   code: top_hat__58__black
            attributes:
                color: black

Product assets

The following example creates an image content item from a local image file, and then uses it as a product asset for a variant (created in previous example):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
-   type: content
    mode: create
    metadata:
        contentType: image
        mainTranslation: eng-GB
    location:
        parentLocationId: 51 # Media/Images
    fields:
        -   fieldDefIdentifier: name
            languageCode: eng-GB
            value: 'Top hat 58cm Black'
        -   fieldDefIdentifier: image
            languageCode: eng-GB
            value:
                alternativeText: 'Top hat 58cm Black'
                fileName: 'top_hat_58cm_black.jpg'
                path: top_hat_58cm_black.jpg
    references:
        -   name: top_hat_58cm_black_image_content_id
            type: content_id

-   type: product_asset
    mode: create
    product_code: top_hat__58__black
    uri: '### "ezcontent://"~reference("top_hat_58cm_black_image_content_id") ###'
    tags: []

This migration uses a reference to store the created image Content ID, and then uses it while creating the asset. It uses an expression syntax to concat (~) the mandatory scheme ezcontent:// and the image content ID through the reference function used on the reference's name.

Product prices

The following example shows how to create a price for a product identified by its code:

1
2
3
4
5
6
7
8
9
-   type: product_price
    mode: create
    product_code: top_hat__58
    currency_code: 'EUR'
    amount: 120
    custom_prices:
        -   customer_group: contractors
            base_amount: 120
            custom_amount: 100

Customer groups

The following example shows how to create a customer group with a defined global price discount:

1
2
3
4
5
6
-   type: customer_group
    mode: create
    identifier: contractors
    names:
        eng-GB: Contractors
    global_price_rate: -20.0

Currencies

The following example shows how to create a currency:

1
2
3
4
5
-   type: currency
    mode: create
    code: TST
    subunits: 3
    enabled: true # default, optional

Commerce

Payment methods

The following example shows how to create a payment method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
-   type: payment_method
    mode: create
    paymentType: online
    name:
        -
            languageCode: eng-GB
            value: PaymentMethodName
        -
            languageCode: ger-DE
            value: PaymentMethodNameGerman
    identifier: PaymentMethodIdentifier
    enabled: true
    description:
        -
            languageCode: eng-GB
            value: PaymentMethodDescription
        -
            languageCode: ger-DE
            value: PaymentMethodDescriptionGerman

Shipping methods

The following example shows how to create a shipping method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
- type: shipping_method
  mode: create
  name:
      -
          languageCode: eng-GB
          value: TestName
  description:
      -
          languageCode: eng-GB
          value: TestDescription
  vatCategory: standard
  identifier: TestIdentifier
  enabled: false
  shippingType: free
  options:
      currency: 1
      price: '12.34'
  regions:
      - germany

- type: shipping_method
  mode: create
  name:
      -
          languageCode: eng-GB
          value: TestName
      -
          languageCode: ger-DE
          value: TestNameGerman
  description:
      -
          languageCode: eng-GB
          value: TestDescription
      -
          languageCode: ger-DE
          value: TestDescriptionGerman
  vatCategory: standard
  identifier: TestIdentifier2
  shippingType: flat_rate
  options:
      price: '12.34'
      currency: 'PLN'
  regions:
      - germany
      - france

Segments

The following example shows how to create a segment group and add segments in it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
-
    type: segment_group
    mode: create
    name: 'Contractors'
    identifier: contractors
    references:
        -
            name: contractors_group_id
            type: segment_group_id

-
    type: segment
    mode: create
    name: 'Painter'
    identifier: painter
    group:
        identifier: contractors

When updating a segment group or segment, you can match the object to update by using its numerical ID or identifier:

1
2
3
4
5
6
-
    type: segment
    mode: update
    name: 'Painter and Finish'
    matcher:
        identifier: painter

Settings

The following example shows how you can create and update a setting stored in the database:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
- type: setting
  mode: create
  group: test
  identifier: my_setting
  value:
      first: first_value
      second: second_value

- type: setting
  mode: update
  group: test
  identifier: my_setting
  value:
    first: first_value_modified

Taxonomies

The following example shows how you can create a "Car" tag in the main Taxonomy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
-   type: content
    mode: create
    metadata:
        contentType: tag
        mainTranslation: eng-GB
        alwaysAvailable: true
        section:
            identifier: taxonomy
    location:
        parentLocationRemoteId: taxonomy_tags_folder
    fields:
        -   fieldDefIdentifier: name
            languageCode: eng-GB
            value: Car
        -   fieldDefIdentifier: identifier
            languageCode: eng-GB
            value: car
        -   fieldDefIdentifier: parent
            languageCode: eng-GB
            value:
                taxonomy_entry_identifier: root

The field identifiers must match the identifiers used in the ibexa_taxonomy configuration file.

If the content type associated with the tags is changed, the configuration should be adjusted when creating migrations.

Note

If there are multiple taxonomies, the taxonomy field is then necessary here (line 21).

You can use the following example to assign tags to a Content (content type Article has an additional Field):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
-   type: content
    mode: create
    metadata:
        contentType: article
        mainTranslation: eng-GB
        alwaysAvailable: false
        section:
            identifier: standard
    location:
        parentLocationId: 42
    fields:
        -   fieldDefIdentifier: title
            languageCode: eng-GB
            value: Test1
        -   fieldDefIdentifier: short_title
            languageCode: eng-GB
            value: test1
        -   fieldDefIdentifier: author
            languageCode: eng-GB
            value:
                -
                    id: '1'
                    name: 'Administrator User'
                    email: [email protected]
        -   fieldDefIdentifier: intro
            languageCode: eng-GB
            value:
                xml: |
                    <?xml version="1.0" encoding="UTF-8"?>
                    <section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" 
                    xmlns:ezxhtml="http://ibexa.co/xmlns/dxp/docbook/xhtml" xmlns:ezcustom="http://ibexa.co/xmlns/dxp/docbook/custom" 
                    version="5.0-variant ezpublish-1.0"><para>test</para></section>
        -   fieldDefIdentifier: enable_comments
            languageCode: eng-GB
            value: false
        -   fieldDefIdentifier: field_631b169ec8035
            languageCode: eng-GB
            value:
                taxonomy_entries_identifiers:
                    - car
                    - plane
                taxonomy: tags

When updating a content type, use:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
-   type: content_type
    mode: update
    match:
        field: content_type_identifier
        value: article
    fields:
        -   identifier: field_631b169ec8035
            type: ibexa_taxonomy_entry_assignment
            position: 7
            translations:
                eng-GB:
                    name: tag
                    description: 'Tagi'
            required: false
            searchable: true
            infoCollector: false
            translatable: true
            category: content
            defaultValue:
                taxonomy_entries: {  }
                taxonomy: null
            fieldSettings:
                taxonomy: tags
            validatorConfiguration: {  }

Criteria

When using update or delete modes, you can use criteria to identify the objects to operate on.

Caution

Criteria only work with objects related to the product catalog.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
type: currency
mode: update
criteria:
    type: field_value
    field: code
    value: EUR
    operator: '=' # default
code: EEE
subunits: 3
enabled: false

Available operators are:

  • =
  • <>
  • <
  • <=
  • >
  • >=
  • IN
  • NIN
  • CONTAINS
  • STARTS_WITH
  • ENDS_WITH

You can combine criteria by using logical criteria and and or:

1
2
3
4
5
6
7
8
9
type: or
criteria:
    -   type: field_value
        field: code
        value: EUR
    -   type: field_value
        field: code
        value: X
        operator: STARTS_WITH

Criteria can be nested.