Migrating projects using file exports (FREE)

Existing projects on any self-managed GitLab instance or GitLab.com can be exported to a file and then imported into a new GitLab instance. You can also:

• Migrate groups using the preferred method.
• Migrate groups using file exports.

Set up project import/export

Before you can import or export a project and its data, you must set it up.

1. On the left sidebar, select Settings > General.
2. Expand Visibility and access controls.
3. Scroll to Import sources.
4. Enable the desired Import sources.

Between CE and EE

You can export projects from the Community Edition to the Enterprise Edition and vice versa. This assumes version history requirements are met.

If you're exporting a project from the Enterprise Edition to the Community Edition, you may lose data that is retained only in the Enterprise Edition. For more information, see downgrading from EE to CE.

Export a project and its data

Before you can import a project, you must export it.

Prerequisites:

• Review the list of items that are exported. Not all items are exported.
• You must have at least the Maintainer role for the project.

To export a project and its data, follow these steps:

1. On the top bar, select Main menu > Projects and find your project.
2. On the left sidebar, select Settings > General.
3. Expand Advanced.
4. Select Export project.
5. After the export is generated, you should receive an email with a link to download the file.
6. Alternatively, you can come back to the project settings and download the file from there or generate a new export. After the file is available, the page should show the Download export button.

The export is generated in your configured shared_path, a temporary shared directory, and then moved to your configured uploads_directory. Every 24 hours, a worker deletes these export files.

Items that are exported

The import_export.yml file lists the items exported and imported when migrating projects using file exports. View this file in the branch for your version of GitLab to see the list of items relevant to you. For example, import_export.yml on the 14-10-stable-ee branch.

Migrating projects with file exports uses the same export and import mechanisms as creating projects from templates at the group and instance levels. Therefore, the list of exported items is the same.

Items that are exported include:

• Project and wiki repositories
• Project uploads
• Project configuration, excluding integrations
• Issues
  • Issue comments
  • Issue resource state events (Introduced in GitLab 15.4)
  • Issue resource milestone events (Introduced in GitLab 15.4)
• Merge requests
  • Merge request diffs
  • Merge request comments
  • Merge request resource state events (Introduced in GitLab 15.4)
  • Merge request multiple assignees (Introduced in GitLab 15.3)
  • Merge request reviewers (Introduced in GitLab 15.3)
  • Merge request approvers (Introduced in GitLab 15.3)
• Labels
• Milestones
• Snippets
• Time tracking and other project entities
• Design Management files and data
• LFS objects
• Issue boards
• Pipelines history
• Push Rules
• Awards
• Group members are exported as project members, as long as the user has the Maintainer role in the exported project's group, or is an administrator

Items that are not exported include:

• Child pipeline history
• Build traces and artifacts
• Package and container registry images
• CI/CD variables
• Pipeline triggers
• Webhooks
• Any encrypted tokens
• Number of required approvals
• Repository size limits
• Deploy keys allowed to push to protected branches
• Secure Files

Import a project and its data

Default maximum import file size changed from 50 MB to unlimited in GitLab 13.8.

WARNING: Only import projects from sources you trust. If you import a project from an untrusted source, it may be possible for an attacker to steal your sensitive data.

Prerequisites:

• You must have exported the project and its data.
• Compare GitLab versions and ensure you are importing to a GitLab version that is the same or later than the GitLab version you exported to.
• Review the Version history for compatibility issues.

To import a project:

1. When creating a new project, select Import project.
2. In Import project from, select GitLab export.
3. Enter your project name and URL. Then select the file you exported previously.
4. Select Import project to begin importing. Your newly imported project page appears shortly.

To get the status of an import, you can query it through the Project import/export API. As described in the API documentation, the query may return an import error or exceptions.

Items that are imported

The following items are imported but changed slightly:

• Project members with the Owner role are imported as Maintainers.
• If an imported project contains merge requests originating from forks, then new branches associated with such merge requests are created in a project during the import/export. Thus, the number of branches in the exported project might be bigger than in the original project.
• If use of the Internal visibility level is restricted, all imported projects are given Private visibility.

Deploy keys aren't imported. To use deploy keys, you must enable them in your imported project and update protected branches.

Import large projects (FREE SELF)

If you have a larger project, consider using a Rake task as described in the developer documentation.

Automate group and project import (PREMIUM)

For information on automating user, group, and project import API calls, see Automate group and project import.

Maximum import file size

Administrators can set the maximum import file size one of two ways:

• With the max_import_size option in the Application settings API.
• In the Admin Area UI.

The default is 0 (unlimited).

For the GitLab.com setting, see the Account and limit settings section of the GitLab.com settings page.

Map users for import

Imported users can be mapped by their public email addresses on self-managed instances, if an administrator (not an owner) does the import.

• The project must be exported by a project or group member with the Owner role.
• Public email addresses are not set by default. Users must set it in their profiles for mapping to work correctly.
• For contributions to be mapped correctly, users must be an existing member of the namespace, or they can be added as a member of the project. Otherwise, a supplementary comment is left to mention that the original author and the merge requests, notes, or issues that are owned by the importer.
• Imported users are set as direct members in the imported project.

For project migration imports performed over GitLab.com groups, preserving author information is possible through a professional services engagement.

Rate Limits

To help avoid abuse, by default, users are rate limited to:

GitLab.com may have different settings from the defaults.

Version history

14.0+

In GitLab 14.0, the JSON format is no longer supported for project and group exports. To allow for a transitional period, you can still import any JSON exports. The new format for imports and exports is NDJSON.

13.0+

Starting with GitLab 13.0, GitLab can import bundles that were exported from a different GitLab deployment. This ability is limited to two previous GitLab minor releases, which is similar to our process for Security Releases.

For example:

12.x

Prior to 13.0 this was a defined compatibility table:

Projects can be exported and imported only between versions of GitLab with matching Import/Export versions.

For example, 8.10.3 and 8.11 have the same Import/Export version (0.1.3) and the exports between them are compatible.

Related topics

• Project import/export API
• Project import/export administration Rake tasks
• Group import/export
• Group import/export API

Troubleshooting

Project fails to import due to mismatch

If the shared runners enablement does not match between the exported project, and the project import, the project fails to import. Review issue 276930, and either:

• Ensure shared runners are enabled in both the source and destination projects.
• Disable shared runners on the parent group when you import the project.

Import workarounds for large repositories

Maximum import size limitations can prevent an import from being successful. If changing the import limits is not possible, you can try one of the workarounds listed here.

Workaround option 1

The following local workflow can be used to temporarily reduce the repository size for another import attempt:

1. Create a temporary working directory from the export:

   EXPORT=<filename-without-extension>
   
   mkdir "$EXPORT"
   tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/
   cd "$EXPORT"/
   git clone project.bundle
   
   # Prevent interference with recreating an importable file later
   mv project.bundle ../"$EXPORT"-original.bundle
   mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz
   
   git switch --create smaller-tmp-main

2. To reduce the repository size, work on this smaller-tmp-main branch: identify and remove large files or interactively rebase and fixup to reduce the number of commits.

   # Reduce the .git/objects/pack/ file size
   cd project
   git reflog expire --expire=now --all
   git gc --prune=now --aggressive
   
   # Prepare recreating an importable file
   git bundle create ../project.bundle <default-branch-name>
   cd ..
   mv project/ ../"$EXPORT"-project
   cd ..
   
   # Recreate an importable file
   tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ .

3. Import this new, smaller file into GitLab.

4. In a full clone of the original repository, use git remote set-url origin <new-url> && git push --force --all to complete the import.

5. Update the imported repository's branch protection rules and its default branch, and delete the temporary, smaller-tmp-main branch, and the local, temporary data.

Workaround option 2

NOTE: This workaround does not account for LFS objects.

Rather than attempting to push all changes at once, this workaround:

• Separates the project import from the Git Repository import
• Incrementally pushes the repository to GitLab

1. Make a local clone of the repository to migrate. In a later step, you push this clone outside of the project export.

2. Download the export and remove the project.bundle (which contains the Git repository):

   tar -czvf new_export.tar.gz --exclude='project.bundle' @old_export.tar.gz

3. Import the export without a Git repository. It asks you to confirm to import without a repository.

4. Save this bash script as a file and run it after adding the appropriate origin.

   #!/bin/sh
   
   # ASSUMPTIONS:
   # - The GitLab location is "origin"
   # - The default branch is "main"
   # - This will attempt to push in chunks of 500MB (dividing the total size by 500MB).
   #   Decrease this size to push in smaller chunks if you still receive timeouts.
   
   git gc
   SIZE=$(git count-objects -v 2> /dev/null | grep size-pack | awk '{print $2}')
   
   # Be conservative... and try to push 2GB at a time
   # (given this assumes each commit is the same size - which is wrong)
   BATCHES=$(($SIZE / 500000))
   TOTAL_COMMITS=$(git rev-list --count HEAD)
   if (( BATCHES > TOTAL_COMMITS )); then
       BATCHES=$TOTAL_COMMITS
   fi
   
   INCREMENTS=$(( ($TOTAL_COMMITS / $BATCHES) - 1 ))
   
   for (( BATCH=BATCHES; BATCH>=1; BATCH-- ))
   do
     COMMIT_NUM=$(( $BATCH - $INCREMENTS ))
     COMMIT_SHA=$(git log -n $COMMIT_NUM --format=format:%H | tail -1)
     git push -u origin ${COMMIT_SHA}:refs/heads/main
   done
   git push -u origin main
   git push -u origin --all
   git push -u origin --tags

Manually execute export steps

Exports sometimes fail without giving enough information to troubleshoot. In these cases, it can be helpful to open a rails console session and loop through all the defined exporters. Execute each line individually, rather than pasting the entire block at once, so you can see any errors each command returns.

# User needs to have permission to export
u = User.find_by_username('someuser')
p = Project.find_by_full_path('some/project')
e = Projects::ImportExport::ExportService.new(p,u)

e.send(:version_saver).send(:save)
e.send(:repo_saver).send(:save)
## continue using `e.send(:exporter_name).send(:save)` going through the list of exporters

# The following line should show you the export_path similar to /var/opt/gitlab/gitlab-rails/shared/tmp/gitlab_exports/@hashed/49/94/4994....
s = Gitlab::ImportExport::Saver.new(exportable: p, shared:p.import_export_shared)

# To try and upload use:
s.send(:compress_and_save)
s.send(:save_upload)

Import using the REST API fails when using a group access token

Group access tokens don't work for project or group import operations. When a group access token initiates an import, the import fails with this message:

Error adding importer user to Project members.
Validation failed: User project bots cannot be added to other groups / projects

To use Import REST API, pass regular user account credentials such as personal access tokens.