Using Git Submodules With Gitlab CI/CD

We use Git submodules to keep a Git repository as a subdirectory of another Git repository.  This is usually done when you have a set of common files you want to refer to in multiple repos.

In my personal experience, I found git submodules especially useful when working with Terraform.

Configure the .gitmodules File

When you use Git submodules, your project should have a file named .gitmodules. 

To create .gitmodules, you need to use the git submodule command. There are two ways you can use it:

• If the repo you want to use as a submodule exists on a relative path, i.e., on the same GitLab server, then use the following command:

x

$ git submodule add ../chaconinc/project-y.git

Cloning into 'project-y'...

remote: Counting objects: 11, done.

remote: Compressing objects: 100% (10/10), done.

remote: Total 11 (delta 0), reused 11 (delta 0)

Unpacking objects: 100% (11/11), done.

Checking connectivity... done.

• For submodules not located on the same GitLab server, use the full URL:  

xxxxxxxxxx

$ git submodule add https://gitserver.com/group/project-y.git

Cloning into 'project-y'...

remote: Counting objects: 11, done.

remote: Compressing objects: 100% (10/10), done.

remote: Total 11 (delta 0), reused 11 (delta 0)

Unpacking objects: 100% (11/11), done.

Checking connectivity... done.

Both the above commands should be executed from within the project in which you want to use 'project-y' as a submodule.

Updating gitlab-ci.yml, Use Git Submodules in CI/CD Jobs

To make submodules work correctly in CI/CD jobs:

1. Make sure you use relative URLs for submodules located in the same GitLab server.

2. You can set the GIT_SUBMODULE_STRATEGY variable to either normal or recursive.

In the variables section of your gitlab-ci file, add the following:

xxxxxxxxxx

variables:

  GIT_SUBMODULE_STRATEGY: recursive 

We used the recursive option so that all folders inside submodules are checked out. 

Next, in the default section of our gitlab-ci file, we had to add the following to make sure the latest version of submodules is checked out on every execution:

Adding tags, setting 'sslVerify' is optional, but we had to add submodule init and update with --recursive --remote to make sure submodules work properly.

x

default:

  tags:

    - <your runner Name>

  before_script:

    - git config --global http.sslVerify "false"

    - git submodule init

    - git submodule update --recursive --remote

Once the above lines were added, submodules for us worked seamlessly in GitLab.