Using Turborepo's --affected flag in CI

It's common for jobs in a Gitlab CI pipeline to use git to checkout a commit in detached head mode instead of checking out a branch. In git terms, the "head" is a pointer that points to the current point in a repository. Running git checkout [branch-name] will checkout the branch in attached head mode; this is how we most frequently use git. When code is checked out in detached head mode, it means that the "head" is pointing to a specific commit instead of a branch. Basically, detached head mode means that git isn't currently on a branch, so there's no branch data available.

While often this works fine and (probably) goes unnoticed in Gitlab CI jobs, it can be problematic if you have code within a job that's trying to compare changes between branches. Turborepo's --affected flag is an example of this. Let's look at what the --affected flag does and how we can configure our Gitlab CI jobs to make sure it runs as expected during continuous integration.

What is the --affected flag?

Most Turborepo workspaces will contain multiple packages. While you may want to run CI jobs for all packages in a pipeline, it's more likely that you want to be able to run CI jobs only on packages that have actually changed. By only running jobs on packages that have changed, jobs will run faster and you can avoid unnecessary package publishes, for example.

The --affected flag is added to the turbo commands in your turbo.json file and tells Turbo to only run those commands on packages that have changed. When you run it on your local computer (or in any environment in attached head mode), Turbo will by default compare the checked-out branch to the base branch you've configured (origin/main, for example). The turbo command will run only for packages that have changed between those two branches. The Turborepo documentation has more examples.

Why doesn't the default --affected behavior work in Gitlab CI?

Since our Gitlab CI pipelines run in detached head mode, the default branch comparison behavior won't work. Turbo won't be able to determine what changed because it won't have branches to compare, so it will run the turbo commands for all of the packages in the monorepo.

Configuring Turbo environment variables in Gitlab CI

Turborepo has two environment variables that you can set to tell it which branches to compare when using the --affected flag: TURBO_SCM_BASE and TURBO_SCM_HEAD. These flags work like the filter command, --filter=[TURBO_SCM_BASE...TURBO_SCM_HEAD], meaning that Turborepo will compare changes from the base branch to the head. We can generally leave the TURBO_SCM_HEAD variable alone as long as we want to compare changes up to the most recent commit. It's the TURBO_SCM_BASE variable that we're most likely to want to change.

These variables can be set in the scripts within the package.json file if you want to hardcode them, but for more flexibility, you can dynamically set them in your .gitlab-ci.yml file.

There are two scenarios where we need to configure the base commit in our Gitlab CI pipelines:

• During merge request events (i.e., as you push changes to an unmerged merge request)
• During merges to your base branch (i.e., merging changes to origin/main)

Configuring the Base Commit for Merge Request Events

Before any job that uses a turbo --affected command, we'll want to run a before_script that exports the TURBO_SCM_BASE environment variable set to our base branch (origin/main in the example below). The before_script should also install git and use git to fetch the base branch, so Turbo has it available for comparison purposes. Here's an example:

1# .gitlab-ci.yml file
2
3image: node:22
4
5.set_turbo_base:
6  before_script:
7    # Tells Turbo which branch to compare your changes to
8    - export TURBO_SCM_BASE="origin/main" # replace origin/main with your base branch
9
10    # Install git
11    - npm install -g git
12
13    # Make sure that the base branch is available for Turbo to compare to
14    - git fetch origin main # replace origin main with your base branch
15  rules:
16    # Runs on merge requests
17    - if: "$CI_PIPELINE_SOURCE == 'merge_request_event'"
18      when: always
19
20
21# Example job that uses the .set_turbo_base job
22build:
23  extends:
24    - .set_turbo_base # The before_script above will run before the script below
25  script:
26    - npm ci
27    - npm run build
Copy to clipboard

Configuring the Base Commit for Merges to the Base Branch

The configuration above will work for jobs that run in a merge request pipeline, but there's a non-obvious issue when merging the branch into your base branch. Once your changes are merged into the base branch, the .set_turbo_base job above will compare your changes... to themselves!

When merging to the base branch, we'll want to compare the base branch to the previous commit on the base branch to detect changes. To get the previous commit on the base branch, we can run $(git rev-parse $(git rev-parse origin/main)^). The rules on the jobs below will determine which job runs.

1# .gitlab-ci.yml file
2
3image: node:22
4
5.set_turbo_base_mr:
6  before_script:
7    # Tells Turbo which branch to compare your changes to
8    - export TURBO_SCM_BASE="origin/main" # replace origin/main with your base branch
9
10    # Install git
11    - npm install -g git
12
13    # Make sure that the base branch is available for Turbo to compare to
14    - git fetch origin main # replace origin main with your base branch
15  rules:
16    # Runs on merge requests
17    - if: "$CI_PIPELINE_SOURCE == 'merge_request_event'"
18      when: always
19
20.set_turbo_base_main:
21  before_script:
22    # Tells Turbo which branch to compare your changes to
23    - export TURBO_SCM_BASE=$(git rev-parse $(git rev-parse origin/main)^) # replace origin/main with your base branch
24
25    # Install git
26    - npm install -g git
27
28    # Make sure that the base branch is available for Turbo to compare to
29    - git fetch origin main # replace origin main with your base branch
30  rules:
31    # Runs when you merge to your base branch
32    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
33      when: always
34
35# Example job that uses the .set_turbo_base job
36build:
37  extends:
38    - .set_turbo_base_mr # extends both jobs above but they run based on the rules of each
39    - .set_turbo_base_main
40  script:
41    - npm ci
42    - npm run build
Copy to clipboard

Your Gitlab CI configuration is probably much more complex than these examples, but hopefully, this is a good starting point for configuring turbo --affected commands in your pipeline.