Skip to main content
1183

How to use pre-commit to automatically correct commits and merge requests with GitLab CI

Created
Active
Last edited
Viewed 66k times
4 min read
80

In this article, I'm going to teach you about pre-commit hooks, how to set them up using pre-commit, and how to automate the whole process of keeping your repo in check using GitLab CI.

At the end of this article, you'll be able to:

  • describe what pre-commit hooks are and why they are useful
  • configure pre-commit hooks on your own projects using git
  • make sure all contributions to your projects conform to your hooks, fully automated, using GitLab CI

pre-commit hooks, in general

As you might guess from the name, pre-commit hooks are a way to add hooks that run before the commit process. They're a great way to make sure your code is ready to be committed. You might have a hook to auto-format your code, do linting, or maybe even running test suites -- all the things you should be doing before every commit, pre-commit hooks help you do them automatically.

Pre-commit hooks are a feature of git. So, if you use git, you can use pre-commit hooks.

pre-commit, the tool in action

Now that we know about pre-commit hooks, let's see it in action. For this, we'll be using a tool called pre-commit. Although pre-commit shares the same name as the git feature, it is a separate tool that we can use to easily setup and run pre-commit pre-made hooks for your git repository. Here, we'll cover the usage quick and dirty. If you like what you're about to see, you should definitely visit pre-commit.com for more detail.

Your first pre-commit hook

The basic steps to use pre-commit are:

  1. Install pre-commit (one time only)
  2. Make a pre-commit configuration and add it to your repo
  3. enjoy automated checks and fixes every time you git commit.

Install pre-commit

Install pre-commit. check the installation instructions for platform-specific instructions for your platform. When you can run pre-commit --help you're ready to move on!

Let's take this moment to also setup the demo repo:

mkdir my-project
cd my-project
git init
pre-commit install # initialize the git hook

Make a pre-commit configuration file

The pre-commit config file (.pre-commit-config.yaml) defines what hooks will run. While you can write your own scripts, one of the big advantages of pre-commit is the ecosystem of available hooks made by other people. No need to reinvent the wheel!

The config file will define, minimally, one or more repos (git repository URLs) containing one or more hooks, and the name of the hook(s) you want to enable.

Here's a simple starter config file:

# .pre-commit-config.yaml
repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.1.0  # this is optional, use `pre-commit autoupdate` to get the latest rev!
    hooks:
    -   id: check-yaml
    -   id: end-of-file-fixer
    -   id: trailing-whitespace

This configuration defines three pre-commit hooks: check-yaml, end-of-file-fixer, and trailing-whitespace. These particular hooks are sourced from the pre-commit-hooks repository (the specified repo:) and are authored by the pre-commit team. There is a wide ecosystem of hooks that can be used, but we'll just be using these for now.

Make sure you git add your pre-commit config!

git add .pre-commit-config.yaml

Leverage pre-commit in your development workflow

We're now ready to put our pre-commit configuration to work. This will happen automatically any time we run git commit. Let's see it in action:

I'll create a README.md with trailing whitespace -- something our pre-commit hook should fix.

echo "Hello World   " > README.md
git add README.md
git commit -m "initial commit"

You should see your pre-commit hooks run (you might also see pre-commit initialize these hooks if this is your first time running them).

The output will be something like this.

$ git commit -m "intital commit"
check yaml...............................................................Passed
fix end of files.........................................................Passed
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing README.md

Note how the trim trailing whitespace hook has Failed status (as we expected). Also notice the message: files were modified by this hook and Fixing README.md.

If we check our README.md file, we'll see that the trailing whitespace has been deleted. pre-commit has done all the fixes for us. We can use git diff to see the autofix changes. Now we can add the changes and commit again:

git add README.md
git commit -m "initial commit"

Now you'll see all the pre-commit hooks pass and the commit is successful!

Using pre-commit collaboratively

Now, one down-side of pre-commit hooks is that they run exclusively on your system. If a contributor to your project does not have the same pre-commit hooks installed, they may commit code that will violate your pre-commit hooks. Not good!

Perhaps at your company, you may be able to make sure all your coworkers setup pre-commit before contributing, this is not really a scalable approach for open source projects which may have dozens or hundreds of contributors. There are still other issues, for example, if someone makes changes directly on the SCM server (for example, using the GitLab Web IDE or in the github.com file editor)

We can help alleviate this problem by applying pre-commit to our CI process. A fantastic CI service, pre-commit.ci exists for this purpose. If you use GitHub, this is a great option. However, this service is not available for GitLab.

Enabling pre-commit auto-fixes in GitLab

For GitLab CI users, we can make sure that:

  1. When changes are committed, a job runs to check whether the pre-commit hooks were properly applied.
  2. When a pre-commit validation fails, we attempt to apply automatic fixes and commit those fixes automatically.

The steps for this are as follows:

  1. Create a project access token and create a CI/CD variable PRE_COMMIT_ACCESS_TOKEN with the token as a value (make sure this token is masked!)
  2. Add the CI configuration for pre-commit. The easiest way to do this is to add an include pointing to a template I've made so you can try this out.

You might use a .gitlab-ci.yml file that looks like this:


include:
  - remote: 'https://gitlab.com/yesolutions/gitlab-ci-templates/raw/main/templates/pre-commit-autofix.yaml'

variables:
  # since we're not using merge request pipelines in this example,
  # we will configure the pre-commit job to run on branch pipelines only.
  # If you ARE using merge request pipelines, you can omit this section
  PRE_COMMIT_DEDUPLICATE_MR_AND_BRANCH: "false"
  PRE_COMMIT_AUTO_FIX_BRANCH_ONLY: "true"

test:
  script:
    - echo "your usual tests go here" 

You can check the template repository for all the configuration options, or even just copy the template file from there.

Now all your commits and contributions will be checked and auto-fixed!

Example

Let's see this work in example.

Here, I'll make a non-compliant change to my repository using the GitLab web IDE:

adding a change

We can see the initial pipeline failed (as expected, since we intentionally used trailing whitespace, which will fail the trailing-whitespace hook).

Additionally, we see a new commit was made (by the CI job) where auto-fixes were applied and the new pipeline that was created as a result passes!

failed pipelines and fix pipelines

Taking a closer look at the commit, we can see the changes applied by the CI job:

fix commit

So there you have it. That's how you can use GitLab CI to automate your pre-commit workflow.

References

  • 38.8k
  • 8
  • 60
  • 110
10
  • Thanks for writing and sharing this, I'm trying it out on non-SaaS enterprise repository and don't think its possible to create a Project Access Token if I've understood the docs. Is it possible to us the AUTO_FIX aspect with group or personal tokens?
    – slackline
    Commented Jul 31, 2022 at 19:09
  • 1
    Featured in a meta post. Commented Mar 2, 2023 at 4:32
  • 1
    As per the documentation in gitlab.com/yesolutions/gitlab-ci-templates, token must have repository write access
    – bluesmonk
    Commented Jun 2, 2023 at 6:27
  • 2
    Thanks for your great article. I just want to mention that the the CI job uses a docker container managed by the author: gitlab.com/yesolutions/docker-pre-commit It could be a good idea to build the container on your system and use yours instead. Commented Oct 31, 2023 at 9:20
  • Thanks! is it possible to have local precommit silently apply fixes and commit? atm i hit commit, it runs, finds a fail, does not commit, and assuming fixable fail i just hit commit again... if its fixable i just want it fixed silently...
    – prosody
    Commented Dec 30, 2023 at 18:39
  • @prosody check out this answer. The short answer seems to be 'no, but you can implement that behavior yourself'. In the linked GitLab CI templates, there is some scripting logic to determine that scenario (that an auto fix completely fixed the problem, then pushes) I'm not sure if that's the best way, but it has been working for me.
    – sytech
    Commented Jul 10 at 12:35
  • "This configuration defines two pre-commit hooks: check-yaml and end-of-file-fixer. These hooks are found in the primary...." Did this paragraph get cut off or do I just not know what a 'primary' is? :D
    – momo
    Commented Aug 12 at 16:35
  • 1
    @momo seems like it was just an unfinished thought, fixed :)
    – sytech
    Commented Aug 12 at 17:07
  • Turned out that it needs to use string values in the variables config (e.g. PRE_COMMIT_DEDUPLICATE_MR_AND_BRANCH: "false") instead of booleans. Otherwise, it will have "variables config should be a hash of key value pairs, value can be a hash" error. Commented Aug 19 at 22:23