In this article, I'm going to teach you about pre-commit hooks, how to set them up using pre-commit
, and how to automate the whole process of keeping your repo in check using GitLab CI.
At the end of this article, you'll be able to:
- describe what pre-commit hooks are and why they are useful
- configure pre-commit hooks on your own projects using
git
- make sure all contributions to your projects conform to your hooks, fully automated, using GitLab CI
pre-commit hooks, in general
As you might guess from the name, pre-commit hooks are a way to add hooks that run before the commit process. They're a great way to make sure your code is ready to be committed. You might have a hook to auto-format your code, do linting, or maybe even running test suites -- all the things you should be doing before every commit, pre-commit hooks help you do them automatically.
Pre-commit hooks are a feature of git. So, if you use git
, you can use pre-commit hooks.
pre-commit, the tool in action
Now that we know about pre-commit hooks, let's see it in action. For this, we'll be using a tool called pre-commit
. Although pre-commit
shares the same name as the git
feature, it is a separate tool that we can use to easily setup and run pre-commit pre-made hooks for your git repository. Here, we'll cover the usage quick and dirty. If you like what you're about to see, you should definitely visit pre-commit.com for more detail.
Your first pre-commit hook
The basic steps to use pre-commit
are:
- Install pre-commit (one time only)
- Make a pre-commit configuration and add it to your repo
- enjoy automated checks and fixes every time you
git commit
.
Install pre-commit
Install pre-commit
. check the installation instructions for platform-specific instructions for your platform. When you can run pre-commit --help
you're ready to move on!
Let's take this moment to also setup the demo repo:
mkdir my-project
cd my-project
git init
pre-commit install # initialize the git hook
Make a pre-commit configuration file
The pre-commit config file (.pre-commit-config.yaml
) defines what hooks will run. While you can write your own scripts, one of the big advantages of pre-commit
is the ecosystem of available hooks made by other people. No need to reinvent the wheel!
The config file will define, minimally, one or more repos (git repository URLs) containing one or more hooks, and the name of the hook(s) you want to enable.
Here's a simple starter config file:
# .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.1.0 # this is optional, use `pre-commit autoupdate` to get the latest rev!
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
This configuration defines three pre-commit hooks: check-yaml
, end-of-file-fixer
, and trailing-whitespace
. These particular hooks are sourced from the pre-commit-hooks repository (the specified repo:
) and are authored by the pre-commit team. There is a wide ecosystem of hooks that can be used, but we'll just be using these for now.
Make sure you git add
your pre-commit config!
git add .pre-commit-config.yaml
Leverage pre-commit in your development workflow
We're now ready to put our pre-commit configuration to work. This will happen automatically any time we run git commit
. Let's see it in action:
I'll create a README.md
with trailing whitespace -- something our pre-commit hook should fix.
echo "Hello World " > README.md
git add README.md
git commit -m "initial commit"
You should see your pre-commit hooks run (you might also see pre-commit initialize these hooks if this is your first time running them).
The output will be something like this.
$ git commit -m "intital commit"
check yaml...............................................................Passed
fix end of files.........................................................Passed
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook
Fixing README.md
Note how the trim trailing whitespace
hook has Failed
status (as we expected). Also notice the message: files were modified by this hook
and Fixing README.md
.
If we check our README.md
file, we'll see that the trailing whitespace has been deleted. pre-commit
has done all the fixes for us. We can use git diff
to see the autofix changes. Now we can add the changes and commit again:
git add README.md
git commit -m "initial commit"
Now you'll see all the pre-commit hooks pass and the commit is successful!
Using pre-commit collaboratively
Now, one down-side of pre-commit hooks is that they run exclusively on your system. If a contributor to your project does not have the same pre-commit hooks installed, they may commit code that will violate your pre-commit hooks. Not good!
Perhaps at your company, you may be able to make sure all your coworkers setup pre-commit
before contributing, this is not really a scalable approach for open source projects which may have dozens or hundreds of contributors. There are still other issues, for example, if someone makes changes directly on the SCM server (for example, using the GitLab Web IDE or in the github.com file editor)
We can help alleviate this problem by applying pre-commit
to our CI process. A fantastic CI service, pre-commit.ci exists for this purpose. If you use GitHub, this is a great option. However, this service is not available for GitLab.
Enabling pre-commit auto-fixes in GitLab
For GitLab CI users, we can make sure that:
- When changes are committed, a job runs to check whether the pre-commit hooks were properly applied.
- When a pre-commit validation fails, we attempt to apply automatic fixes and commit those fixes automatically.
The steps for this are as follows:
- Create a project access token and create a CI/CD variable
PRE_COMMIT_ACCESS_TOKEN
with the token as a value (make sure this token is masked!) - Add the CI configuration for pre-commit. The easiest way to do this is to add an include pointing to a template I've made so you can try this out.
You might use a .gitlab-ci.yml
file that looks like this:
include:
- remote: 'https://gitlab.com/yesolutions/gitlab-ci-templates/raw/main/templates/pre-commit-autofix.yaml'
variables:
# since we're not using merge request pipelines in this example,
# we will configure the pre-commit job to run on branch pipelines only.
# If you ARE using merge request pipelines, you can omit this section
PRE_COMMIT_DEDUPLICATE_MR_AND_BRANCH: "false"
PRE_COMMIT_AUTO_FIX_BRANCH_ONLY: "true"
test:
script:
- echo "your usual tests go here"
You can check the template repository for all the configuration options, or even just copy the template file from there.
Now all your commits and contributions will be checked and auto-fixed!
Example
Let's see this work in example.
Here, I'll make a non-compliant change to my repository using the GitLab web IDE:
We can see the initial pipeline failed (as expected, since we intentionally used trailing whitespace, which will fail the trailing-whitespace
hook).
Additionally, we see a new commit was made (by the CI job) where auto-fixes were applied and the new pipeline that was created as a result passes!
Taking a closer look at the commit, we can see the changes applied by the CI job:
So there you have it. That's how you can use GitLab CI to automate your pre-commit workflow.
repository write
accessvariables
config (e.g.PRE_COMMIT_DEDUPLICATE_MR_AND_BRANCH: "false"
) instead of booleans. Otherwise, it will have "variables config should be a hash of key value pairs, value can be a hash" error.