davit.tech

pre-commit hook for Django project

25, May 2020 - 6 min read

Introduction ☜

In Python projects, we often encounter code style issues. They can be prevented by automating code style checks before someone tries to push them to a shared repository․

For demonstration purposes, we are going to create a simple Django project:

/somewhere/$ django-admin startproject pre_commit

and initialize a git repository in it:

/somewhere/$ cd pre_commit
/somewhere/pre_commit/$ git init
Initialized empty Git repository in /somewhere/pre_commit/.git/

1st step of this automation will be creating and enabling the pre-commit git hook.


pre-commit ☜

Git has a way to fire off custom scripts when certain important actions occur. pre-commit client-side hook is one of them. Git runs the hook script first before you even type in a commit message. A good place to inspect the code in files that are about to be committed.

Hooks are all stored in the hooks subdirectory of the Git directory. In most projects, that’s .git/hooks. By default, there are some sample hook files written as shell scripts, but any properly named executable script will do the job.

In our case, we will create an executable file called pre-commit under .git/hooks directory.

/somewhere/pre_commit/$ touch .git/hooks/pre-commit
/somewhere/pre_commit/$ chmod +x .git/hooks/pre-commit

To understand how it works, let's add the following into the hook script:

.git/hooks/pre-commit
echo "Hook failed!"
exit 1

If we run git status, we will see some untracked files that are generated by django-admin command. Let's stage them and try to do the first commit:

/somewhere/pre_commit/$ git add .
/somewhere/pre_commit/$ git commit
Hook failed!

We saw the message Hook failed! right after committing. And we were unable to type a message for the commit. We were not able to do a commit. The reason for that is the hook script exited with the status 1 instead of 0 which is the number that is used to indicate the successful termination of a script.

If we replace 1 with 0 we will be able to commit:

.git/hooks/pre-commit
echo "Hook succeded!"
exit 0
/somewhere/pre_commit/$ git add .
/somewhere/pre_commit/$ git commit
Hook succeded!
1 
2 # Please enter the commit message for your changes. Lines starting
3 # with '#' will be ignored, and an empty message aborts the commit.
4 #
5 # On branch master
6 # Changes to be committed:
7 #   added:   somefile.py
8 #

Now that we know how pre-commit hook works, we need to write a command that will check the code style of python files that we work on and exit with the non-zero(1) status if the check fails or with the zero status otherwise.


flake8 ☜

flake8 is a tool for style guide enforcement. We are going to use it for code style checks. We can install it as a python package using pip:

/somewhere/pre_commit/$ pip install flake8

To start using flake8, we just need to type it as a command and pass a path to a file or directory. If we pass settings.py file generated by Django we will see this:

/somewhere/pre_commit/$ flake8 pre_commit/settings.py
pre_commit/settings.py:90:80: E501 line too long (91 > 79 characters)
pre_commit/settings.py:93:80: E501 line too long (81 > 79 characters)
pre_commit/settings.py:96:80: E501 line too long (82 > 79 characters)
pre_commit/settings.py:99:80: E501 line too long (83 > 79 characters)

Notice that all 4 issues that we see are about the lengths of some lines in settings.py file. This is the default file generated using django-admin startproject command. The problem is that Django uses different number(119) as maximum allowed length of line compared with the number defined in PEP-8 which is 79 characters.

Fortunatelly, flake8 allows us to configure this maximum allowed length value in a file called setup.cfg:

/somewhere/pre_commit/setup.cfg
[flake8]
max-line-length = 119

Now 119 is the maximum allowed length of lines for our project. Running check again:

/somewhere/pre_commit/$ flake8 pre_commit/settings.py
/somewhere/pre_commit/$

and the output is empty. 119 is the maximum allowed length for the Django framework source code, so it's a good idea to use the same number in Django projects.

Until now, we did a code style check on a single file, but we would like to run the check on all staged python files. The final command is:

/somewhere/pre_commit/$ flake8 $(git status -s | grep -E '\.py$' | cut -c 4-)

The reason is that the project might grow and we might have many files there and we want to avoid checking all of them, instead, we want to check only those which are being changed.

In some cases we would also need to check code style on all files that we wrote by running flake8 from the root directory of the project. But there might be folders or files that we need to exclude. Common examples are .venv, .git directories:

/somewhere/pre_commit/setup.cfg
[flake8]
exclude = .venv,.git
max-line-length = 119

We would also like to exclude all */migrations/* directories, because sometimes the auto-generated migration files in these directories are not PEP-8 compliant:

/somewhere/pre_commit/setup.cfg
[flake8]
exclude = .venv,.git,*/migrations/*
max-line-length = 119

Now, that the final flake8 command is ready we need to figure out how to connect it with the pre-commit hook.

The echo $? command shows the exit status of the previous executed command. With the help of it, we can see what's the status of the flake8 command if there are not code styles issues and the status in case of code style issues:

/somewhere/pre_commit/$ flake8 pre_commit/settings.py
/somewhere/pre_commit/$ echo $?
0

no issues - zero exit. Let's force flake8 to warn us about some errors by temporarily commenting the max-line-length param in setup.cfg file:

/somewhere/pre_commit/setup.cfg
[flake8]
...
# max-line-length = 119

now let's run again:

/somewhere/pre_commit/$ flake8 pre_commit/settings.py
pre_commit/settings.py:90:80: E501 line too long (91 > 79 characters)
pre_commit/settings.py:93:80: E501 line too long (81 > 79 characters)
pre_commit/settings.py:96:80: E501 line too long (82 > 79 characters)
pre_commit/settings.py:99:80: E501 line too long (83 > 79 characters)
/somewhere/pre_commit/$ echo $?
1

issues - non-zero exit: perfect for putting it into the pre-commit git hook file:

/somewhere/pre_commit/.git/hooks/pre-commit
flake8 $(git status -s | grep -E '\.py$' | cut -c 4-)

Now, whenever our flake8 command exits with a non-zero status, git won't allow us to continue committing.


isort ☜

Another problem with the code style is import orders. isort comes to help us with that. It is a Python utility/library to sort imports alphabetically and automatically separated into sections and by type.

According to Django's official docs in Django project the preferred import order is:

# future
from __future__ import unicode_literals

# standard library
import json
from itertools import chain

# third-party
import bcrypt

# Django
from django.http import Http404
from django.http.response import (
    Http404, HttpResponse, HttpResponseNotAllowed, StreamingHttpResponse,
    cookie,
)

# local Django
from .models import LogEntry

# try/except
try:
    import yaml
except ImportError:
    yaml = None

CONSTANT = 'foo'


class Example:
    # ...
  1. future
  2. standard library
  3. third-party
  4. Django
  5. local Django
  6. try/except

Installing isort is as simple as:

/somewhere/pre_commit/$ pip install isort

and running is very similar to flake8:

/somewhere/pre_commit/$ isort pre_commit/settings.py
/somewhere/pre_commit/$ echo $?
0

no issues with the file and the command finished with the zero status code.

We can configure parameters for isort in the same file as we did for flake8:

/somewhere/pre_commit/setup.cfg
[flake8]
exclude = .venv,.git,*/migrations/*
max-line-length = 119

[isort]
sections = FUTURE,STDLIB,THIRDPARTY,LOCALFOLDER

We can change the section order with sections option.

We also can define our own sections and their order.

/somewhere/pre_commit/setup.cfg
[flake8]
...

[isort]
known_django=django
sections = FUTURE,STDLIB,THIRDPARTY,DJANGO,LOCALFOLDER

It's important to mention that isort command by default checks and applies changes to your files immideatelly and if you want to force it to check and warn only(similar to flake8) you should pass --diff to it:

/somewhere/pre_commit/$ isort --diff pre_commit/settings.py

To run the command only over the staged files as we did with flake8, we add the following to the command:

/somewhere/pre_commit/$ isort --diff $(git status -s | grep -E '\.py$' | cut -c 4-)

This command is also ready to be moved into the hook:

/somewhere/pre_commit/.git/hooks/pre-commit
STAGED_FILES = $(git status -s | grep -E '\.py$' | cut -c 4-)
flake8 $STAGED_FILES
isort --diff $STAGED_FILES

Conclusion ☜

Now, the hook is ready to be used, but the problem is that it works only on our machine. One way of sharing this hook with our teammates is starting tracking it with git by moving into the root directory of our project and committing to a shared repository:

/somewhere/pre_commit/$ mv .git/hooks/pre-commit .

and after pushing it to the remote repository we guide our teammates to install it using symbolic links:

/somewhere/pre_commit/$ ln -s ../../pre-commit .git/hooks/pre-commit

This approach allows us to modify the hook later and make it work on all connected machines without doing any additional operation.

This might interest you.