In Python projects, we often encounter code style issues. They can be prevented by automating code style checks before someone tries to push them to a shared repository․
For demonstration purposes, we are going to create a simple Django project:
/somewhere/$ django-admin startproject pre_commit
and initialize a git repository in it:
/somewhere/$ cd pre_commit /somewhere/pre_commit/$ git init Initialized empty Git repository in /somewhere/pre_commit/.git/
1st step of this automation will be creating and enabling the
pre-commit git hook.
Git has a way to fire off custom scripts when certain important actions occur.
pre-commit client-side hook is one of them. Git runs the hook script first before you even type in a commit message.
A good place to inspect the code in files that are about to be committed.
Hooks are all stored in the hooks subdirectory of the Git directory. In most projects, that’s
.git/hooks. By default, there are some sample hook files written as shell scripts, but any properly named executable script will do the job.
In our case, we will create an executable file called
/somewhere/pre_commit/$ touch .git/hooks/pre-commit /somewhere/pre_commit/$ chmod +x .git/hooks/pre-commit
To understand how it works, let's add the following into the hook script:.git/hooks/pre-commit
echo "Hook failed!" exit 1
If we run
git status, we will see some untracked files that are generated by
django-admin command. Let's
stage them and try to do the first
/somewhere/pre_commit/$ git add . /somewhere/pre_commit/$ git commit Hook failed!
We saw the message Hook failed! right after committing. And we were unable to type a message for the commit. We were not able to do a commit. The reason for that is the hook script exited with the status 1 instead of 0 which is the number that is used to indicate the successful termination of a script.
If we replace 1 with 0 we will be able to commit:.git/hooks/pre-commit
echo "Hook succeded!" exit 0
/somewhere/pre_commit/$ git add . /somewhere/pre_commit/$ git commit Hook succeded! 1 2 # Please enter the commit message for your changes. Lines starting 3 # with '#' will be ignored, and an empty message aborts the commit. 4 # 5 # On branch master 6 # Changes to be committed: 7 # added: somefile.py 8 #
Now that we know how
pre-commit hook works, we need to write a command that will check the code style of python files that we work on and exit with the non-zero(1) status if the check fails or with the zero status otherwise.
flake8 is a tool for style guide enforcement. We are going to use it for code style checks. We can install it as a python package using pip:
/somewhere/pre_commit/$ pip install flake8
To start using
flake8, we just need to type it as a command and pass a path to a file or directory. If we pass
settings.py file generated by Django we will see this:
/somewhere/pre_commit/$ flake8 pre_commit/settings.py pre_commit/settings.py:90:80: E501 line too long (91 > 79 characters) pre_commit/settings.py:93:80: E501 line too long (81 > 79 characters) pre_commit/settings.py:96:80: E501 line too long (82 > 79 characters) pre_commit/settings.py:99:80: E501 line too long (83 > 79 characters)
Notice that all 4 issues that we see are about the lengths of some lines in
This is the default file generated using
django-admin startproject command. The problem is that Django uses different number(119) as maximum allowed length of line compared with the number defined in PEP-8 which is 79 characters.
flake8 allows us to configure this maximum allowed length value in a file called
[flake8] max-line-length = 119
Now 119 is the maximum allowed length of lines for our project. Running check again:
/somewhere/pre_commit/$ flake8 pre_commit/settings.py /somewhere/pre_commit/$
and the output is empty. 119 is the maximum allowed length for the Django framework source code, so it's a good idea to use the same number in Django projects.
Until now, we did a code style check on a single file, but we would like to run the check on all staged python files. The final command is:
/somewhere/pre_commit/$ flake8 $(git status -s | grep -E '\.py$' | cut -c 4-)
The reason is that the project might grow and we might have many files there and we want to avoid checking all of them, instead, we want to check only those which are being changed.
In some cases we would also need to check code style on all files that we wrote by running
flake8 from the root directory of the project. But there might be folders or files that we need to exclude. Common examples are
[flake8] exclude = .venv,.git max-line-length = 119
We would also like to exclude all
*/migrations/* directories, because sometimes the auto-generated migration files in these directories are not PEP-8 compliant:
[flake8] exclude = .venv,.git,*/migrations/* max-line-length = 119
Now, that the final
flake8 command is ready we need to figure out how to connect it with the
echo $? command shows the exit status of the previous executed command. With the help of it, we can see what's the status of the
flake8 command if there are not code styles issues and the status in case of code style issues:
/somewhere/pre_commit/$ flake8 pre_commit/settings.py /somewhere/pre_commit/$ echo $? 0
no issues - zero exit. Let's force
flake8 to warn us about some errors by temporarily commenting the max-line-length param in setup.cfg file:
[flake8] ... # max-line-length = 119
now let's run again:
/somewhere/pre_commit/$ flake8 pre_commit/settings.py pre_commit/settings.py:90:80: E501 line too long (91 > 79 characters) pre_commit/settings.py:93:80: E501 line too long (81 > 79 characters) pre_commit/settings.py:96:80: E501 line too long (82 > 79 characters) pre_commit/settings.py:99:80: E501 line too long (83 > 79 characters) /somewhere/pre_commit/$ echo $? 1
issues - non-zero exit: perfect for putting it into the pre-commit git hook file:/somewhere/pre_commit/.git/hooks/pre-commit
flake8 $(git status -s | grep -E '\.py$' | cut -c 4-)
Now, whenever our flake8 command exits with a non-zero status, git won't allow us to continue committing.
Another problem with the code style is import orders.
isort comes to help us with that. It is a Python utility/library to sort imports alphabetically and automatically separated into sections and by type.
According to Django's official docs in Django project the preferred import order is:
# future from __future__ import unicode_literals # standard library import json from itertools import chain # third-party import bcrypt # Django from django.http import Http404 from django.http.response import ( Http404, HttpResponse, HttpResponseNotAllowed, StreamingHttpResponse, cookie, ) # local Django from .models import LogEntry # try/except try: import yaml except ImportError: yaml = None CONSTANT = 'foo' class Example: # ...
- standard library
- local Django
isort is as simple as:
/somewhere/pre_commit/$ pip install isort
and running is very similar to flake8:
/somewhere/pre_commit/$ isort pre_commit/settings.py /somewhere/pre_commit/$ echo $? 0
no issues with the file and the command finished with the zero status code.
We can configure parameters for
isort in the same file as we did for
[flake8] exclude = .venv,.git,*/migrations/* max-line-length = 119 [isort] sections = FUTURE,STDLIB,THIRDPARTY,LOCALFOLDER
We can change the section order with
We also can define our own sections and their order./somewhere/pre_commit/setup.cfg
[flake8] ... [isort] known_django=django sections = FUTURE,STDLIB,THIRDPARTY,DJANGO,LOCALFOLDER
It's important to mention that
isort command by default checks and applies changes to your files immideatelly and if you want to force it to check and warn only(similar to
flake8) you should pass
--diff to it:
/somewhere/pre_commit/$ isort --diff pre_commit/settings.py
To run the command only over the staged files as we did with
flake8, we add the following to the command:
/somewhere/pre_commit/$ isort --diff $(git status -s | grep -E '\.py$' | cut -c 4-)
This command is also ready to be moved into the hook:/somewhere/pre_commit/.git/hooks/pre-commit
STAGED_FILES = $(git status -s | grep -E '\.py$' | cut -c 4-) flake8 $STAGED_FILES isort --diff $STAGED_FILES
Now, the hook is ready to be used, but the problem is that it works only on our machine. One way of sharing this hook with our teammates is starting tracking it with git by moving into the root directory of our project and committing to a shared repository:
/somewhere/pre_commit/$ mv .git/hooks/pre-commit .
and after pushing it to the remote repository we guide our teammates to install it using symbolic links:
/somewhere/pre_commit/$ ln -s ../../pre-commit .git/hooks/pre-commit
This approach allows us to modify the hook later and make it work on all connected machines without doing any additional operation.