testing a Django 2.2 website with SQLite3 on CentOS 7

by Sebastien Mirolo on Wed, 8 May 2019

Django 2.2 was released on Apr 1st 2019. Since it is the most recent as well a Long Term Support (LTS) version, it was time to update the Jenkins bot infrastructure to include Django 2.2 in the Django versions tested.

Installing Python 3.6 on CentOS 7

Python 2.7 is the default python on CentOS 7. Django starting at version 2.0 dropped support for Python 2.7. So we will need to install a Python3 tooling. With CentOS 8 around the corner, it is worth taking a peek at which version of Python3 will be the default on CentOS 8. Early indications are Python 3.6 is the default in RHEL 8. Let's see how to install Python 3.6 on CentOS 7.

First we will need to add an extra source to yum so we can install later version of packages - i.e. packages that are absent from the default CentOS 7 install.

The SCL repository, now maintained by a CentOS SIG, rebuilds the Red Hat Software Collections, adds some additional packages of their own, and contains newer versions of various programs that can be installed alongside existing older packages.

The IUS Community Project is a community project that provides RPM packages for newer versions of select software for Enterprise Linux distributions.

Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for RHEL and CentOS amongst others.

SCL packages are typically prefixed by rh-, IUS packages suffixed by u. EPEL packages seem to simply keep the package name present in their Fedora counterpart.

We will install Python 3.6 from EPEL here.

$ sudo yum install epel-release
$ sudo yum update
$ sudo yum install python36 python36-libs python36-devel python36-pip

Alternative

# IUS Community
$ sudo yum install -y https://centos7.iuscommunity.org/ius-release.rpm
$ sudo yum update
$ sudo yum install -y python36u python36u-libs python36u-devel python36u-pip

Django 2.2 and SQLite3

The SQLite3 version installed by default on CentOS7 is 3.7.17, unfortunately Django 2.2 requires version 3.8.3 or later.

$ /usr/bin/sqlite3 --version
3.7.17
$ cat django2.2/db/backends/sqlite3/base.py
...
raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)
...

Installing a SQLite3 version compatible with Django1.11 and Django2.2

At this point, we cannot just install any version of SQLite3 because SQLite 3.26 breaks database migration ForeignKey constraint, leaving <table_name>__old in db schema, and you end up with following errors in Django 1.11.18.

Problem installing fixtures: no such table: <table_name>__old

Downloading an earlier version of SQLite3 is not that straightforward though. It requires to go to the SQLite3 download page, parse the schema used to create package names (i.e. https://www.sqlite.org/{year}/{sqlite-autoconf-{version}.tar.gz) and hope we can download an earlier version that way.

$ wget https://www.sqlite.org/2018/sqlite-autoconf-3240000.tar.gz
$ tar zxvf sqlite-autoconf-3240000.tar.gz
$ ./configure --prefix=/usr/local
$ make
$ sudo make install

Using recently installed SQLite3 version with Django2.2

Before version 2.2, Django would try to load pysqlite2 first, then fallback to sqlite3.

$ cat django1.11/db/backends/sqlite3/base.py
try:
    try:
        from pysqlite2 import dbapi2 as Database
    except ImportError:
        from sqlite3 import dbapi2 as Database
except ImportError as exc:
    raise ImproperlyConfigured("Error loading either pysqlite2 or sqlite3 modules (tried in that order): %s" % exc)

This is not the case any more. Starting with Django 2.2, the code looks like:

$ cat django2.2/db/backends/sqlite3/base.py
from sqlite3 import dbapi2 as Database

It is thus impossible to install a newer version of SQLite and a pysqlite2 package pointing to it in a virtual environment now.

We will rely on the true and tested LD_LIBRARY_PATH approach.

$ python3.6 -c "import sqlite3; print(sqlite3.sqlite_version)"
3.7.17

$ export LD_LIBRARY_PATH=/usr/local/lib
$ python3.6 -c "import sqlite3; print(sqlite3.sqlite_version)"
3.24.0

Since we are running all the tests through a Jenkins bot, it works well to add the LD_LIBRARY_PATH statement in the run script. Otherwise we would have to put that statement in the virtualenv activate script.

More to read

If you are looking for more posts on Django, Porting a Django app to Jinja2 templates and Documenting an API implemented with Django Rest Framework are worth reading next.

More technical posts are also available on the DjaoDjin blog, as well as business lessons we learned running a subscription hosting platform.

by Sebastien Mirolo on Wed, 8 May 2019


Bring fully-featured SaaS products to production faster.

Follow us on