Schema Migration with Django

Database schema migration is a fact of life and South has imposed itself as the de facto solution within the Django community (at least until Django 1.7). The south docs are explicit, yet it is good to read the excellent south-explained post before diving in.

Current (2023) versions of Django manage migrations directly. This post is here purely for historical reference.

What isn't always clear dealing with south is which commands should run on a developer's machine, which should run on a production machine and what gets into your source control system at what time. That is my focus for this post.

First steps first, we install south in our environment and add south to the settings.py.

$ pip install South
$ pip search South
South                     - South: Migrations for Django
  INSTALLED: 1.0 (latest)

$ diff -u prev settings.py
+# Tell South to use a normal syncdb and not a migrate when running unit tests.
+# This will disable the annoying South logging messages we see otherwise
+# while running behave tests.
+SOUTH_TESTS_MIGRATE = False

INSTALLED_APPS = (
+    'south',
...
)

Pocking around

So far we created our models and debugged our new app locally. It looks all great and its about time to push it into production - Good timing to create the initial schema migration.

Just to make sure we are clean, we will delete the test database and recreate it.

$ rm var/db/django.sqlite
$ python ./manage.py syncdb
Creating table ...
Installing custom SQL ...
Installing indexes ...
Synced:
...

Not synced (use migrations):
 - django_extensions
(use ./manage.py migrate to migrate these)

We notice here that syncdb did not create some of the tables (i.e. the ones for the django_extensions app). How did that happen?

As it turns out the django_extensions app has a special migrations/ sub-directory. That is how South figures out which app tables are migrated and which aren't.

At this point, the South tutorial tells us to create a schema migration for our app and run a fake migration based of it (since ./manage.py syncdb already created the tables).

$ python ./manage.py schemamigration app_name --initial
$ python ./manage.py migrate app_name 0001 --fake

Because we are curious...

$ sqlite3 var/db/django.sqlite \
    "SELECT * FROM sqlite_master WHERE type='table';" | grep south
$ sqlite3 var/db/django.sqlite \
    "SELECT * FROM south_migrationhistory;"
1|app_name|0001_initial|2014-07-19 20:57:08.302140

We learn from these little experiments we just cannot add South to INSTALLED_APPS and forget about it.

1. There are too many chances a developer will forget to run a schemamigration after changing a Model and before pushing her changes, thus resulting in inconsistencies in the development workspace of another developer pulling her work. It is actually very confusing to do a ./manage.py migrate and find out the SQL tables don't match the Models defined in models.py. (South updates the database schema solely based on the migrations module. If you add a field in a Model, do not run a schemamigration command followed my a migrate nothing happens.)
2. Running a schemamigration and committing the resulting generated files in the migrations/ sub-directory creates clutter in the source repository we would rather not have. What's the point of a versioning system if there appear a new file in the source tree for every update to models.py?
3. What do we do when we update to a newer version of a prerequisite (requirements.txt) that does not include a South migrations/ directory?

Out of more curiosity, what happens when we create a schemamigration without a change in models.py? What happens when we migrate and there are no change?

$ python ./manage.py schemamigration app_name --auto
Nothing seems to have changed.
$ python ./manage.py migrate
Running migrations for app_name:
- Nothing to migrate.
 - Loading initial data for app_name.
Installed 0 object(s) from 0 fixture(s)

OK, pretty harmless. It is good. Though apparently better we don't have any initial_data fixtures around or add the --no-initial-data command line flag to ./manage.py migrate.

Lessons Learned

As a policy we thus move to

• Only use South where it matters (i.e. production database) and modify our settings.py accordingly
  Terminal

  if DEBUG:
      ENV_INSTALLED_APPS = (
          )
  else:
      ENV_INSTALLED_APPS = (
          'south',
      )
  
  INSTALLED_APPS = ENV_INSTALLED_APPS + (
  ...
  )

• Do not commit migrations/ sub-directories. Instead consistently use the deployment script to create them for ALL apps specified in INSTALLED_APPS, then call ./manage.py migrate.