Nginx, Gunicorn and Django

I decided today to bring a new web stack consisting of nginx, gunicorn and django on a fedora 17 system. We are also throwing django-registration in the mix since the service requires authentication.

First things first, we need to install the packages on the local system.

$ yum install nginx python-gunicorn Django django-registration

We are developing a webapp written in an interpreted language (python) so a straightforward rsync should deploy the code to production, otherwise it weakens the rationale of using python for the job. Though production will run nginx, gunicorn and django, we still want to be able to debug the code on development machines with a simple manage.py runserver command. Hence thinking about file paths in advance is quite important. The following setup supports a flexible dev/prod approach.

*siteTop*/app                 # django project python code
*siteTop*/htdocs              # root for static html pages served by nginx
*siteTop*/htdocs/static       # root for static files served by nginx and django

The nginx configuration is simple and straightforward. Nginx redirects all pages to https and serves static content from htdocs.

upstream proxy_*domain* {
          server  127.0.0.1:8000;
  }

server {
          listen          80;
          server_name     *domain*;

          location / {
                  rewrite ^/(.*)$ https://$http_host/$1 redirect;
          }

  }

server {
        listen       443;
        server_name  *domain*;

        client_max_body_size 4G;
        keepalive_timeout 5;

        ssl                  on;
        ssl_certificate      /etc/ssl/certs/*domain*.pem;
        ssl_certificate_key  /etc/ssl/private/*domain*.key;

        ssl_session_timeout  5m;

        ssl_protocols  SSLv3 TLSv1;
        ssl_ciphers  ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
        ssl_prefer_server_ciphers   on;

        # path for static files
        root /var/www/*domain*/htdocs;

        location / {
            # checks for static file, if not found proxy to app
            try_files $uri @forward_to_app;
        }

        location @forward_to_app {
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_redirect off;
            proxy_intercept_errors on;

            proxy_pass      http://proxy_*domain*;
        }

        error_page 500 501 502 503 504 505 506 507 508 510 511 /50x.html;
        location = /50x.html {
            ssi on;
            internal;
            auth_basic off;
            root /var/www/*domain*/htdocs;
        }
    }

The django settings.py is also straightforward. The only interesting bits are figuring out the APP_ROOT and paths to static files.

$ diff -u prev settings.py
+import os.path
+APP_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

STATICFILES_DIRS = (
+    APP_ROOT + '/htdocs/static',
)
TEMPLATE_DIRS = (
+    APP_ROOT + '/app/templates'
)

Getting gunicorn hooked-up with django and behaving as expected was a lot more challenging.

First I tried to run gunicorn_django. I might have misread the documentation but I tried to pass on the command-line the directory where settings.py is located. Running in daemon mode, I saw both, gunicorn master and worker, up through ps, no error in the log files, and yet, I couldn't fetch a page. Only when I finally decided to run in non-daemon mode did it become obvious that gunicorn was running in an infinite loop.

Error: Can't find 'app' in your PYTHONPATH.
Error: Can't find 'app' in your PYTHONPATH.
...

Everything started to look fine when I passed the actual settings.py file on the command-line; well at least on Fedora 17. When I decided to run the same command on OSX, I got

Could not import settings 'app/settings.py' (Is it on sys.path?)

That is a weird error, especially that ls shows the file is there and definitely in the PYTHONPATH. Digging through the django code, I found an ImportError was caught and re-written as this error message in django/conf/__init__.py. As it turns out, my OSX python complains that importlib cannot import a module by filename.

I thus decided to use the second way of running gunicorn I saw advertised, through manage.py.

$ pip install gunicorn
$ diff -u prev settings.py
INSTALLED_APPS = (
+    'gunicorn',
     ...
)
$ python ./manage.py run_gunicorn

That worked fine; still I couldn't seem to change the gunicorn process name despite all attempts. As it turns out, no error, no warning, just a silent fail because setproctitle wasn't installed on my system.

$ yum install python-setproctitle

From that point on we could run the webapp, both in prod through nginx, gunicorn and django and directly through manage.py runserver in development.