Integrating Django i18n with Jinja2 and Vue.js
by Dima Knivets on Tue, 16 Apr 2019
It is straightforward to add translations to a Django project with built in
Django templates. However, if you are using other template engines like Jinja2,
it might take more effort to make those work together. Things can get tricky if
you want to manage translations for both the frontend and the backend using the
same Django tools. I'm not going to cover the basics of i18n in Django hereas
there is already a great deal of information on the Django website, but
instead talk about the out of the box stuff that is not integrated.
Jinja2
Jinja2 doesn't support Django templates i18n tags or any other i18n
functionality out of the box. There is also a command line tool in Django that
scans your project and produces a .po file with translation
strings. Unfortunately, it only works with the default Django templates out
of the box. The following should help solve those problems.
Jinja2 has a i18n extension that adds a {% trans %} block and
a _() function. The first one looks like a similar tag in Django
templates, but they are two different things. You can read more about them
here. Here's an example:
{% trans 'hello world' %}
# Jinja2 template
{% trans %}
hello world
{% endtrans %}
This extension is disabled by default, so first we need to add some configuration. Jinja2 is customized with the help of the Environment class. When Django is used with Jinja2 it uses a default environment, which needs to be overridden in order to enable the extension. I’m assuming you already have a custom Jinja2 environment in your project, if not you can learn how to configure a custom environment here. Below isan example of a basic Jinja2 environment file with i18n support
from django.utils.translation import gettext, ngettext
from jinja2 import Environment
def environment(**options):
# i18n extension
options['extensions'] = ['jinja2.ext.i18n']
env = Environment(**options)
# i18n template functions
env.install_gettext_callables(gettext=gettext, ngettext=ngettext,
newstyle=True)
return env
At this point Django will translate the i18n strings in Jinja2 templates
if those strings are present in .po and .mo
translation files. We can collect the strings manually, but it takes time and
is error-prone. So let's leverage Django makemessages command
to do this for us. As I said, it doesn't work with Jinja2 templates out of
the box and there is no clean way to enable the support, so we'll resort
to a few hacks in order to make this work. As I searched for the solution
I stumbled upon a django-jinja project which actually solved this problem.
Howeverinstead of using the whole project, I'm going to extract
this single feature:
put the code inside the
import re
from django import VERSION as DJANGO_VERSION
from django.core.management.commands import makemessages
from django.template.base import BLOCK_TAG_START, BLOCK_TAG_END
if DJANGO_VERSION[:2] < (1, 11):
from django.utils.translation import trans_real
else:
from django.utils.translation import template as trans_real
strip_whitespace_right = re.compile(r"(%s-?\s*(trans|pluralize).*?-%s)\s+" % (BLOCK_TAG_START, BLOCK_TAG_END), re.U)
strip_whitespace_left = re.compile(r"\s+(%s-\s*(endtrans|pluralize).*?-?%s)" % (BLOCK_TAG_START, BLOCK_TAG_END), re.U)
def strip_whitespaces(src):
src = strip_whitespace_left.sub(r'\1', src)
src = strip_whitespace_right.sub(r'\1', src)
return src
class Command(makemessages.Command):
def handle(self, *args, **options):
old_endblock_re = trans_real.endblock_re
old_block_re = trans_real.block_re
old_constant_re = trans_real.constant_re
old_templatize = trans_real.templatize
# Extend the regular expressions that are used to detect
# translation blocks with an "OR jinja-syntax" clause.
trans_real.endblock_re = re.compile(
trans_real.endblock_re.pattern + '|' + r"""^-?\s*endtrans\s*-?$""")
trans_real.block_re = re.compile(
trans_real.block_re.pattern + '|' + r"""^-?\s*trans(?:\s+(?!'|")(?=.*?=.*?)|\s*-?$)""")
trans_real.plural_re = re.compile(
trans_real.plural_re.pattern + '|' + r"""^-?\s*pluralize(?:\s+.+|-?$)""")
trans_real.constant_re = re.compile(r""".*?_\(((?:".*?(?<!\\)")|(?:'.*?(?<!\\)')).*?\)""")
def my_templatize(src, origin=None, **kwargs):
new_src = strip_whitespaces(src)
return old_templatize(new_src, origin, **kwargs)
trans_real.templatize = my_templatize
try:
super(Command, self).handle(*args, **options)
finally:
trans_real.endblock_re = old_endblock_re
trans_real.block_re = old_block_re
trans_real.templatize = old_templatize
trans_real.constant_re = old_constant_re
This code, basically, overrides the builtin makemessages command to add Jinja2
support and falls back to the default command when it deals with Django
templates. In order to collect the translation strings inside a .po
file, go to an app directory that you want to translate, make sure there is
a locale directory inside this app (django-admin makemessages -l {locale_name},
where {locale_name} is an actual locale name,
like de. If you followed the steps, you will have a .po file is not enough though,
so once you've provided the translations, run in the app directory:
$ django-admin compilemessages
This command will produce a .mo file which will
be used by Django at runtime. At this point, the Jinja2 18n functionality
is pretty much on the same level as the Django templates. Awesome!
Vue.js
There is a myriad of i18n libraries on the client side, but with this approach
wemanage the translations for the frontend and the backend separately. Ideally,
a single tool would do the work. Django docs has a whole
section
about working with translations on the client side. Basically, it provides
a view that generates a JavaScript code. This code adds i18n functions
to the client and an object that contains all the translations from
the .po, .mo files.
It also supports a collection of translation strings from the JavaScript files.
To enable this functionality, add this to urls.py in your project:
from django.views.i18n import JavaScriptCatalog
urlpatterns = [
path('jsi18n/', JavaScriptCatalog.as_view(), name='javascript-catalog'),
]
And this line to your base.html:
<script type="text/javascript" src="{% url 'javascript-catalog' %}"></script>
If you don't have url helpers in your Jinja2 configuration, set the
src attribute manually to jsi18n/.
Now, when you load a page you have an access to gettext, interpolate and
a bunch of other i18n functions.
Adapt your strings to use those functions and when finished Django will
be able to collect those strings into a .po file.
Once again, go to an app folder and run:
$ django-admin makemessages -d djangojs -l {locale_name}
The flag -d djangojs tells Django to scan the JavaScript files.
Once you are finished with the translations, you'll need to run:
$ django-admin compilemessages
The command to produce a .mo file.
At this point, JavaScript code is internationalized and the translations
for both the frontend and the backend are managed by a single tool – nice!
Bonus: automatic language detection & optimizations
The client side i18n integration works smoothly, but we still need to make an extra request to get the client side code. This is fine in most scenarios. However, we use django-assets package which takes all the assets and minifies them into a single file to reduce the size and the number of requests to the server. We also leverage CDN services to distribute assets to data centers around the globe. Out of the box JavaScriptCatalog view generates a code that contains translation strings for a single language only. We can't really cache this to support automatic language detection on page load. We can however solve those problems by moving the code into a Django management command. This generates a static file with the i18n code, that includes translations for every locale that we support. We can then minify the assets at a build time and serve them via CDN, while a client will be able to pick the right translations because they are all included in this file.
JavaScriptCatalog is a Django view which means it expects a HTTP request, however we'd like to call it as a function without providing a request object. Extracting functionality from this class in order to create a standalone function would be a tedious process, so I settled on subclassing it and creating a method that leverages the JavaScriptCatalog methods to generate the JavaScript code. I duplicated the JavaScript code source in order to make modifications to support the automatic language detection. Finally, I adapted the resulting function into a management command. This is what I ended up with:
"""Command to generate JavaScript file used for i18n on the frontend"""
import json, os
from django.conf import settings
from django.core.management.base import BaseCommand
from django.views.i18n import JavaScriptCatalog, get_formats
from django.template import Context, Engine
from django.utils.translation.trans_real import DjangoTranslation
js_catalog_template = r"""
{% autoescape off %}
(function(globals) {
var activeLang = navigator.language || navigator.userLanguage || 'en';
var django = globals.django || (globals.django = {});
var plural = {{ plural }};
django.pluralidx = function(n) {
var v = plural[activeLang];
if(v){
if (typeof(v) == 'boolean') {
return v ? 1 : 0;
} else {
return v;
}
} else {
return (n == 1) ? 0 : 1;
}
};
/* gettext library */
django.catalog = django.catalog || {};
{% if catalog_str %}
var newcatalog = {{ catalog_str }};
for (var ln in newcatalog) {
django.catalog[ln] = newcatalog[ln];
}
{% endif %}
if (!django.jsi18n_initialized) {
django.gettext = function(msgid) {
var lnCatalog = django.catalog[activeLang]
if(lnCatalog){
var value = lnCatalog[msgid];
if (typeof(value) != 'undefined') {
return (typeof(value) == 'string') ? value : value[0];
}
}
return msgid;
};
django.ngettext = function(singular, plural, count) {
var lnCatalog = django.catalog[activeLang]
if(lnCatalog){
var value = lnCatalog[singular];
if (typeof(value) != 'undefined') {
} else {
return value.constructor === Array ? value[django.pluralidx(count)] : value;
}
}
return (count == 1) ? singular : plural;
};
django.gettext_noop = function(msgid) { return msgid; };
django.pgettext = function(context, msgid) {
var value = django.gettext(context + '\x04' + msgid);
if (value.indexOf('\x04') != -1) {
value = msgid;
}
return value;
};
django.npgettext = function(context, singular, plural, count) {
var value = django.ngettext(context + '\x04' + singular, context + '\x04' + plural, count);
if (value.indexOf('\x04') != -1) {
value = django.ngettext(singular, plural, count);
}
return value;
};
django.interpolate = function(fmt, obj, named) {
if (named) {
return fmt.replace(/%\(\w+\)s/g, function(match){return String(obj[match.slice(2,-2)])});
} else {
return fmt.replace(/%s/g, function(match){return String(obj.shift())});
}
};
/* formatting library */
django.formats = {{ formats_str }};
django.get_format = function(format_type) {
var value = django.formats[format_type];
if (typeof(value) == 'undefined') {
return format_type;
} else {
return value;
}
};
/* add to global namespace */
globals.pluralidx = django.pluralidx;
globals.gettext = django.gettext;
globals.ngettext = django.ngettext;
globals.gettext_noop = django.gettext_noop;
globals.pgettext = django.pgettext;
globals.npgettext = django.npgettext;
globals.interpolate = django.interpolate;
globals.get_format = django.get_format;
django.jsi18n_initialized = true;
}
}(this));
{% endautoescape %}
"""
class Command(BaseCommand):
"""Generate JavaScript file for i18n purposes"""
help = 'Generate JavaScript file for i18n purposes'
def add_arguments(self, parser):
parser.add_argument('PATH', nargs=1, type=str)
def handle(self, *args, **options):
contents = self.generate_i18n_js()
path = os.path.join(settings.BASE_DIR, options['PATH'][0])
with open(path, 'w') as f:
f.write(contents)
self.stdout.write('wrote file into %s\n' % path)
def generate_i18n_js(self):
class InlineJavaScriptCatalog(JavaScriptCatalog):
def render_to_str(self):
# hardcoding locales as it is not trivial to
# get user apps and its locales, and including
# all django supported locales is not efficient
codes = ['en', 'de', 'ru', 'es', 'fr', 'pt']
catalog = {}
plural = {}
# this function is not i18n-enabled
formats = get_formats()
for code in codes:
self.translation = DjangoTranslation(code, domain=self.domain)
_catalog = self.get_catalog()
_plural = self.get_plural()
if _catalog:
catalog[code] = _catalog
if _plural:
plural[code] = _plural
template = Engine().from_string(js_catalog_template)
context = {
'catalog_str': json.dumps(catalog, sort_keys=True, indent=2),
'formats_str': json.dumps(formats, sort_keys=True, indent=2),
'plural': plural,
}
return template.render(Context(context))
return InlineJavaScriptCatalog().render_to_str()
You can test the command by calling ./manage.py generate_i18n_js {app/static/path}
where {app/static/path} is the path to put the generated file. Finally,
you might want to include this file in your base.html template to see if
it works:
...
<head>
...
<script src="{a_url_to_the_i18n_file}"></script>
...
</head>
...
That's it! Besides having a convenient system to manage the translations, the code is fast and the automatic language detection still works.
Summary
Django has a powerful and a somewhat easy to use i18n library that works pretty
well with the default template engine, but needs a bit of work to adapt it to
other template engines. It is great that it also handles the client side part,
but the downside is this results in two separate
More to read
If you are looking for more posts on handling various issues in a multi-language stack, Date/time, back and forth between Javascript, Django and PostgreSQL and Django Rest Framework, AngularJS and permissions are worth reading next.
More technical posts are also available on the DjaoDjin blog, as well as business lessons we learned running a SaaS hosting platform.
by Dima Knivets on Tue, 16 Apr 2019
