Integrating Django i18n with Jinja2 and Vue.js

by Dima Knivets on Tue, 16 Apr 2019

It is straightforward to add translations to a Django project with built in Django templates. However, if you are using other template engines like Jinja2, it might take more effort to make those work together. Things can get tricky if you want to manage translations for both the frontend and the backend using the same Django tools. I'm not going to cover the basics of i18n in Django hereas there is already a great deal of information on the Django website, but instead talk about the out of the box stuff that is not integrated.

Jinja2

Jinja2 doesn't support Django templates i18n tags or any other i18n functionality out of the box. There is also a command line tool in Django that scans your project and produces a .po file with translation strings. Unfortunately, it only works with the default Django templates out of the box. The following should help solve those problems.

Jinja2 has a i18n extension that adds a {% trans %} block and a _() function. The first one looks like a similar tag in Django templates, but they are two different things. You can read more about them here. Here's an example:

Django template
{% trans 'hello world' %}

# Jinja2 template
{% trans %}
hello world
{% endtrans %}

This extension is disabled by default, so first we need to add some configuration. Jinja2 is customized with the help of the Environment class. When Django is used with Jinja2 it uses a default environment, which needs to be overridden in order to enable the extension. I’m assuming you already have a custom Jinja2 environment in your project, if not you can learn how to configure a custom environment here. Below isan example of a basic Jinja2 environment file with i18n support

Python
from django.utils.translation import gettext, ngettext
from jinja2 import Environment

def environment(**options):
    # i18n extension
    options['extensions'] = ['jinja2.ext.i18n']

    env = Environment(**options)

    # i18n template functions
    env.install_gettext_callables(gettext=gettext, ngettext=ngettext,
        newstyle=True)

    return env

At this point Django will translate the i18n strings in Jinja2 templates if those strings are present in .po and .mo translation files. We can collect the strings manually, but it takes time and is error-prone. So let's leverage Django makemessages command to do this for us. As I said, it doesn't work with Jinja2 templates out of the box and there is no clean way to enable the support, so we'll resort to a few hacks in order to make this work. As I searched for the solution I stumbled upon a django-jinja project which actually solved this problem. Howeverinstead of using the whole project, I'm going to extract this single feature: put the code inside the {project_root}/{app_name}/management/commands/makemessages.py file:

Python
import re

from django import VERSION as DJANGO_VERSION
from django.core.management.commands import makemessages
from django.template.base import BLOCK_TAG_START, BLOCK_TAG_END

if DJANGO_VERSION[:2] < (1, 11):
    from django.utils.translation import trans_real
else:
    from django.utils.translation import template as trans_real

strip_whitespace_right = re.compile(r"(%s-?\s*(trans|pluralize).*?-%s)\s+" % (BLOCK_TAG_START, BLOCK_TAG_END), re.U)
strip_whitespace_left = re.compile(r"\s+(%s-\s*(endtrans|pluralize).*?-?%s)" % (BLOCK_TAG_START, BLOCK_TAG_END), re.U)


def strip_whitespaces(src):
    src = strip_whitespace_left.sub(r'\1', src)
    src = strip_whitespace_right.sub(r'\1', src)
    return src


class Command(makemessages.Command):

    def handle(self, *args, **options):
        old_endblock_re = trans_real.endblock_re
        old_block_re = trans_real.block_re
        old_constant_re = trans_real.constant_re

        old_templatize = trans_real.templatize
        # Extend the regular expressions that are used to detect
        # translation blocks with an "OR jinja-syntax" clause.
        trans_real.endblock_re = re.compile(
            trans_real.endblock_re.pattern + '|' + r"""^-?\s*endtrans\s*-?$""")
        trans_real.block_re = re.compile(
            trans_real.block_re.pattern + '|' + r"""^-?\s*trans(?:\s+(?!'|")(?=.*?=.*?)|\s*-?$)""")
        trans_real.plural_re = re.compile(
            trans_real.plural_re.pattern + '|' + r"""^-?\s*pluralize(?:\s+.+|-?$)""")
        trans_real.constant_re = re.compile(r""".*?_\(((?:".*?(?<!\\)")|(?:'.*?(?<!\\)')).*?\)""")

        def my_templatize(src, origin=None, **kwargs):
            new_src = strip_whitespaces(src)
            return old_templatize(new_src, origin, **kwargs)

        trans_real.templatize = my_templatize

        try:
            super(Command, self).handle(*args, **options)
        finally:
            trans_real.endblock_re = old_endblock_re
            trans_real.block_re = old_block_re
            trans_real.templatize = old_templatize
            trans_real.constant_re = old_constant_re

This code, basically, overrides the builtin makemessages command to add Jinja2 support and falls back to the default command when it deals with Django templates. In order to collect the translation strings inside a .po file, go to an app directory that you want to translate, make sure there is a locale directory inside this app ({app_name}/locale) and run django-admin makemessages -l {locale_name}, where {locale_name} is an actual locale name, like de. If you followed the steps, you will have a {app_name}/locale/{locale_name}/LC_MESSAGES/django.po translation file that contains the strings from Jinja2 templates that you'll need to translate. Having a .po file is not enough though, so once you've provided the translations, run in the app directory:

Terminal
$ django-admin compilemessages

This command will produce a .mo file which will be used by Django at runtime. At this point, the Jinja2 18n functionality is pretty much on the same level as the Django templates. Awesome!

Vue.js

There is a myriad of i18n libraries on the client side, but with this approach wemanage the translations for the frontend and the backend separately. Ideally, a single tool would do the work. Django docs has a whole section about working with translations on the client side. Basically, it provides a view that generates a JavaScript code. This code adds i18n functions to the client and an object that contains all the translations from the .po, .mo files. It also supports a collection of translation strings from the JavaScript files.

To enable this functionality, add this to urls.py in your project:

Django
from django.views.i18n import JavaScriptCatalog

urlpatterns = [
    path('jsi18n/', JavaScriptCatalog.as_view(), name='javascript-catalog'),
]

And this line to your base.html:

Django Template
<script type="text/javascript" src="{% url 'javascript-catalog' %}"></script>

If you don't have url helpers in your Jinja2 configuration, set the src attribute manually to jsi18n/.

Now, when you load a page you have an access to gettext, interpolate and a bunch of other i18n functions. Adapt your strings to use those functions and when finished Django will be able to collect those strings into a .po file. Once again, go to an app folder and run:

Terminal
$ django-admin makemessages -d djangojs -l {locale_name}

The flag -d djangojs tells Django to scan the JavaScript files. Once you are finished with the translations, you'll need to run:

Terminal
$ django-admin compilemessages

The command to produce a .mo file. At this point, JavaScript code is internationalized and the translations for both the frontend and the backend are managed by a single tool – nice!

Bonus: automatic language detection & optimizations

The client side i18n integration works smoothly, but we still need to make an extra request to get the client side code. This is fine in most scenarios. However, we use django-assets package which takes all the assets and minifies them into a single file to reduce the size and the number of requests to the server. We also leverage CDN services to distribute assets to data centers around the globe. Out of the box JavaScriptCatalog view generates a code that contains translation strings for a single language only. We can't really cache this to support automatic language detection on page load. We can however solve those problems by moving the code into a Django management command. This generates a static file with the i18n code, that includes translations for every locale that we support. We can then minify the assets at a build time and serve them via CDN, while a client will be able to pick the right translations because they are all included in this file.

JavaScriptCatalog is a Django view which means it expects a HTTP request, however we'd like to call it as a function without providing a request object. Extracting functionality from this class in order to create a standalone function would be a tedious process, so I settled on subclassing it and creating a method that leverages the JavaScriptCatalog methods to generate the JavaScript code. I duplicated the JavaScript code source in order to make modifications to support the automatic language detection. Finally, I adapted the resulting function into a management command. This is what I ended up with:

Python
"""Command to generate JavaScript file used for i18n on the frontend"""

import json, os

from django.conf import settings
from django.core.management.base import BaseCommand
from django.views.i18n import JavaScriptCatalog, get_formats
from django.template import Context, Engine
from django.utils.translation.trans_real import DjangoTranslation

js_catalog_template = r"""
{% autoescape off %}
(function(globals) {
  var activeLang = navigator.language || navigator.userLanguage || 'en';
  var django = globals.django || (globals.django = {});
  var plural = {{ plural }};
  django.pluralidx = function(n) {
    var v = plural[activeLang];
    if(v){
        if (typeof(v) == 'boolean') {
          return v ? 1 : 0;
        } else {
          return v;
        }
    } else {
        return (n == 1) ? 0 : 1;
    }
  };
  /* gettext library */
  django.catalog = django.catalog || {};
  {% if catalog_str %}
  var newcatalog = {{ catalog_str }};
  for (var ln in newcatalog) {
    django.catalog[ln] = newcatalog[ln];
  }
  {% endif %}
  if (!django.jsi18n_initialized) {
    django.gettext = function(msgid) {
      var lnCatalog = django.catalog[activeLang]
      if(lnCatalog){
          var value = lnCatalog[msgid];
          if (typeof(value) != 'undefined') {
            return (typeof(value) == 'string') ? value : value[0];
          }
      }
      return msgid;
    };
    django.ngettext = function(singular, plural, count) {
      var lnCatalog = django.catalog[activeLang]
      if(lnCatalog){
          var value = lnCatalog[singular];
          if (typeof(value) != 'undefined') {
          } else {
            return value.constructor === Array ? value[django.pluralidx(count)] : value;
          }
      }
      return (count == 1) ? singular : plural;
    };
    django.gettext_noop = function(msgid) { return msgid; };
    django.pgettext = function(context, msgid) {
      var value = django.gettext(context + '\x04' + msgid);
      if (value.indexOf('\x04') != -1) {
        value = msgid;
      }
      return value;
    };
    django.npgettext = function(context, singular, plural, count) {
      var value = django.ngettext(context + '\x04' + singular, context + '\x04' + plural, count);
      if (value.indexOf('\x04') != -1) {
        value = django.ngettext(singular, plural, count);
      }
      return value;
    };
    django.interpolate = function(fmt, obj, named) {
      if (named) {
        return fmt.replace(/%\(\w+\)s/g, function(match){return String(obj[match.slice(2,-2)])});
      } else {
        return fmt.replace(/%s/g, function(match){return String(obj.shift())});
      }
    };
    /* formatting library */
    django.formats = {{ formats_str }};
    django.get_format = function(format_type) {
      var value = django.formats[format_type];
      if (typeof(value) == 'undefined') {
        return format_type;
      } else {
        return value;
      }
    };
    /* add to global namespace */
    globals.pluralidx = django.pluralidx;
    globals.gettext = django.gettext;
    globals.ngettext = django.ngettext;
    globals.gettext_noop = django.gettext_noop;
    globals.pgettext = django.pgettext;
    globals.npgettext = django.npgettext;
    globals.interpolate = django.interpolate;
    globals.get_format = django.get_format;
    django.jsi18n_initialized = true;
  }
}(this));
{% endautoescape %}
"""


class Command(BaseCommand):
    """Generate JavaScript file for i18n purposes"""
    help = 'Generate JavaScript file for i18n purposes'

    def add_arguments(self, parser):
        parser.add_argument('PATH', nargs=1, type=str)

    def handle(self, *args, **options):
        contents = self.generate_i18n_js()
        path = os.path.join(settings.BASE_DIR, options['PATH'][0])
        with open(path, 'w') as f:
            f.write(contents)
        self.stdout.write('wrote file into %s\n' % path)

    def generate_i18n_js(self):
        class InlineJavaScriptCatalog(JavaScriptCatalog):
            def render_to_str(self):
                # hardcoding locales as it is not trivial to
                # get user apps and its locales, and including
                # all django supported locales is not efficient
                codes = ['en', 'de', 'ru', 'es', 'fr', 'pt']
                catalog = {}
                plural = {}
                # this function is not i18n-enabled
                formats = get_formats()
                for code in codes:
                    self.translation = DjangoTranslation(code, domain=self.domain)
                    _catalog = self.get_catalog()
                    _plural = self.get_plural()
                    if _catalog:
                        catalog[code] = _catalog
                    if _plural:
                        plural[code] = _plural
                template = Engine().from_string(js_catalog_template)
                context = {
                    'catalog_str': json.dumps(catalog, sort_keys=True, indent=2),
                    'formats_str': json.dumps(formats, sort_keys=True, indent=2),
                    'plural': plural,
                }
                return template.render(Context(context))

        return InlineJavaScriptCatalog().render_to_str()

You can test the command by calling ./manage.py generate_i18n_js {app/static/path} where {app/static/path} is the path to put the generated file. Finally, you might want to include this file in your base.html template to see if it works:

Django Template
...
<head>
...
    <script src="{a_url_to_the_i18n_file}"></script>
...
</head>
...

That's it! Besides having a convenient system to manage the translations, the code is fast and the automatic language detection still works.

Summary

Django has a powerful and a somewhat easy to use i18n library that works pretty well with the default template engine, but needs a bit of work to adapt it to other template engines. It is great that it also handles the client side part, but the downside is this results in two separate files to maintain: one for the backend and one for the frontend. While not critical, It could've been easier, I suppose.

More to read

If you are looking for more posts on handling various issues in a multi-language stack, Date/time, back and forth between Javascript, Django and PostgreSQL and Django Rest Framework, AngularJS and permissions are worth reading next.

More technical posts are also available on the DjaoDjin blog, as well as business lessons we learned running a SaaS hosting platform.

by Dima Knivets on Tue, 16 Apr 2019


Receive news about DjaoDjin in your inbox.

Bring fully-featured SaaS products to production faster.

Follow us on