.. -*- coding: utf-8 -*- ============================================= Status of Internationalization (i18n) Support ============================================= .. Note:: This documents the design of i18n in HUE, as well as the various degrees of i18n support in different HUE applications. --------------- Design Overview --------------- Thanks to Django, HUE always receives unicode data from the browser. Data operations within the boundary of HUE should be done in unicode as much as possible. Data encoding occurs when the data exits HUE, in one of the following ways: Model Storage These are data to go into a Django model, such as user firstname, lastname, Beeswax saved query. Since the Django model layer handles unicode data, HUE does not need to do any encoding. Hadoop Metadata These include namespace metadata, Hive Metastore data, and so on. In general, Hadoop metadata (where they support international characters) all seem to be in UTF-8. Hadoop Data This category includes file data, Hive table contents, HBase contents, etc. The goal is to support custom encodings as specified by the user. Eventually, on a view by view basis, the user may select the correct data encoding. Meanwhile, for applications and views that do not yet expose such flexibility, they may use the ``DEFAULT_SITE_ENCODING`` configuration variable to interpret binary Hadoop Data. --------------------- Implementation Status --------------------- HDFS ==== HDFS namespace is in UTF-8. So we always encode to (and decode from) UTF-8. The ``libs.hadoop.fs.hadoopfs`` module speaks UTF-8 and handles this for us. The File Browser correctly handles HDFS namespace. *It does not yet expose file contents encoding in the UI*. Hive ==== Hive metadata (such as table comments) is also in UTF-8. The ``beeswax.db_utils`` module converts between HUE (unicode) and Hive (UTF-8). Currently, Hive seems to only work with UTF-8 data.