| 12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455 |
- .. -*- coding: utf-8 -*-
- =============================================
- Status of Internationalization (i18n) Support
- =============================================
- .. Note::
- This documents the design of i18n in HUE, as well as the various degrees of
- i18n support in different HUE applications.
- ---------------
- Design Overview
- ---------------
- Thanks to Django, HUE always receives unicode data from the browser. Data
- operations within the boundary of HUE should be done in unicode as much as
- possible. Data encoding occurs when the data exits HUE, in one of the following
- ways:
- Model Storage
- These are data to go into a Django model, such as user firstname, lastname,
- Beeswax saved query. Since the Django model layer handles unicode data, HUE
- does not need to do any encoding.
- Hadoop Metadata
- These include namespace metadata, Hive Metastore data, and so on. In
- general, Hadoop metadata (where they support international characters) all
- seem to be in UTF-8.
- Hadoop Data
- This category includes file data, Hive table contents, HBase contents, etc.
- The goal is to support custom encodings as specified by the user.
- Eventually, on a view by view basis, the user may select the correct data
- encoding. Meanwhile, for applications and views that do not yet expose such
- flexibility, they may use the ``DEFAULT_SITE_ENCODING`` configuration
- variable to interpret binary Hadoop Data.
- ---------------------
- Implementation Status
- ---------------------
- HDFS
- ====
- HDFS namespace is in UTF-8. So we always encode to (and decode from) UTF-8.
- The ``libs.hadoop.fs.hadoopfs`` module speaks UTF-8 and handles this for us.
- The File Browser correctly handles HDFS namespace. *It does not yet expose file
- contents encoding in the UI*.
- Hive
- ====
- Hive metadata (such as table comments) is also in UTF-8. The
- ``beeswax.db_utils`` module converts between HUE (unicode) and Hive (UTF-8).
- Currently, Hive seems to only work with UTF-8 data.
|