i18n.rst 1.9 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455
  1. .. -*- coding: utf-8 -*-
  2. =============================================
  3. Status of Internationalization (i18n) Support
  4. =============================================
  5. .. Note::
  6. This documents the design of i18n in HUE, as well as the various degrees of
  7. i18n support in different HUE applications.
  8. ---------------
  9. Design Overview
  10. ---------------
  11. Thanks to Django, HUE always receives unicode data from the browser. Data
  12. operations within the boundary of HUE should be done in unicode as much as
  13. possible. Data encoding occurs when the data exits HUE, in one of the following
  14. ways:
  15. Model Storage
  16. These are data to go into a Django model, such as user firstname, lastname,
  17. Beeswax saved query. Since the Django model layer handles unicode data, HUE
  18. does not need to do any encoding.
  19. Hadoop Metadata
  20. These include namespace metadata, Hive Metastore data, and so on. In
  21. general, Hadoop metadata (where they support international characters) all
  22. seem to be in UTF-8.
  23. Hadoop Data
  24. This category includes file data, Hive table contents, HBase contents, etc.
  25. The goal is to support custom encodings as specified by the user.
  26. Eventually, on a view by view basis, the user may select the correct data
  27. encoding. Meanwhile, for applications and views that do not yet expose such
  28. flexibility, they may use the ``DEFAULT_SITE_ENCODING`` configuration
  29. variable to interpret binary Hadoop Data.
  30. ---------------------
  31. Implementation Status
  32. ---------------------
  33. HDFS
  34. ====
  35. HDFS namespace is in UTF-8. So we always encode to (and decode from) UTF-8.
  36. The ``libs.hadoop.fs.hadoopfs`` module speaks UTF-8 and handles this for us.
  37. The File Browser correctly handles HDFS namespace. *It does not yet expose file
  38. contents encoding in the UI*.
  39. Hive
  40. ====
  41. Hive metadata (such as table comments) is also in UTF-8. The
  42. ``beeswax.db_utils`` module converts between HUE (unicode) and Hive (UTF-8).
  43. Currently, Hive seems to only work with UTF-8 data.