messages.txt 12 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310
  1. .. -*- mode: rst; encoding: utf-8 -*-
  2. =============================
  3. Working with Message Catalogs
  4. =============================
  5. .. contents:: Contents
  6. :depth: 3
  7. .. sectnum::
  8. Introduction
  9. ============
  10. The ``gettext`` translation system enables you to mark any strings used in your
  11. application as subject to localization, by wrapping them in functions such as
  12. ``gettext(str)`` and ``ngettext(singular, plural, num)``. For brevity, the
  13. ``gettext`` function is often aliased to ``_(str)``, so you can write:
  14. .. code-block:: python
  15. print _("Hello")
  16. instead of just:
  17. .. code-block:: python
  18. print "Hello"
  19. to make the string "Hello" localizable.
  20. Message catalogs are collections of translations for such localizable messages
  21. used in an application. They are commonly stored in PO (Portable Object) and MO
  22. (Machine Object) files, the formats of which are defined by the GNU `gettext`_
  23. tools and the GNU `translation project`_.
  24. .. _`gettext`: http://www.gnu.org/software/gettext/
  25. .. _`translation project`: http://sourceforge.net/projects/translation
  26. The general procedure for building message catalogs looks something like this:
  27. * use a tool (such as ``xgettext``) to extract localizable strings from the
  28. code base and write them to a POT (PO Template) file.
  29. * make a copy of the POT file for a specific locale (for example, "en_US")
  30. and start translating the messages
  31. * use a tool such as ``msgfmt`` to compile the locale PO file into an binary
  32. MO file
  33. * later, when code changes make it necessary to update the translations, you
  34. regenerate the POT file and merge the changes into the various
  35. locale-specific PO files, for example using ``msgmerge``
  36. Python provides the `gettext module`_ as part of the standard library, which
  37. enables applications to work with appropriately generated MO files.
  38. .. _`gettext module`: http://docs.python.org/lib/module-gettext.html
  39. As ``gettext`` provides a solid and well supported foundation for translating
  40. application messages, Babel does not reinvent the wheel, but rather reuses this
  41. infrastructure, and makes it easier to build message catalogs for Python
  42. applications.
  43. Message Extraction
  44. ==================
  45. Babel provides functionality similar to that of the ``xgettext`` program,
  46. except that only extraction from Python source files is built-in, while support
  47. for other file formats can be added using a simple extension mechanism.
  48. Unlike ``xgettext``, which is usually invoked once for every file, the routines
  49. for message extraction in Babel operate on directories. While the per-file
  50. approach of ``xgettext`` works nicely with projects using a ``Makefile``,
  51. Python projects rarely use ``make``, and thus a different mechanism is needed
  52. for extracting messages from the heterogeneous collection of source files that
  53. many Python projects are composed of.
  54. When message extraction is based on directories instead of individual files,
  55. there needs to be a way to configure which files should be treated in which
  56. manner. For example, while many projects may contain ``.html`` files, some of
  57. those files may be static HTML files that don't contain localizable message,
  58. while others may be `Django`_ templates, and still others may contain `Genshi`_
  59. markup templates. Some projects may even mix HTML files for different templates
  60. languages (for whatever reason). Therefore the way in which messages are
  61. extracted from source files can not only depend on the file extension, but
  62. needs to be controllable in a precise manner.
  63. .. _`Django`: http://www.djangoproject.com/
  64. .. _`Genshi`: http://genshi.edgewall.org/
  65. Babel accepts a configuration file to specify this mapping of files to
  66. extraction methods, which is described below.
  67. .. _`frontends`:
  68. ----------
  69. Front-Ends
  70. ----------
  71. Babel provides two different front-ends to access its functionality for working
  72. with message catalogs:
  73. * A `Command-line interface <cmdline.html>`_, and
  74. * `Integration with distutils/setuptools <setup.html>`_
  75. Which one you choose depends on the nature of your project. For most modern
  76. Python projects, the distutils/setuptools integration is probably more
  77. convenient.
  78. .. _`mapping`:
  79. -------------------------------------------
  80. Extraction Method Mapping and Configuration
  81. -------------------------------------------
  82. The mapping of extraction methods to files in Babel is done via a configuration
  83. file. This file maps extended glob patterns to the names of the extraction
  84. methods, and can also set various options for each pattern (which options are
  85. available depends on the specific extraction method).
  86. For example, the following configuration adds extraction of messages from both
  87. Genshi markup templates and text templates:
  88. .. code-block:: ini
  89. # Extraction from Python source files
  90. [python: **.py]
  91. # Extraction from Genshi HTML and text templates
  92. [genshi: **/templates/**.html]
  93. ignore_tags = script,style
  94. include_attrs = alt title summary
  95. [genshi: **/templates/**.txt]
  96. template_class = genshi.template:TextTemplate
  97. encoding = ISO-8819-15
  98. The configuration file syntax is based on the format commonly found in ``.INI``
  99. files on Windows systems, and as supported by the ``ConfigParser`` module in
  100. the Python standard library. Section names (the strings enclosed in square
  101. brackets) specify both the name of the extraction method, and the extended glob
  102. pattern to specify the files that this extraction method should be used for,
  103. separated by a colon. The options in the sections are passed to the extraction
  104. method. Which options are available is specific to the extraction method used.
  105. The extended glob patterns used in this configuration are similar to the glob
  106. patterns provided by most shells. A single asterisk (``*``) is a wildcard for
  107. any number of characters (except for the pathname component separator "/"),
  108. while a question mark (``?``) only matches a single character. In addition,
  109. two subsequent asterisk characters (``**``) can be used to make the wildcard
  110. match any directory level, so the pattern ``**.txt`` matches any file with the
  111. extension ``.txt`` in any directory.
  112. Lines that start with a ``#`` or ``;`` character are ignored and can be used
  113. for comments. Empty lines are ignored, too.
  114. .. note:: if you're performing message extraction using the command Babel
  115. provides for integration into ``setup.py`` scripts, you can also
  116. provide this configuration in a different way, namely as a keyword
  117. argument to the ``setup()`` function. See `Distutils/Setuptools
  118. Integration`_ for more information.
  119. .. _`distutils/setuptools integration`: setup.html
  120. Default Extraction Methods
  121. --------------------------
  122. Babel comes with only two builtin extractors: ``python`` (which extracts
  123. messages from Python source files) and ``ignore`` (which extracts nothing).
  124. The ``python`` extractor is by default mapped to the glob pattern ``**.py``,
  125. meaning it'll be applied to all files with the ``.py`` extension in any
  126. directory. If you specify your own mapping configuration, this default mapping
  127. is discarded, so you need to explicitly add it to your mapping (as shown in the
  128. example above.)
  129. .. _`referencing extraction methods`:
  130. Referencing Extraction Methods
  131. ------------------------------
  132. To be able to use short extraction method names such as “genshi”, you need to
  133. have `pkg_resources`_ installed, and the package implementing that extraction
  134. method needs to have been installed with its meta data (the `egg-info`_).
  135. If this is not possible for some reason, you need to map the short names to
  136. fully qualified function names in an extract section in the mapping
  137. configuration. For example:
  138. .. code-block:: ini
  139. # Some custom extraction method
  140. [extractors]
  141. custom = mypackage.module:extract_custom
  142. [custom: **.ctm]
  143. some_option = foo
  144. Note that the builtin extraction methods ``python`` and ``ignore`` are available
  145. by default, even if `pkg_resources`_ is not installed. You should never need to
  146. explicitly define them in the ``[extractors]`` section.
  147. .. _`egg-info`: http://peak.telecommunity.com/DevCenter/PythonEggs
  148. .. _`pkg_resources`: http://peak.telecommunity.com/DevCenter/PkgResources
  149. --------------------------
  150. Writing Extraction Methods
  151. --------------------------
  152. Adding new methods for extracting localizable methods is easy. First, you'll
  153. need to implement a function that complies with the following interface:
  154. .. code-block:: python
  155. def extract_xxx(fileobj, keywords, comment_tags, options):
  156. """Extract messages from XXX files.
  157. :param fileobj: the file-like object the messages should be extracted
  158. from
  159. :param keywords: a list of keywords (i.e. function names) that should
  160. be recognized as translation functions
  161. :param comment_tags: a list of translator tags to search for and
  162. include in the results
  163. :param options: a dictionary of additional options (optional)
  164. :return: an iterator over ``(lineno, funcname, message, comments)``
  165. tuples
  166. :rtype: ``iterator``
  167. """
  168. .. note:: Any strings in the tuples produced by this function must be either
  169. ``unicode`` objects, or ``str`` objects using plain ASCII characters.
  170. That means that if sources contain strings using other encodings, it
  171. is the job of the extractor implementation to do the decoding to
  172. ``unicode`` objects.
  173. Next, you should register that function as an entry point. This requires your
  174. ``setup.py`` script to use `setuptools`_, and your package to be installed with
  175. the necessary metadata. If that's taken care of, add something like the
  176. following to your ``setup.py`` script:
  177. .. code-block:: python
  178. def setup(...
  179. entry_points = """
  180. [babel.extractors]
  181. xxx = your.package:extract_xxx
  182. """,
  183. That is, add your extraction method to the entry point group
  184. ``babel.extractors``, where the name of the entry point is the name that people
  185. will use to reference the extraction method, and the value being the module and
  186. the name of the function (separated by a colon) implementing the actual
  187. extraction.
  188. .. note:: As shown in `Referencing Extraction Methods`_, declaring an entry
  189. point is not strictly required, as users can still reference the
  190. extraction function directly. But whenever possible, the entry point
  191. should be declared to make configuration more convenient.
  192. .. _`setuptools`: http://peak.telecommunity.com/DevCenter/setuptools
  193. -------------------
  194. Translator Comments
  195. -------------------
  196. First of all what are comments tags. Comments tags are excerpts of text to
  197. search for in comments, only comments, right before the `python gettext`_
  198. calls, as shown on the following example:
  199. .. _`python gettext`: http://docs.python.org/lib/module-gettext.html
  200. .. code-block:: python
  201. # NOTE: This is a comment about `Foo Bar`
  202. _('Foo Bar')
  203. The comments tag for the above example would be ``NOTE:``, and the translator
  204. comment for that tag would be ``This is a comment about `Foo Bar```.
  205. The resulting output in the catalog template would be something like::
  206. #. This is a comment about `Foo Bar`
  207. #: main.py:2
  208. msgid "Foo Bar"
  209. msgstr ""
  210. Now, you might ask, why would I need that?
  211. Consider this simple case; you have a menu item called “manual”. You know what
  212. it means, but when the translator sees this they will wonder did you mean:
  213. 1. a document or help manual, or
  214. 2. a manual process?
  215. This is the simplest case where a translation comment such as
  216. “The installation manual” helps to clarify the situation and makes a translator
  217. more productive.
  218. .. note:: Whether translator comments can be extracted depends on the extraction
  219. method in use. The Python extractor provided by Babel does implement
  220. this feature, but others may not.