futurize.rst 9.9 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321
  1. .. _forwards-conversion:
  2. ``futurize``: Py2 to Py2/3
  3. --------------------------
  4. .. include:: futurize_overview.rst
  5. .. _forwards-conversion-stage1:
  6. Stage 1: "safe" fixes
  7. ~~~~~~~~~~~~~~~~~~~~~
  8. Run the first stage of the conversion process with::
  9. futurize --stage1 mypackage/*.py
  10. or, if you are using zsh, recursively::
  11. futurize --stage1 mypackage/**/*.py
  12. This applies fixes that modernize Python 2 code without changing the effect of
  13. the code. With luck, this will not introduce any bugs into the code, or will at
  14. least be trivial to fix. The changes are those that bring the Python code
  15. up-to-date without breaking Py2 compatibility. The resulting code will be
  16. modern Python 2.6-compatible code plus ``__future__`` imports from the
  17. following set:
  18. .. code-block:: python
  19. from __future__ import absolute_import
  20. from __future__ import division
  21. from __future__ import print_function
  22. Only those ``__future__`` imports deemed necessary will be added unless
  23. the ``--all-imports`` command-line option is passed to ``futurize``, in
  24. which case they are all added.
  25. The ``from __future__ import unicode_literals`` declaration is not added
  26. unless the ``--unicode-literals`` flag is passed to ``futurize``.
  27. The changes include::
  28. - except MyException, e:
  29. + except MyException as e:
  30. - print >>stderr, "Blah"
  31. + from __future__ import print_function
  32. + print("Blah", stderr)
  33. - class MyClass:
  34. + class MyClass(object):
  35. - def next(self):
  36. + def __next__(self):
  37. - if d.has_key(key):
  38. + if key in d:
  39. Implicit relative imports fixed, e.g.::
  40. - import mymodule
  41. + from __future__ import absolute_import
  42. + from . import mymodule
  43. .. and all unprefixed string literals '...' gain a b prefix to be b'...'.
  44. .. (This last step can be prevented using --no-bytes-literals if you already have b'...' markup in your code, whose meaning would otherwise be lost.)
  45. Stage 1 does not add any imports from the ``future`` package. The output of
  46. stage 1 will probably not (yet) run on Python 3.
  47. The goal for this stage is to create most of the ``diff`` for the entire
  48. porting process, but without introducing any bugs. It should be uncontroversial
  49. and safe to apply to every Python 2 package. The subsequent patches introducing
  50. Python 3 compatibility should then be shorter and easier to review.
  51. The complete set of fixers applied by ``futurize --stage1`` is:
  52. .. code-block:: python
  53. lib2to3.fixes.fix_apply
  54. lib2to3.fixes.fix_except
  55. lib2to3.fixes.fix_exitfunc
  56. lib2to3.fixes.fix_funcattrs
  57. lib2to3.fixes.fix_has_key
  58. lib2to3.fixes.fix_idioms
  59. lib2to3.fixes.fix_intern
  60. lib2to3.fixes.fix_isinstance
  61. lib2to3.fixes.fix_methodattrs
  62. lib2to3.fixes.fix_ne
  63. lib2to3.fixes.fix_numliterals
  64. lib2to3.fixes.fix_paren
  65. lib2to3.fixes.fix_reduce
  66. lib2to3.fixes.fix_renames
  67. lib2to3.fixes.fix_repr
  68. lib2to3.fixes.fix_standarderror
  69. lib2to3.fixes.fix_sys_exc
  70. lib2to3.fixes.fix_throw
  71. lib2to3.fixes.fix_tuple_params
  72. lib2to3.fixes.fix_types
  73. lib2to3.fixes.fix_ws_comma
  74. lib2to3.fixes.fix_xreadlines
  75. libfuturize.fixes.fix_absolute_import
  76. libfuturize.fixes.fix_next_call
  77. libfuturize.fixes.fix_print_with_import
  78. libfuturize.fixes.fix_raise
  79. The following fixers from ``lib2to3`` are not applied:
  80. .. code-block:: python
  81. lib2to3.fixes.fix_import
  82. The ``fix_absolute_import`` fixer in ``libfuturize.fixes`` is applied instead of
  83. ``lib2to3.fixes.fix_import``. The new fixer both makes implicit relative
  84. imports explicit and adds the declaration ``from __future__ import
  85. absolute_import`` at the top of each relevant module.
  86. .. code-block:: python
  87. lib2to3.fixes.fix_next
  88. The ``fix_next_call`` fixer in ``libfuturize.fixes`` is applied instead of
  89. ``fix_next`` in stage 1. The new fixer changes any ``obj.next()`` calls to
  90. ``next(obj)``, which is Py2/3 compatible, but doesn't change any ``next`` method
  91. names to ``__next__``, which would break Py2 compatibility.
  92. ``fix_next`` is applied in stage 2.
  93. .. code-block:: python
  94. lib2to3.fixes.fix_print
  95. The ``fix_print_with_import`` fixer in ``libfuturize.fixes`` changes the code to
  96. use print as a function and also adds ``from __future__ import
  97. print_function`` to the top of modules using ``print()``.
  98. In addition, it avoids adding an extra set of parentheses if these already
  99. exist. So ``print(x)`` does not become ``print((x))``.
  100. .. code-block:: python
  101. lib2to3.fixes.fix_raise
  102. This fixer translates code to use the Python 3-only ``with_traceback()``
  103. method on exceptions.
  104. .. code-block:: python
  105. lib2to3.fixes.fix_set_literal
  106. This converts ``set([1, 2, 3]``) to ``{1, 2, 3}``, breaking Python 2.6 support.
  107. .. code-block:: python
  108. lib2to3.fixes.fix_ws_comma
  109. This performs cosmetic changes. This is not applied by default because it
  110. does not serve to improve Python 2/3 compatibility. (In some cases it may
  111. also reduce readability: see issue #58.)
  112. .. _forwards-conversion-stage2:
  113. Stage 2: Py3-style code with wrappers for Py2
  114. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  115. Run stage 2 of the conversion process with::
  116. futurize --stage2 myfolder/*.py
  117. This stage adds a dependency on the ``future`` package. The goal for stage 2 is
  118. to make further mostly safe changes to the Python 2 code to use Python 3-style
  119. code that then still runs on Python 2 with the help of the appropriate builtins
  120. and utilities in ``future``.
  121. For example::
  122. name = raw_input('What is your name?\n')
  123. for k, v in d.iteritems():
  124. assert isinstance(v, basestring)
  125. class MyClass(object):
  126. def __unicode__(self):
  127. return u'My object'
  128. def __str__(self):
  129. return unicode(self).encode('utf-8')
  130. would be converted by Stage 2 to this code::
  131. from builtins import input
  132. from builtins import str
  133. from future.utils import iteritems, python_2_unicode_compatible
  134. name = input('What is your name?\n')
  135. for k, v in iteritems(d):
  136. assert isinstance(v, (str, bytes))
  137. @python_2_unicode_compatible
  138. class MyClass(object):
  139. def __str__(self):
  140. return u'My object'
  141. Stage 2 also renames standard-library imports to their Py3 names and adds these
  142. two lines::
  143. from future import standard_library
  144. standard_library.install_aliases()
  145. For example::
  146. import ConfigParser
  147. becomes::
  148. from future import standard_library
  149. standard_library.install_aliases()
  150. import configparser
  151. The complete list of fixers applied in Stage 2 is::
  152. lib2to3.fixes.fix_basestring
  153. lib2to3.fixes.fix_dict
  154. lib2to3.fixes.fix_exec
  155. lib2to3.fixes.fix_getcwdu
  156. lib2to3.fixes.fix_input
  157. lib2to3.fixes.fix_itertools
  158. lib2to3.fixes.fix_itertools_imports
  159. lib2to3.fixes.fix_filter
  160. lib2to3.fixes.fix_long
  161. lib2to3.fixes.fix_map
  162. lib2to3.fixes.fix_nonzero
  163. lib2to3.fixes.fix_operator
  164. lib2to3.fixes.fix_raw_input
  165. lib2to3.fixes.fix_zip
  166. libfuturize.fixes.fix_cmp
  167. libfuturize.fixes.fix_division
  168. libfuturize.fixes.fix_execfile
  169. libfuturize.fixes.fix_future_builtins
  170. libfuturize.fixes.fix_future_standard_library
  171. libfuturize.fixes.fix_future_standard_library_urllib
  172. libfuturize.fixes.fix_metaclass
  173. libpasteurize.fixes.fix_newstyle
  174. libfuturize.fixes.fix_object
  175. libfuturize.fixes.fix_unicode_keep_u
  176. libfuturize.fixes.fix_xrange_with_import
  177. Not applied::
  178. lib2to3.fixes.fix_buffer # Perhaps not safe. Test this.
  179. lib2to3.fixes.fix_callable # Not needed in Py3.2+
  180. lib2to3.fixes.fix_execfile # Some problems: see issue #37.
  181. # We use the custom libfuturize.fixes.fix_execfile instead.
  182. lib2to3.fixes.fix_future # Removing __future__ imports is bad for Py2 compatibility!
  183. lib2to3.fixes.fix_imports # Called by libfuturize.fixes.fix_future_standard_library
  184. lib2to3.fixes.fix_imports2 # We don't handle this yet (dbm)
  185. lib2to3.fixes.fix_metaclass # Causes SyntaxError in Py2! Use the one from ``six`` instead
  186. lib2to3.fixes.fix_unicode # Strips off the u'' prefix, which removes a potentially
  187. # helpful source of information for disambiguating
  188. # unicode/byte strings.
  189. lib2to3.fixes.fix_urllib # Included in libfuturize.fix_future_standard_library_urllib
  190. lib2to3.fixes.fix_xrange # Custom one because of a bug with Py3.3's lib2to3
  191. Fixes applied with the ``futurize --conservative`` option::
  192. libfuturize.fixes.fix_division_safe # instead of libfuturize.fixes.fix_division.
  193. .. Ideally the output of this stage should not be a ``SyntaxError`` on either
  194. .. Python 3 or Python 2.
  195. .. _forwards-conversion-text:
  196. Separating text from bytes
  197. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  198. After applying stage 2, the recommended step is to decide which of your Python
  199. 2 strings represent text and which represent binary data and to prefix all
  200. string literals with either ``b`` or ``u`` accordingly. Furthermore, to ensure
  201. that these types behave similarly on Python 2 as on Python 3, also wrap
  202. byte-strings or text in the ``bytes`` and ``str`` types from ``future``. For
  203. example::
  204. from builtins import bytes, str
  205. b = bytes(b'\x00ABCD')
  206. s = str(u'This is normal text')
  207. Any unadorned string literals will then represent native platform strings
  208. (byte-strings on Py2, unicode strings on Py3).
  209. An alternative is to pass the ``--unicode-literals`` flag::
  210. $ futurize --unicode-literals mypython2script.py
  211. After running this, all string literals that were not explicitly marked up as
  212. ``b''`` will mean text (Python 3 ``str`` or Python 2 ``unicode``).
  213. .. _forwards-conversion-stage3:
  214. Post-conversion
  215. ~~~~~~~~~~~~~~~
  216. After running ``futurize``, we recommend first running your tests on Python 3 and making further code changes until they pass on Python 3.
  217. The next step would be manually tweaking the code to re-enable Python 2
  218. compatibility with the help of the ``future`` package. For example, you can add
  219. the ``@python_2_unicode_compatible`` decorator to any classes that define custom
  220. ``__str__`` methods. See :ref:`what-else` for more info.