| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321 |
- .. _forwards-conversion:
- ``futurize``: Py2 to Py2/3
- --------------------------
- .. include:: futurize_overview.rst
- .. _forwards-conversion-stage1:
- Stage 1: "safe" fixes
- ~~~~~~~~~~~~~~~~~~~~~
- Run the first stage of the conversion process with::
- futurize --stage1 mypackage/*.py
- or, if you are using zsh, recursively::
- futurize --stage1 mypackage/**/*.py
- This applies fixes that modernize Python 2 code without changing the effect of
- the code. With luck, this will not introduce any bugs into the code, or will at
- least be trivial to fix. The changes are those that bring the Python code
- up-to-date without breaking Py2 compatibility. The resulting code will be
- modern Python 2.6-compatible code plus ``__future__`` imports from the
- following set:
- .. code-block:: python
- from __future__ import absolute_import
- from __future__ import division
- from __future__ import print_function
- Only those ``__future__`` imports deemed necessary will be added unless
- the ``--all-imports`` command-line option is passed to ``futurize``, in
- which case they are all added.
- The ``from __future__ import unicode_literals`` declaration is not added
- unless the ``--unicode-literals`` flag is passed to ``futurize``.
- The changes include::
- - except MyException, e:
- + except MyException as e:
- - print >>stderr, "Blah"
- + from __future__ import print_function
- + print("Blah", stderr)
- - class MyClass:
- + class MyClass(object):
- - def next(self):
- + def __next__(self):
- - if d.has_key(key):
- + if key in d:
- Implicit relative imports fixed, e.g.::
- - import mymodule
- + from __future__ import absolute_import
- + from . import mymodule
- .. and all unprefixed string literals '...' gain a b prefix to be b'...'.
- .. (This last step can be prevented using --no-bytes-literals if you already have b'...' markup in your code, whose meaning would otherwise be lost.)
- Stage 1 does not add any imports from the ``future`` package. The output of
- stage 1 will probably not (yet) run on Python 3.
- The goal for this stage is to create most of the ``diff`` for the entire
- porting process, but without introducing any bugs. It should be uncontroversial
- and safe to apply to every Python 2 package. The subsequent patches introducing
- Python 3 compatibility should then be shorter and easier to review.
- The complete set of fixers applied by ``futurize --stage1`` is:
- .. code-block:: python
- lib2to3.fixes.fix_apply
- lib2to3.fixes.fix_except
- lib2to3.fixes.fix_exitfunc
- lib2to3.fixes.fix_funcattrs
- lib2to3.fixes.fix_has_key
- lib2to3.fixes.fix_idioms
- lib2to3.fixes.fix_intern
- lib2to3.fixes.fix_isinstance
- lib2to3.fixes.fix_methodattrs
- lib2to3.fixes.fix_ne
- lib2to3.fixes.fix_numliterals
- lib2to3.fixes.fix_paren
- lib2to3.fixes.fix_reduce
- lib2to3.fixes.fix_renames
- lib2to3.fixes.fix_repr
- lib2to3.fixes.fix_standarderror
- lib2to3.fixes.fix_sys_exc
- lib2to3.fixes.fix_throw
- lib2to3.fixes.fix_tuple_params
- lib2to3.fixes.fix_types
- lib2to3.fixes.fix_ws_comma
- lib2to3.fixes.fix_xreadlines
- libfuturize.fixes.fix_absolute_import
- libfuturize.fixes.fix_next_call
- libfuturize.fixes.fix_print_with_import
- libfuturize.fixes.fix_raise
- The following fixers from ``lib2to3`` are not applied:
- .. code-block:: python
- lib2to3.fixes.fix_import
- The ``fix_absolute_import`` fixer in ``libfuturize.fixes`` is applied instead of
- ``lib2to3.fixes.fix_import``. The new fixer both makes implicit relative
- imports explicit and adds the declaration ``from __future__ import
- absolute_import`` at the top of each relevant module.
- .. code-block:: python
- lib2to3.fixes.fix_next
- The ``fix_next_call`` fixer in ``libfuturize.fixes`` is applied instead of
- ``fix_next`` in stage 1. The new fixer changes any ``obj.next()`` calls to
- ``next(obj)``, which is Py2/3 compatible, but doesn't change any ``next`` method
- names to ``__next__``, which would break Py2 compatibility.
- ``fix_next`` is applied in stage 2.
- .. code-block:: python
- lib2to3.fixes.fix_print
- The ``fix_print_with_import`` fixer in ``libfuturize.fixes`` changes the code to
- use print as a function and also adds ``from __future__ import
- print_function`` to the top of modules using ``print()``.
- In addition, it avoids adding an extra set of parentheses if these already
- exist. So ``print(x)`` does not become ``print((x))``.
- .. code-block:: python
- lib2to3.fixes.fix_raise
- This fixer translates code to use the Python 3-only ``with_traceback()``
- method on exceptions.
- .. code-block:: python
- lib2to3.fixes.fix_set_literal
- This converts ``set([1, 2, 3]``) to ``{1, 2, 3}``, breaking Python 2.6 support.
- .. code-block:: python
- lib2to3.fixes.fix_ws_comma
- This performs cosmetic changes. This is not applied by default because it
- does not serve to improve Python 2/3 compatibility. (In some cases it may
- also reduce readability: see issue #58.)
- .. _forwards-conversion-stage2:
- Stage 2: Py3-style code with wrappers for Py2
- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Run stage 2 of the conversion process with::
- futurize --stage2 myfolder/*.py
- This stage adds a dependency on the ``future`` package. The goal for stage 2 is
- to make further mostly safe changes to the Python 2 code to use Python 3-style
- code that then still runs on Python 2 with the help of the appropriate builtins
- and utilities in ``future``.
- For example::
- name = raw_input('What is your name?\n')
- for k, v in d.iteritems():
- assert isinstance(v, basestring)
- class MyClass(object):
- def __unicode__(self):
- return u'My object'
- def __str__(self):
- return unicode(self).encode('utf-8')
- would be converted by Stage 2 to this code::
- from builtins import input
- from builtins import str
- from future.utils import iteritems, python_2_unicode_compatible
- name = input('What is your name?\n')
- for k, v in iteritems(d):
- assert isinstance(v, (str, bytes))
- @python_2_unicode_compatible
- class MyClass(object):
- def __str__(self):
- return u'My object'
- Stage 2 also renames standard-library imports to their Py3 names and adds these
- two lines::
- from future import standard_library
- standard_library.install_aliases()
- For example::
- import ConfigParser
- becomes::
- from future import standard_library
- standard_library.install_aliases()
- import configparser
- The complete list of fixers applied in Stage 2 is::
- lib2to3.fixes.fix_basestring
- lib2to3.fixes.fix_dict
- lib2to3.fixes.fix_exec
- lib2to3.fixes.fix_getcwdu
- lib2to3.fixes.fix_input
- lib2to3.fixes.fix_itertools
- lib2to3.fixes.fix_itertools_imports
- lib2to3.fixes.fix_filter
- lib2to3.fixes.fix_long
- lib2to3.fixes.fix_map
- lib2to3.fixes.fix_nonzero
- lib2to3.fixes.fix_operator
- lib2to3.fixes.fix_raw_input
- lib2to3.fixes.fix_zip
- libfuturize.fixes.fix_cmp
- libfuturize.fixes.fix_division
- libfuturize.fixes.fix_execfile
- libfuturize.fixes.fix_future_builtins
- libfuturize.fixes.fix_future_standard_library
- libfuturize.fixes.fix_future_standard_library_urllib
- libfuturize.fixes.fix_metaclass
- libpasteurize.fixes.fix_newstyle
- libfuturize.fixes.fix_object
- libfuturize.fixes.fix_unicode_keep_u
- libfuturize.fixes.fix_xrange_with_import
- Not applied::
- lib2to3.fixes.fix_buffer # Perhaps not safe. Test this.
- lib2to3.fixes.fix_callable # Not needed in Py3.2+
- lib2to3.fixes.fix_execfile # Some problems: see issue #37.
- # We use the custom libfuturize.fixes.fix_execfile instead.
- lib2to3.fixes.fix_future # Removing __future__ imports is bad for Py2 compatibility!
- lib2to3.fixes.fix_imports # Called by libfuturize.fixes.fix_future_standard_library
- lib2to3.fixes.fix_imports2 # We don't handle this yet (dbm)
- lib2to3.fixes.fix_metaclass # Causes SyntaxError in Py2! Use the one from ``six`` instead
- lib2to3.fixes.fix_unicode # Strips off the u'' prefix, which removes a potentially
- # helpful source of information for disambiguating
- # unicode/byte strings.
- lib2to3.fixes.fix_urllib # Included in libfuturize.fix_future_standard_library_urllib
- lib2to3.fixes.fix_xrange # Custom one because of a bug with Py3.3's lib2to3
- Fixes applied with the ``futurize --conservative`` option::
- libfuturize.fixes.fix_division_safe # instead of libfuturize.fixes.fix_division.
- .. Ideally the output of this stage should not be a ``SyntaxError`` on either
- .. Python 3 or Python 2.
- .. _forwards-conversion-text:
- Separating text from bytes
- ~~~~~~~~~~~~~~~~~~~~~~~~~~
- After applying stage 2, the recommended step is to decide which of your Python
- 2 strings represent text and which represent binary data and to prefix all
- string literals with either ``b`` or ``u`` accordingly. Furthermore, to ensure
- that these types behave similarly on Python 2 as on Python 3, also wrap
- byte-strings or text in the ``bytes`` and ``str`` types from ``future``. For
- example::
- from builtins import bytes, str
- b = bytes(b'\x00ABCD')
- s = str(u'This is normal text')
- Any unadorned string literals will then represent native platform strings
- (byte-strings on Py2, unicode strings on Py3).
- An alternative is to pass the ``--unicode-literals`` flag::
- $ futurize --unicode-literals mypython2script.py
- After running this, all string literals that were not explicitly marked up as
- ``b''`` will mean text (Python 3 ``str`` or Python 2 ``unicode``).
- .. _forwards-conversion-stage3:
- Post-conversion
- ~~~~~~~~~~~~~~~
- After running ``futurize``, we recommend first running your tests on Python 3 and making further code changes until they pass on Python 3.
- The next step would be manually tweaking the code to re-enable Python 2
- compatibility with the help of the ``future`` package. For example, you can add
- the ``@python_2_unicode_compatible`` decorator to any classes that define custom
- ``__str__`` methods. See :ref:`what-else` for more info.
|