build.txt 10 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307
  1. How to build lxml from source
  2. =============================
  3. To build lxml from source, you need libxml2 and libxslt properly
  4. installed, *including the header files*. These are likely shipped in
  5. separate ``-dev`` or ``-devel`` packages like ``libxml2-dev``, which
  6. you must install before trying to build lxml. The build process also
  7. requires setuptools_. The lxml source distribution comes with a
  8. script called ``ez_setup.py`` that can be used to install them.
  9. .. _setuptools: http://peak.telecommunity.com/DevCenter/setuptools
  10. .. contents::
  11. ..
  12. 1 Cython
  13. 2 Subversion
  14. 3 Setuptools
  15. 4 Running the tests and reporting errors
  16. 5 Contributing an egg
  17. 6 Static linking on Windows
  18. 7 Building Debian packages from SVN sources
  19. Cython
  20. ------
  21. .. _EasyInstall: http://peak.telecommunity.com/DevCenter/EasyInstall
  22. .. _Cython: http://www.cython.org
  23. The lxml.etree and lxml.objectify modules are written in Cython_.
  24. Since we distribute the Cython-generated .c files with lxml releases,
  25. however, you do not need Cython to build lxml from the normal release
  26. sources. We even encourage you to *not install Cython* for a normal
  27. release build, as the generated C code can vary quite heavily between
  28. Cython versions, which may or may not generate correct code for lxml.
  29. The pre-generated release sources were tested and therefore are known
  30. to work.
  31. So, if you want a reliable build of lxml, we suggest to a) use a
  32. source release of lxml and b) disable or uninstall Cython for the
  33. build.
  34. *Only* if you are interested in building lxml from a Subversion
  35. checkout (e.g. to test a bug fix that has not been release yet) or if
  36. want to be an lxml developer, then you do need a working Cython
  37. installation. You can use EasyInstall_ to install it::
  38. easy_install Cython==0.11
  39. lxml currently requires Cython 0.11, later release versions should
  40. work as well.
  41. Subversion
  42. ----------
  43. The lxml package is developed in a Subversion repository. You can retrieve
  44. the current developer version by calling::
  45. svn co http://codespeak.net/svn/lxml/trunk lxml
  46. This will create a directory ``lxml`` and download the source into it.
  47. You can also browse the `Subversion repository`_ through the web, use
  48. your favourite SVN client to access it, or browse the `Subversion
  49. history`_.
  50. .. _`Subversion repository`: http://codespeak.net/svn/lxml/
  51. .. _`Subversion history`: https://codespeak.net/viewvc/lxml/
  52. Setuptools
  53. ----------
  54. Usually, building lxml is done through setuptools. Do a Subversion checkout
  55. (or download the source tar-ball and unpack it) and then type::
  56. python setup.py build
  57. or::
  58. python setup.py bdist_egg
  59. If you want to test lxml from the source directory, it is better to build it
  60. in-place like this::
  61. python setup.py build_ext -i
  62. or, in Unix-like environments::
  63. make
  64. If you get errors about missing header files (e.g. ``libxml/xmlversion.h``)
  65. then you need to make sure the development packages of both libxml2
  66. and libxslt are properly installed. Try passing the following option to
  67. setup.py to make sure the right config is found::
  68. python setup.py build --with-xslt-config=/path/to/xslt-config
  69. If this doesn't help, you may have to add the location of the header
  70. files to the include path like::
  71. python setup.py build_ext -i -I /usr/include/libxml2
  72. where the file is in ``/usr/include/libxml2/libxml/xmlversion.h``
  73. To use lxml.etree in-place, you can place lxml's ``src`` directory on your
  74. Python module search path (PYTHONPATH) and then import ``lxml.etree`` to play
  75. with it::
  76. # cd lxml
  77. # PYTHONPATH=src python
  78. Python 2.5.1
  79. Type "help", "copyright", "credits" or "license" for more information.
  80. >>> from lxml import etree
  81. >>>
  82. To recompile after changes, note that you may have to run ``make clean`` or
  83. delete the file ``src/lxml/etree.c``. Distutils do not automatically pick up
  84. changes that affect files other than the main file ``src/lxml/etree.pyx``.
  85. Running the tests and reporting errors
  86. --------------------------------------
  87. The source distribution (tgz) and the Subversion repository contain a test
  88. suite for lxml. You can run it from the top-level directory::
  89. python test.py
  90. Note that the test script only tests the in-place build (see distutils
  91. building above), as it searches the ``src`` directory. You can use the
  92. following one-step command to trigger an in-place build and test it::
  93. make test
  94. This also runs the ElementTree and cElementTree compatibility tests. To call
  95. them separately, make sure you have lxml on your PYTHONPATH first, then run::
  96. python selftest.py
  97. and::
  98. python selftest2.py
  99. If the tests give failures, errors, or worse, segmentation faults, we'd really
  100. like to know. Please contact us on the `mailing list`_, and please specify
  101. the version of lxml, libxml2, libxslt and Python you were using, as well as
  102. your operating system type (Linux, Windows, MacOs, ...).
  103. .. _`mailing list`: http://codespeak.net/mailman/listinfo/lxml-dev
  104. Building an egg
  105. ---------------
  106. This is the procedure to make an lxml egg for your platform (assuming
  107. that you have setuptools_ installed):
  108. * Download the lxml-x.y.tar.gz release. This contains the pregenerated C so
  109. that you can be sure you build exactly from the release sources. Unpack
  110. them and cd into the resulting directory.
  111. * python setup.py build
  112. * If you're on a unixy platform, cd into ``build/lib.your.platform`` and strip
  113. any ``.so`` file you find there. This reduces the size of the egg
  114. considerably.
  115. * ``python setup.py bdist_egg``
  116. This will put the egg into the ``dist`` directory.
  117. Building lxml on MacOS-X
  118. ------------------------
  119. Apple regularly ships new system releases with horribly outdated
  120. system libraries. This is specifically the case for libxml2 and
  121. libxslt, where the system provided versions are too old to build lxml.
  122. While the Unix environment in MacOS-X makes it relatively easy to
  123. install Unix/Linux style package management tools and new software, it
  124. actually seems to be hard to get libraries set up for exclusive usage
  125. that MacOS-X ships in an older version. Alternative distributions
  126. (like macports) install their libraries in addition to the system
  127. libraries, but the compiler and the runtime loader on MacOS still sees
  128. the system libraries before the new libraries. This can lead to
  129. undebuggable crashes where the newer library seems to be loaded but
  130. the older system library is used.
  131. Apple discourages static building against libraries, which would help
  132. working around this problem. Apple does not ship static library
  133. binaries with its system and several package management systems follow
  134. this decision. Therefore, building static binaries requires building
  135. the dependencies first. The ``setup.py`` script does this
  136. automatically when you call it like this::
  137. python setup.py build --static-deps
  138. This will download and build the latest versions of libxml2 and
  139. libxslt from the official FTP download site. If you want to use
  140. specific versions, or want to prevent any online access, you can
  141. download both ``tar.gz`` release files yourself, place them into a
  142. subdirectory ``libs`` in the lxml distribution, and call ``setup.py``
  143. with the desired target versions like this::
  144. python setup.py build --static-deps \
  145. --libxml2-version=2.7.3 \
  146. --libxslt-version=1.1.24 \
  147. Instead of ``build``, you can use any target, like ``bdist_egg`` if
  148. you want to use setuptools to build an installable egg.
  149. Static linking on Windows
  150. -------------------------
  151. Most operating systems have proper package management that makes installing
  152. current versions of libxml2 and libxslt easy. The most famous exception is
  153. Microsoft Windows, which entirely lacks these capabilities. It can therefore
  154. be interesting to statically link the external libraries into lxml.etree to
  155. avoid having to install them separately.
  156. Download lxml and all required libraries to the same directory. The iconv,
  157. libxml2, libxslt, and zlib libraries are all available from the ftp site
  158. ftp://ftp.zlatkovic.com/pub/libxml/.
  159. Your directory should now have the following files in it (although most likely
  160. different versions)::
  161. iconv-1.9.1.win32.zip
  162. libxml2-2.6.23.win32.zip
  163. libxslt-1.1.15.win32.zip
  164. lxml-1.0.0.tgz
  165. zlib-1.2.3.win32.zip
  166. Now extract each of those files in the *same* directory. This should give you
  167. something like this::
  168. iconv-1.9.1.win32/
  169. iconv-1.9.1.win32.zip
  170. libxml2-2.6.23.win32/
  171. libxml2-2.6.23.win32.zip
  172. libxslt-1.1.15.win32/
  173. libxslt-1.1.15.win32.zip
  174. lxml-1.0.0/
  175. lxml-1.0.0.tgz
  176. zlib-1.2.3.win32/
  177. zlib-1.2.3.win32.zip
  178. Go to the lxml directory and edit the file ``setup.py``. There should
  179. be a section near the top that looks like this::
  180. STATIC_INCLUDE_DIRS = []
  181. STATIC_LIBRARY_DIRS = []
  182. STATIC_CFLAGS = []
  183. Change this section to something like this, but take care to use the correct
  184. version numbers::
  185. STATIC_INCLUDE_DIRS = [
  186. "..\\libxml2-2.6.23.win32\\include",
  187. "..\\libxslt-1.1.15.win32\\include",
  188. "..\\zlib-1.2.3.win32\\include",
  189. "..\\iconv-1.9.1.win32\\include"
  190. ]
  191. STATIC_LIBRARY_DIRS = [
  192. "..\\libxml2-2.6.23.win32\\lib",
  193. "..\\libxslt-1.1.15.win32\\lib",
  194. "..\\zlib-1.2.3.win32\\lib",
  195. "..\\iconv-1.9.1.win32\\lib"
  196. ]
  197. STATIC_CFLAGS = []
  198. Add any CFLAGS you might consider useful to the third list. Now you
  199. should be able to pass the ``--static`` option to setup.py and
  200. everything should work well. Try calling::
  201. python setup.py bdist_wininst --static
  202. This will create a windows installer in the ``pkg`` directory.
  203. Building Debian packages from SVN sources
  204. -----------------------------------------
  205. `Andreas Pakulat`_ proposed the following approach.
  206. .. _`Andreas Pakulat`: http://codespeak.net/pipermail/lxml-dev/2006-May/001254.html
  207. * ``apt-get source lxml``
  208. * remove the unpacked directory
  209. * tar.gz the lxml SVN version and replace the orig.tar.gz that lies in the
  210. directory
  211. * check md5sum of created tar.gz file and place new sum and size in dsc file
  212. * do ``dpkg-source -x lxml-[VERSION].dsc`` and cd into the newly created directory
  213. * run ``dch -i`` and add a comment like "use trunk version", this will
  214. increase the debian version number so apt/dpkg won't get confused
  215. * run ``dpkg-buildpackage -rfakeroot -us -uc`` to build the package
  216. In case ``dpkg-buildpackage`` tells you that some dependecies are missing, you
  217. can either install them manually or run ``apt-get build-dep lxml``.
  218. That will give you .deb packages in the parent directory which can be
  219. installed using ``dpkg -i``.