README.rst 8.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243
  1. ===============
  2. pycparser v2.18
  3. ===============
  4. :Author: `Eli Bendersky <http://eli.thegreenplace.net>`_
  5. .. contents::
  6. :backlinks: none
  7. .. sectnum::
  8. Introduction
  9. ============
  10. What is pycparser?
  11. ------------------
  12. **pycparser** is a parser for the C language, written in pure Python. It is a
  13. module designed to be easily integrated into applications that need to parse
  14. C source code.
  15. What is it good for?
  16. --------------------
  17. Anything that needs C code to be parsed. The following are some uses for
  18. **pycparser**, taken from real user reports:
  19. * C code obfuscator
  20. * Front-end for various specialized C compilers
  21. * Static code checker
  22. * Automatic unit-test discovery
  23. * Adding specialized extensions to the C language
  24. One of the most popular uses of **pycparser** is in the `cffi
  25. <https://cffi.readthedocs.io/en/latest/>`_ library, which uses it to parse the
  26. declarations of C functions and types in order to auto-generate FFIs.
  27. **pycparser** is unique in the sense that it's written in pure Python - a very
  28. high level language that's easy to experiment with and tweak. To people familiar
  29. with Lex and Yacc, **pycparser**'s code will be simple to understand. It also
  30. has no external dependencies (except for a Python interpreter), making it very
  31. simple to install and deploy.
  32. Which version of C does pycparser support?
  33. ------------------------------------------
  34. **pycparser** aims to support the full C99 language (according to the standard
  35. ISO/IEC 9899). Some features from C11 are also supported, and patches to support
  36. more are welcome.
  37. **pycparser** supports very few GCC extensions, but it's fairly easy to set
  38. things up so that it parses code with a lot of GCC-isms successfully. See the
  39. `FAQ <https://github.com/eliben/pycparser/wiki/FAQ>`_ for more details.
  40. What grammar does pycparser follow?
  41. -----------------------------------
  42. **pycparser** very closely follows the C grammar provided in Annex A of the C99
  43. standard (ISO/IEC 9899).
  44. How is pycparser licensed?
  45. --------------------------
  46. `BSD license <https://github.com/eliben/pycparser/blob/master/LICENSE>`_.
  47. Contact details
  48. ---------------
  49. For reporting problems with **pycparser** or submitting feature requests, please
  50. open an `issue <https://github.com/eliben/pycparser/issues>`_, or submit a
  51. pull request.
  52. Installing
  53. ==========
  54. Prerequisites
  55. -------------
  56. * **pycparser** was tested on Python 2.7, 3.4 and 3.5, on both Linux and
  57. Windows. It should work on any later version (in both the 2.x and 3.x lines)
  58. as well.
  59. * **pycparser** has no external dependencies. The only non-stdlib library it
  60. uses is PLY, which is bundled in ``pycparser/ply``. The current PLY version is
  61. 3.10, retrieved from `<http://www.dabeaz.com/ply/>`_
  62. Installation process
  63. --------------------
  64. Installing **pycparser** is very simple. Once you download and unzip the
  65. package, you just have to execute the standard ``python setup.py install``. The
  66. setup script will then place the ``pycparser`` module into ``site-packages`` in
  67. your Python's installation library.
  68. Alternatively, since **pycparser** is listed in the `Python Package Index
  69. <http://pypi.python.org/pypi/pycparser>`_ (PyPI), you can install it using your
  70. favorite Python packaging/distribution tool, for example with::
  71. > pip install pycparser
  72. Known problems
  73. --------------
  74. * Some users who've installed a new version of **pycparser** over an existing
  75. version ran into a problem using the newly installed library. This has to do
  76. with parse tables staying around as ``.pyc`` files from the older version. If
  77. you see unexplained errors from **pycparser** after an upgrade, remove it (by
  78. deleting the ``pycparser`` directory in your Python's ``site-packages``, or
  79. wherever you installed it) and install again.
  80. Using
  81. =====
  82. Interaction with the C preprocessor
  83. -----------------------------------
  84. In order to be compilable, C code must be preprocessed by the C preprocessor -
  85. ``cpp``. ``cpp`` handles preprocessing directives like ``#include`` and
  86. ``#define``, removes comments, and performs other minor tasks that prepare the C
  87. code for compilation.
  88. For all but the most trivial snippets of C code **pycparser**, like a C
  89. compiler, must receive preprocessed C code in order to function correctly. If
  90. you import the top-level ``parse_file`` function from the **pycparser** package,
  91. it will interact with ``cpp`` for you, as long as it's in your PATH, or you
  92. provide a path to it.
  93. Note also that you can use ``gcc -E`` or ``clang -E`` instead of ``cpp``. See
  94. the ``using_gcc_E_libc.py`` example for more details. Windows users can download
  95. and install a binary build of Clang for Windows `from this website
  96. <http://llvm.org/releases/download.html>`_.
  97. What about the standard C library headers?
  98. ------------------------------------------
  99. C code almost always ``#include``\s various header files from the standard C
  100. library, like ``stdio.h``. While (with some effort) **pycparser** can be made to
  101. parse the standard headers from any C compiler, it's much simpler to use the
  102. provided "fake" standard includes in ``utils/fake_libc_include``. These are
  103. standard C header files that contain only the bare necessities to allow valid
  104. parsing of the files that use them. As a bonus, since they're minimal, it can
  105. significantly improve the performance of parsing large C files.
  106. The key point to understand here is that **pycparser** doesn't really care about
  107. the semantics of types. It only needs to know whether some token encountered in
  108. the source is a previously defined type. This is essential in order to be able
  109. to parse C correctly.
  110. See `this blog post
  111. <http://eli.thegreenplace.net/2015/on-parsing-c-type-declarations-and-fake-headers>`_
  112. for more details.
  113. Basic usage
  114. -----------
  115. Take a look at the ``examples`` directory of the distribution for a few examples
  116. of using **pycparser**. These should be enough to get you started.
  117. Advanced usage
  118. --------------
  119. The public interface of **pycparser** is well documented with comments in
  120. ``pycparser/c_parser.py``. For a detailed overview of the various AST nodes
  121. created by the parser, see ``pycparser/_c_ast.cfg``.
  122. There's also a `FAQ available here <https://github.com/eliben/pycparser/wiki/FAQ>`_.
  123. In any case, you can always drop me an `email <eliben@gmail.com>`_ for help.
  124. Modifying
  125. =========
  126. There are a few points to keep in mind when modifying **pycparser**:
  127. * The code for **pycparser**'s AST nodes is automatically generated from a
  128. configuration file - ``_c_ast.cfg``, by ``_ast_gen.py``. If you modify the AST
  129. configuration, make sure to re-generate the code.
  130. * Make sure you understand the optimized mode of **pycparser** - for that you
  131. must read the docstring in the constructor of the ``CParser`` class. For
  132. development you should create the parser without optimizations, so that it
  133. will regenerate the Yacc and Lex tables when you change the grammar.
  134. Package contents
  135. ================
  136. Once you unzip the ``pycparser`` package, you'll see the following files and
  137. directories:
  138. README.rst:
  139. This README file.
  140. LICENSE:
  141. The pycparser license
  142. setup.py:
  143. Installation script
  144. examples/:
  145. A directory with some examples of using **pycparser**
  146. pycparser/:
  147. The **pycparser** module source code.
  148. tests/:
  149. Unit tests.
  150. utils/fake_libc_include:
  151. Minimal standard C library include files that should allow to parse any C code.
  152. utils/internal/:
  153. Internal utilities for my own use. You probably don't need them.
  154. Contributors
  155. ============
  156. Some people have contributed to **pycparser** by opening issues on bugs they've
  157. found and/or submitting patches. The list of contributors is in the CONTRIBUTORS
  158. file in the source distribution. After **pycparser** moved to Github I stopped
  159. updating this list because Github does a much better job at tracking
  160. contributions.
  161. CI Status
  162. =========
  163. **pycparser** has automatic testing enabled through the convenient
  164. `Travis CI project <https://travis-ci.org>`_. Here is the latest build status:
  165. .. image:: https://travis-ci.org/eliben/pycparser.png?branch=master
  166. :align: center
  167. :target: https://travis-ci.org/eliben/pycparser
  168. AppVeyor also helps run tests on Windows:
  169. .. image:: https://ci.appveyor.com/api/projects/status/wrup68o5y8nuk1i9?svg=true
  170. :align: center
  171. :target: https://ci.appveyor.com/project/eliben/pycparser/