html5parser.txt 3.3 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980
  1. ===============
  2. html5lib Parser
  3. ===============
  4. `html5lib`_ is a Python package that implements the HTML5 parsing algorithm
  5. which is heavily influenced by current browsers and based on the `WHATWG
  6. HTML5 specification`_.
  7. .. _html5lib: http://code.google.com/p/html5lib/
  8. .. _BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/
  9. .. _WHATWG HTML5 specification: http://www.whatwg.org/specs/web-apps/current-work/
  10. lxml can benefit from the parsing capabilities of `html5lib` through
  11. the ``lxml.html.html5parser`` module. It provides a similar interface
  12. to the ``lxml.html`` module by providing ``fromstring()``,
  13. ``parse()``, ``document_fromstring()``, ``fragment_fromstring()`` and
  14. ``fragments_fromstring()`` that work like the regular html parsing
  15. functions.
  16. Differences to regular HTML parsing
  17. ===================================
  18. There are a few differences in the returned tree to the regular HTML
  19. parsing functions from ``lxml.html``. html5lib normalizes some elements
  20. and element structures to a common format. For example even if a tables
  21. does not have a `tbody` html5lib will inject one automatically:
  22. .. sourcecode:: pycon
  23. >>> from lxml.html import tostring, html5parser
  24. >>> tostring(html5parser.fromstring("<table><td>foo"))
  25. '<table><tbody><tr><td>foo</td></tr></tbody></table>'
  26. Also the parameters the functions accept are different.
  27. Function Reference
  28. ==================
  29. ``parse(filename_url_or_file)``:
  30. Parses the named file or url, or if the object has a ``.read()``
  31. method, parses from that.
  32. ``document_fromstring(html, guess_charset=True)``:
  33. Parses a document from the given string. This always creates a
  34. correct HTML document, which means the parent node is ``<html>``,
  35. and there is a body and possibly a head.
  36. If a bytestring is passed and ``guess_charset`` is true the chardet
  37. library (if installed) will guess the charset if ambiguities exist.
  38. ``fragment_fromstring(string, create_parent=False, guess_charset=False)``:
  39. Returns an HTML fragment from a string. The fragment must contain
  40. just a single element, unless ``create_parent`` is given;
  41. e.g,. ``fragment_fromstring(string, create_parent='div')`` will
  42. wrap the element in a ``<div>``. If ``create_parent`` is true the
  43. default parent tag (div) is used.
  44. If a bytestring is passed and ``guess_charset`` is true the chardet
  45. library (if installed) will guess the charset if ambiguities exist.
  46. ``fragments_fromstring(string, no_leading_text=False, parser=None)``:
  47. Returns a list of the elements found in the fragment. The first item in
  48. the list may be a string. If ``no_leading_text`` is true, then it will
  49. be an error if there is leading text, and it will always be a list of
  50. only elements.
  51. If a bytestring is passed and ``guess_charset`` is true the chardet
  52. library (if installed) will guess the charset if ambiguities exist.
  53. ``fromstring(string)``:
  54. Returns ``document_fromstring`` or ``fragment_fromstring``, based
  55. on whether the string looks like a full document, or just a
  56. fragment.
  57. Additionally all parsing functions accept an ``parser`` keyword argument
  58. that can be set to a custom parser instance. To create custom parsers
  59. you can subclass the ``HTMLParser`` and ``XHTMLParser`` from the same
  60. module. Note that these are the parser classes provided by html5lib.