| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608 |
- ====================================
- Python extensions for XPath and XSLT
- ====================================
- This document describes how to use Python extension functions in XPath
- and XSLT like this:
- .. sourcecode:: xml
- <xsl:value-of select="f:myPythonFunction(.//sometag)" />
- and extension elements in XSLT as in the following example:
- .. sourcecode:: xml
- <xsl:template match="*">
- <my:python-extension>
- <some-content />
- </my:python-extension>
- </xsl:template>
- .. contents::
- ..
- 1 XPath Extension functions
- 1.1 The FunctionNamespace
- 1.2 Global prefix assignment
- 1.3 The XPath context
- 1.4 Evaluators and XSLT
- 1.5 Evaluator-local extensions
- 1.6 What to return from a function
- 2 XSLT extension elements
- 2.1 Declaring extension elements
- 2.2 Applying XSL templates
- 2.3 Working with read-only elements
- ..
- >>> try: from StringIO import StringIO
- ... except ImportError:
- ... from io import BytesIO
- ... def StringIO(s):
- ... if isinstance(s, str): s = s.encode("UTF-8")
- ... return BytesIO(s)
- XPath Extension functions
- =========================
- Here is how an extension function looks like. As the first argument,
- it always receives a context object (see below). The other arguments
- are provided by the respective call in the XPath expression, one in
- the following examples. Any number of arguments is allowed:
- .. sourcecode:: pycon
- >>> def hello(context, a):
- ... return "Hello %s" % a
- >>> def ola(context, a):
- ... return "Ola %s" % a
- >>> def loadsofargs(context, *args):
- ... return "Got %d arguments." % len(args)
- The FunctionNamespace
- ---------------------
- In order to use a function in XPath or XSLT, it needs to have a
- (namespaced) name by which it can be called during evaluation. This
- is done using the FunctionNamespace class. For simplicity, we choose
- the empty namespace (None):
- .. sourcecode:: pycon
- >>> from lxml import etree
- >>> ns = etree.FunctionNamespace(None)
- >>> ns['hello'] = hello
- >>> ns['countargs'] = loadsofargs
- This registers the function `hello` with the name `hello` in the default
- namespace (None), and the function `loadsofargs` with the name `countargs`.
- Now we're going to create a document that we can run XPath expressions
- against:
- .. sourcecode:: pycon
- >>> root = etree.XML('<a><b>Haegar</b></a>')
- >>> doc = etree.ElementTree(root)
- Done. Now we can have XPath expressions call our new function:
- .. sourcecode:: pycon
- >>> print(root.xpath("hello('Dr. Falken')"))
- Hello Dr. Falken
- >>> print(root.xpath('hello(local-name(*))'))
- Hello b
- >>> print(root.xpath('hello(string(b))'))
- Hello Haegar
- >>> print(root.xpath('countargs(., b, ./*)'))
- Got 3 arguments.
- Note how we call both a Python function (`hello`) and an XPath built-in
- function (`string`) in exactly the same way. Normally, however, you would
- want to separate the two in different namespaces. The FunctionNamespace class
- allows you to do this:
- .. sourcecode:: pycon
- >>> ns = etree.FunctionNamespace('http://mydomain.org/myfunctions')
- >>> ns['hello'] = hello
- >>> prefixmap = {'f' : 'http://mydomain.org/myfunctions'}
- >>> print(root.xpath('f:hello(local-name(*))', namespaces=prefixmap))
- Hello b
- Global prefix assignment
- ------------------------
- In the last example, you had to specify a prefix for the function namespace.
- If you always use the same prefix for a function namespace, you can also
- register it with the namespace:
- .. sourcecode:: pycon
- >>> ns = etree.FunctionNamespace('http://mydomain.org/myother/functions')
- >>> ns.prefix = 'es'
- >>> ns['hello'] = ola
- >>> print(root.xpath('es:hello(local-name(*))'))
- Ola b
- This is a global assignment, so take care not to assign the same prefix to
- more than one namespace. The resulting behaviour in that case is completely
- undefined. It is always a good idea to consistently use the same meaningful
- prefix for each namespace throughout your application.
- The prefix assignment only works with functions and FunctionNamespace objects,
- not with the general Namespace object that registers element classes. The
- reasoning is that elements in lxml do not care about prefixes anyway, so it
- would rather complicate things than be of any help.
- The XPath context
- -----------------
- Functions get a context object as first parameter. In lxml 1.x, this value
- was None, but since lxml 2.0 it provides two properties: ``eval_context`` and
- ``context_node``. The context node is the Element where the current function
- is called:
- .. sourcecode:: pycon
- >>> def print_tag(context, nodes):
- ... print("%s: %s" % (context.context_node.tag, [ n.tag for n in nodes ]))
- >>> ns = etree.FunctionNamespace('http://mydomain.org/printtag')
- >>> ns.prefix = "pt"
- >>> ns["print_tag"] = print_tag
- >>> ignore = root.xpath("//*[pt:print_tag(.//*)]")
- a: ['b']
- b: []
- The ``eval_context`` is a dictionary that is local to the evaluation. It
- allows functions to keep state:
- .. sourcecode:: pycon
- >>> def print_context(context):
- ... context.eval_context[context.context_node.tag] = "done"
- ... print(sorted(context.eval_context.items()))
- >>> ns["print_context"] = print_context
- >>> ignore = root.xpath("//*[pt:print_context()]")
- [('a', 'done')]
- [('a', 'done'), ('b', 'done')]
- Evaluators and XSLT
- -------------------
- Extension functions work for all ways of evaluating XPath expressions and for
- XSL transformations:
- .. sourcecode:: pycon
- >>> e = etree.XPathEvaluator(doc)
- >>> print(e('es:hello(local-name(/a))'))
- Ola a
- >>> namespaces = {'f' : 'http://mydomain.org/myfunctions'}
- >>> e = etree.XPathEvaluator(doc, namespaces=namespaces)
- >>> print(e('f:hello(local-name(/a))'))
- Hello a
- >>> xslt = etree.XSLT(etree.XML('''
- ... <stylesheet version="1.0"
- ... xmlns="http://www.w3.org/1999/XSL/Transform"
- ... xmlns:es="http://mydomain.org/myother/functions">
- ... <output method="text" encoding="ASCII"/>
- ... <template match="/">
- ... <value-of select="es:hello(string(//b))"/>
- ... </template>
- ... </stylesheet>
- ... '''))
- >>> print(xslt(doc))
- Ola Haegar
- It is also possible to register namespaces with a single evaluator after its
- creation. While the following example involves no functions, the idea should
- still be clear:
- .. sourcecode:: pycon
-
- >>> f = StringIO('<a xmlns="http://mydomain.org/myfunctions" />')
- >>> ns_doc = etree.parse(f)
- >>> e = etree.XPathEvaluator(ns_doc)
- >>> e('/a')
- []
- This returns nothing, as we did not ask for the right namespace. When we
- register the namespace with the evaluator, however, we can access it via a
- prefix:
- .. sourcecode:: pycon
- >>> e.register_namespace('foo', 'http://mydomain.org/myfunctions')
- >>> e('/foo:a')[0].tag
- '{http://mydomain.org/myfunctions}a'
- Note that this prefix mapping is only known to this evaluator, as opposed to
- the global mapping of the FunctionNamespace objects:
- .. sourcecode:: pycon
- >>> e2 = etree.XPathEvaluator(ns_doc)
- >>> e2('/foo:a')
- Traceback (most recent call last):
- ...
- lxml.etree.XPathEvalError: Undefined namespace prefix
- Evaluator-local extensions
- --------------------------
- Apart from the global registration of extension functions, there is also a way
- of making extensions known to a single Evaluator or XSLT. All evaluators and
- the XSLT object accept a keyword argument ``extensions`` in their constructor.
- The value is a dictionary mapping (namespace, name) tuples to functions:
- .. sourcecode:: pycon
- >>> extensions = {('local-ns', 'local-hello') : hello}
- >>> namespaces = {'l' : 'local-ns'}
- >>> e = etree.XPathEvaluator(doc, namespaces=namespaces, extensions=extensions)
- >>> print(e('l:local-hello(string(b))'))
- Hello Haegar
- For larger numbers of extension functions, you can define classes or modules
- and use the ``Extension`` helper:
- .. sourcecode:: pycon
- >>> class MyExt:
- ... def function1(self, _, arg):
- ... return '1'+arg
- ... def function2(self, _, arg):
- ... return '2'+arg
- ... def function3(self, _, arg):
- ... return '3'+arg
- >>> ext_module = MyExt()
- >>> functions = ('function1', 'function2')
- >>> extensions = etree.Extension( ext_module, functions, ns='local-ns' )
- >>> e = etree.XPathEvaluator(doc, namespaces=namespaces, extensions=extensions)
- >>> print(e('l:function1(string(b))'))
- 1Haegar
- The optional second argument to ``Extension`` can either be be a
- sequence of names to select from the module, a dictionary that
- explicitly maps function names to their XPath alter-ego or ``None``
- (explicitly passed) to take all available functions under their
- original name (if their name does not start with '_').
- The additional ``ns`` keyword argument takes a namespace URI or
- ``None`` (also if left out) for the default namespace. The following
- examples will therefore all do the same thing:
- .. sourcecode:: pycon
- >>> functions = ('function1', 'function2', 'function3')
- >>> extensions = etree.Extension( ext_module, functions )
- >>> e = etree.XPathEvaluator(doc, extensions=extensions)
- >>> print(e('function1(function2(function3(string(b))))'))
- 123Haegar
- >>> extensions = etree.Extension( ext_module, functions, ns=None )
- >>> e = etree.XPathEvaluator(doc, extensions=extensions)
- >>> print(e('function1(function2(function3(string(b))))'))
- 123Haegar
- >>> extensions = etree.Extension(ext_module)
- >>> e = etree.XPathEvaluator(doc, extensions=extensions)
- >>> print(e('function1(function2(function3(string(b))))'))
- 123Haegar
- >>> functions = {
- ... 'function1' : 'function1',
- ... 'function2' : 'function2',
- ... 'function3' : 'function3'
- ... }
- >>> extensions = etree.Extension(ext_module, functions)
- >>> e = etree.XPathEvaluator(doc, extensions=extensions)
- >>> print(e('function1(function2(function3(string(b))))'))
- 123Haegar
- For convenience, you can also pass a sequence of extensions:
- .. sourcecode:: pycon
- >>> extensions1 = etree.Extension(ext_module)
- >>> extensions2 = etree.Extension(ext_module, ns='local-ns')
- >>> e = etree.XPathEvaluator(doc, extensions=[extensions1, extensions2],
- ... namespaces=namespaces)
- >>> print(e('function1(l:function2(function3(string(b))))'))
- 123Haegar
- What to return from a function
- ------------------------------
- .. _`XPath return values`: xpathxslt.html#xpath-return-values
- Extension functions can return any data type for which there is an XPath
- equivalent (see the documentation on `XPath return values`). This includes
- numbers, boolean values, elements and lists of elements. Note that integers
- will also be returned as floats:
- .. sourcecode:: pycon
- >>> def returnsFloat(_):
- ... return 1.7
- >>> def returnsInteger(_):
- ... return 1
- >>> def returnsBool(_):
- ... return True
- >>> def returnFirstNode(_, nodes):
- ... return nodes[0]
- >>> ns = etree.FunctionNamespace(None)
- >>> ns['float'] = returnsFloat
- >>> ns['int'] = returnsInteger
- >>> ns['bool'] = returnsBool
- >>> ns['first'] = returnFirstNode
- >>> e = etree.XPathEvaluator(doc)
- >>> e("float()")
- 1.7
- >>> e("int()")
- 1.0
- >>> int( e("int()") )
- 1
- >>> e("bool()")
- True
- >>> e("count(first(//b))")
- 1.0
- As the last example shows, you can pass the results of functions back into
- the XPath expression. Elements and sequences of elements are treated as
- XPath node-sets:
- .. sourcecode:: pycon
- >>> def returnsNodeSet(_):
- ... results1 = etree.Element('results1')
- ... etree.SubElement(results1, 'result').text = "Alpha"
- ... etree.SubElement(results1, 'result').text = "Beta"
- ...
- ... results2 = etree.Element('results2')
- ... etree.SubElement(results2, 'result').text = "Gamma"
- ... etree.SubElement(results2, 'result').text = "Delta"
- ...
- ... results3 = etree.SubElement(results2, 'subresult')
- ... return [results1, results2, results3]
- >>> ns['new-node-set'] = returnsNodeSet
- >>> e = etree.XPathEvaluator(doc)
- >>> r = e("new-node-set()/result")
- >>> print([ t.text for t in r ])
- ['Alpha', 'Beta', 'Gamma', 'Delta']
- >>> r = e("new-node-set()")
- >>> print([ t.tag for t in r ])
- ['results1', 'results2', 'subresult']
- >>> print([ len(t) for t in r ])
- [2, 3, 0]
- >>> r[0][0].text
- 'Alpha'
- >>> etree.tostring(r[0])
- b'<results1><result>Alpha</result><result>Beta</result></results1>'
- >>> etree.tostring(r[1])
- b'<results2><result>Gamma</result><result>Delta</result><subresult/></results2>'
- >>> etree.tostring(r[2])
- b'<subresult/>'
- The current implementation deep-copies newly created elements in node-sets.
- Only the elements and their children are passed on, no outlying parents or
- tail texts will be available in the result. This also means that in the above
- example, the `subresult` elements in `results2` and `results3` are no longer
- identical within the node-set, they belong to independent trees:
- .. sourcecode:: pycon
- >>> print("%s - %s" % (r[1][-1].tag, r[2].tag))
- subresult - subresult
- >>> print(r[1][-1] == r[2])
- False
- >>> print(r[1][-1].getparent().tag)
- results2
- >>> print(r[2].getparent())
- None
- This is an implementation detail that you should be aware of, but you should
- avoid relying on it in your code. Note that elements taken from the source
- document (the most common case) do not suffer from this restriction. They
- will always be passed unchanged.
- XSLT extension elements
- =======================
- Just like the XPath extension functions described above, lxml supports
- custom extension *elements* in XSLT. This means, you can write XSLT
- code like this:
- .. sourcecode:: xml
- <xsl:template match="*">
- <my:python-extension>
- <some-content />
- </my:python-extension>
- </xsl:template>
- And then you can implement the element in Python like this:
- .. sourcecode:: pycon
- >>> class MyExtElement(etree.XSLTExtension):
- ... def execute(self, context, self_node, input_node, output_parent):
- ... print("Hello from XSLT!")
- ... output_parent.text = "I did it!"
- ... # just copy own content input to output
- ... output_parent.extend( list(self_node) )
- The arguments passed to the ``.execute()`` method are
- context
- The opaque evaluation context. You need this when calling back
- into the XSLT processor.
- self_node
- A read-only Element object that represents the extension element
- in the stylesheet.
- input_node
- The current context Element in the input document (also read-only).
- output_parent
- The current insertion point in the output document. You can
- append elements or set the text value (not the tail). Apart from
- that, the Element is read-only.
- Declaring extension elements
- ----------------------------
- In XSLT, extension elements can be used like any other XSLT element,
- except that they must be declared as extensions using the standard
- XSLT ``extension-element-prefixes`` option:
- .. sourcecode:: pycon
- >>> xslt_ext_tree = etree.XML('''
- ... <xsl:stylesheet version="1.0"
- ... xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- ... xmlns:my="testns"
- ... extension-element-prefixes="my">
- ... <xsl:template match="/">
- ... <foo><my:ext><child>XYZ</child></my:ext></foo>
- ... </xsl:template>
- ... <xsl:template match="child">
- ... <CHILD>--xyz--</CHILD>
- ... </xsl:template>
- ... </xsl:stylesheet>''')
- To register the extension, add its namespace and name to the extension
- mapping of the XSLT object:
- .. sourcecode:: pycon
- >>> my_extension = MyExtElement()
- >>> extensions = { ('testns', 'ext') : my_extension }
- >>> transform = etree.XSLT(xslt_ext_tree, extensions = extensions)
- Note how we pass an instance here, not the class of the extension.
- Now we can run the transformation and see how our extension is
- called:
- .. sourcecode:: pycon
- >>> root = etree.XML('<dummy/>')
- >>> result = transform(root)
- Hello from XSLT!
- >>> str(result)
- '<?xml version="1.0"?>\n<foo>I did it!<child>XYZ</child></foo>\n'
- Applying XSL templates
- ----------------------
- XSLT extensions are a very powerful feature that allows you to
- interact directly with the XSLT processor. You have full read-only
- access to the input document and the stylesheet, and you can even call
- back into the XSLT processor to process templates. Here is an example
- that passes an Element into the ``.apply_templates()`` method of the
- ``XSLTExtension`` instance:
- .. sourcecode:: pycon
- >>> class MyExtElement(etree.XSLTExtension):
- ... def execute(self, context, self_node, input_node, output_parent):
- ... child = self_node[0]
- ... results = self.apply_templates(context, child)
- ... output_parent.append(results[0])
- >>> my_extension = MyExtElement()
- >>> extensions = { ('testns', 'ext') : my_extension }
- >>> transform = etree.XSLT(xslt_ext_tree, extensions = extensions)
- >>> root = etree.XML('<dummy/>')
- >>> result = transform(root)
- >>> str(result)
- '<?xml version="1.0"?>\n<foo><CHILD>--xyz--</CHILD></foo>\n'
- Here, we applied the templates to a child of the extension element
- itself, i.e. to an element inside the stylesheet instead of an element
- of the input document.
- The return value of ``.apply_templates()`` is always a list. It may
- contain a mix of elements and strings, collected from the XSLT processing
- result. If you want to append these values to the output parent, be aware
- that you cannot use the ``.append()`` method to add strings. In many
- cases, you would only be interested in elements anyway, so you can discard
- strings (e.g. formatting whitespace) and append the rest.
- If you want to include string results in the output, you can either build
- an appropriate tree yourself and append that, or you can manually add the
- string values to the current output tree, e.g. by concatenating them with
- the ``.tail`` of the last element that was appended.
- Note that you can also let lxml build the result tree for you by passing
- the ``output_parent`` into the ``.apply_templates()`` method. In this
- case, the result will be None and all content found by applying templates
- will be appended to the output parent.
- If you do not care about string results at all, e.g. because you already
- know that they will only contain whitespace, you can pass the option
- ``elements_only=True`` to the ``.apply_templates()`` method, or pass
- ``remove_blank_text=True`` to remove only those strings that consist
- entirely of whitespace.
- Working with read-only elements
- -------------------------------
- There is one important thing to keep in mind: all Elements that the
- ``execute()`` method gets to deal with are read-only Elements, so you
- cannot modify them. They also will not easily work in the API. For
- example, you cannot pass them to the ``tostring()`` function or wrap
- them in an ``ElementTree``.
- What you can do, however, is to deepcopy them to make them normal
- Elements, and then modify them using the normal etree API. So this
- will work:
- .. sourcecode:: pycon
- >>> from copy import deepcopy
- >>> class MyExtElement(etree.XSLTExtension):
- ... def execute(self, context, self_node, input_node, output_parent):
- ... child = deepcopy(self_node[0])
- ... child.text = "NEW TEXT"
- ... output_parent.append(child)
- >>> my_extension = MyExtElement()
- >>> extensions = { ('testns', 'ext') : my_extension }
- >>> transform = etree.XSLT(xslt_ext_tree, extensions = extensions)
- >>> root = etree.XML('<dummy/>')
- >>> result = transform(root)
- >>> str(result)
- '<?xml version="1.0"?>\n<foo><child>NEW TEXT</child></foo>\n'
|