46

I'm searching in a HTML document using XPath from lxml in python. How can I get the path to a certain element? Here's the example from ruby nokogiri:

page.xpath('//text()').each do |textnode|
    path = textnode.path
    puts path
end

print for example '/html/body/div/div[1]/div[1]/p/text()[1]' and this is the string I want to get in python.

4 Answers 4

71

Use getpath from ElementTree objects.

from lxml import etree

root = etree.fromstring('''
    <foo><bar>Data</bar><bar><baz>data</baz>
    <baz>data</baz></bar></foo>
    ''')

tree = etree.ElementTree(root)
for e in root.iter():
    print(tree.getpath(e))

Prints

/foo
/foo/bar[1]
/foo/bar[2]
/foo/bar[2]/baz[1]
/foo/bar[2]/baz[2]
4
  • 1
    I think it should be for e in tree.iter():, i.e. tree.iter.
    – Jabba
    Commented Sep 17, 2011 at 23:05
  • 2
    @Jabba And why do you think that? Have you tried the code I provided the way it is? It seems to work, no? Do you have a reason to think otherwise?
    – nosklo
    Commented Sep 29, 2011 at 15:11
  • It might have not existed when you wrote this originally and not that it actually matters, but you can also do tree = root.getroottree() to get an ElementTree object. Commented Apr 13, 2017 at 18:36
  • 1
    Just for the sake of completion, when I use https://requests-html.kennethreitz.org/ to pull a webpage, I do the following: webpage = session.get('URL HERE') then root = etree.HTML(webpage.text) then tree = etree.ElementTree(root) and then simply the for loop as stated in the answer.
    – JoHKa
    Commented Nov 3, 2021 at 2:19
20

See the Xpath and XSLT with lxml from the lxml documentation This gives the path of the element containg the text

An example would be

import cStringIO
from lxml import etree

f = cStringIO.StringIO('<foo><bar><x1>hello</x1><x1>world</x1></bar></foo>')
tree = lxml.etree.parse(f)
find_text = etree.XPath("//text()")

# and print out the required data
print [tree.getpath( text.getparent()) for text in find_text(tree)]

# answer I get is 
>>> ['/foo/bar/x1[1]', '/foo/bar/x1[2]']
14

If all you have in your section of code is the element and you want the element's xpath do then element.getroottree().getpath(element) will do the job.

from lxml import etree

xml = '''
<test>
    <a/>
    <b>
       <i/>
       <ii/>
    </b>
</test>
'''
tree = etree.fromstring(xml)

for element in tree.iter():
    print element.getroottree().getpath(element)
5
root = etree.parse(open('tmp.txt'))

for e in root.iter():
    print root.getpath(e)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.