2

I have an XML file containing texts in some languages. I want to extract the texts in just one language and store them in a separate file. How can I do this? Here is some of the beginning lines of my file:

<?xml version="1.0" encoding="UTF-8"?>
<tmx version="1.4b">
  <header creationtool="ORESAligner" creationtoolversion="1.0" datatype="plaintext" segtype="paragraph" adminlang="en-us" srclang="EN" o-tmf="ORES"/>
  <body>
    <tu tuid="55_100:6">
      <prop type="session">55</prop>
      <prop type="committee">3</prop>
      <tuv xml:lang="EN">
        <seg>RESOLUTION 55/100</seg>
      </tuv>
      <tuv xml:lang="AR">
        <seg>القرار 55/100</seg>
      </tuv>
      <tuv xml:lang="ZH">
        <seg>第55/100号决议</seg>
      </tuv>
      <tuv xml:lang="FR">
        <seg>RÉSOLUTION 55/100</seg>
      </tuv>
      <tuv xml:lang="RU">
        <seg>РЕЗОЛЮЦИЯ 55/100</seg>
      </tuv>
      <tuv xml:lang="ES">
        <seg>RESOLUCIÓN 55/100</seg>
      </tuv>
    </tu>
  </body>
</tmx>

Now say I want just English texts. the desired output should be:

RESOLUTION 55/100

How should I use this script? I am newbie in working XML files, and don't know how can I use this XPath expression. As I know xmlstarlet is able to modify XML files. But I don't know how...?

0

4 Answers 4

3

Extract English Nodes with XmlStarlet

You could use xmlstarlet to query your XML using XPath, and return just the nodes with an English-language attribute. For example:

$ xmlstarlet sel -t -v "//tuv[@xml:lang='EN']/seg/text()" /tmp/foo
RESOLUTION 55/100

Store Node Values in a File with Language Extension

If you want to store those values in some language-based file, then you could dump the values of each node found into a file with a language-based extension (e.g. "EN" for English).

# Don't overwrite LANG; use some other variable.
language='EN'

xmlstarlet sel \
    --noblanks \
    --text \
    --template \
    --match "//tuv[@xml:lang='${language}']" \
    --value-of seg \
    -n \
    /tmp/foo > "/tmp/foo.$language"

With this example, the contents of all matching nodes will be written to /tmp/foo.EN for further processing. You can certainly adjust the shell redirection to suit any additional requirements.

0
1

If the xml file is well formatted, you can use a simple sed command:

sed -n '/xml:lang="EN"/ {
N
s_.*<seg>\([^<]*\)</seg>_\1_p
}
' input_file

Description:

sed -n '/xml:lang="EN"/ {           # 1) exec sed with no print flag, find a line
                                    # matching xml:lang="EN"
N                                   # 2) read the next line
s_.*<seg>\([^<]*\)</seg>_\1_p       # 3) replace everything until </seg> with 
                                    # the text between <seg> and </seg> and print
}
' input_file

If you want to keep the seg tags you can change the 3rd step:

sed -n '/xml:lang="EN"/ {
N
s_.*\(<seg>[^<]*</seg>\)_\1_p
}
' input_file
4
  • Thank you Perreal. It works pretty good. But what if I want to keep <seg> and </seg> tags? How should I change this command? May I ask you to describe this script?
    – Hakim
    Jul 22, 2012 at 11:28
  • 1
    Horrible suggestion! XML is not designed to be processed in this way. It will only work if the format of the XML is exactly known, including irrelevant details such as where the newlines and spaces appear. Use an XML parser to process XML, always. Downvoting. Jul 22, 2012 at 11:43
  • So what do you recommend? Which XML Parser is easier to work with?
    – Hakim
    Jul 22, 2012 at 11:45
  • Depends on your environment, e.g. Java, .NET, PHP, etc. If you're working with files at the shell level, consider xmlsh. Jul 22, 2012 at 11:51
1

The following XPath expression extracts the information you are looking for:

/tmx/body/tu/tuv[@xml:lang='EN']/seg

There are many tools that allow you to process XML files using XPath expressions. If you are working from the command line you could look at xmlsh.

It's hard to tell the context of the requirement, but I would imagine that as it grows beyond the simple case given here, you will want to look at XSLT and/or XQuery.

1
  • 1
    How should I use it? I am newbie in working XML files, and don't know how can I use this XPath expression. As I know xmlstarlet is able to modify XML files. But I don't know how...?
    – Hakim
    Jul 22, 2012 at 12:15
0

You can use the command line tool xml_grep like this:

xml_grep --cond "tu/tuv[@xml:lang='EN']/seg" --text_only file.tmx

The argument to --cond is an XPath-like expression. Its syntax is similar to what xstarlet etc. expect, but not identical.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.