How to extract data from XML nodes in Scala

Problem: In a Scala application,you want to extract information from XML you receive,so you can use the data in your application.

Solution

Use the methods of theScalaElemandNodeSeqclasses to extract the data. The most commonly used methods of theElemclass are shown here:

Commonly used methods of the Elem class

Method                 Description
------                 -----------

x  "div"              Searches the XML literal x for elements of type <div>. 
                       Only searches immediate child nodes (no grandchild or “descendant” nodes).

x  "div"             Searches the XML literal x for elements of type <div>. Returns matching 
                       elements from child nodes at any depth of the XML tree.

x.attribute("class")   Returns the value of the given attribute in the current node.
                       <a x="10" y="20">foo</a>.attribute("x")   // returns Some(10).

x.attributes           Returns all attributes of the current node,prefixed and unprefixed,in no particular order.
                       scala> <a x="10" y="20">foo</a>.attributes
                       res0: scala.xml.MetaData =  x="10" y="20"

x.child                Returns the children of the current node.
                       <a><b>foo</b></a>.child   // returns <b>foo</b>.

x.copy(...)            Returns a copy of the element,letting you replace data during the 
                       copy process.

x.label                The name of the current element. 
                       <a><b>foo</b></a>.label   // returns a.

x.text                 Returns a concatenation of text(n) for each child n.

x.toString             Emits the XML literal as a String. 
                       Use scala.xml.PrettyPrinter to format the output,if desired.

Examples

The following examples demonstrate most of the methods just shown. Given this XML literal:

scala> val x = <div class="content"><p>Hello</p><p>world</p></div>
x: scala.xml.Elem = <div class="content"><p>Hello</p><p>world</p></div>

you can search for and extract subelements with theandXPath methods:

scala> x  "p"
res0: scala.xml.NodeSeq = NodeSeq(<p>Hello</p>,<p>world</p>)

scala> x  "p"
res1: scala.xml.NodeSeq = NodeSeq(<p>Hello</p>,<p>world</p>)

These methods will be demonstrated more in subsequent recipes.

The label method returns the name of the current element. A<p>tag returns p,a<div>tag returns div,etc.:

scala> x.label
res2: String = div

scala> <name>Joe</name>.label
res3: String = name

Thetextmethod returns the text from all subelements,which the Scaladoc describes as,“a concatenation of all text(n) for each child n”:

scala> x.text
res4: String = Helloworld

Later examples will demonstrate how to improve on this result.

Element attributes are extracted with theattributeorattributesmethods. The following examples demonstrate how to call these methods,and the values they return:

scala> x.attribute("class")
res5: Option[Seq[scala.xml.Node]] = Some(content)

scala> x.attributes("class")
res6: Seq[scala.xml.Node] = content

scala> x.attributes.get("class")
res7: Option[Seq[scala.xml.Node]] = Some(content)

The following examples demonstrate how those same method calls behave when you search for an attribute that doesn’t exist:

scala> x.attribute("foo")
res8: Option[Seq[scala.xml.Node]] = None

scala> x.attributes("foo")
res9: Seq[scala.xml.Node] = null

scala> x.attributes.get("foo")
res10: Option[Seq[scala.xml.Node]] = None

scala> x.attributes.get("foo").getOrElse("N/A")
res11: Object = N/A

To demonstrate more ways to work with element attributes,let’s create a new element:

scala> val w = <forecast day="Thu" date="10 Nov 2011" low="37" high="58" />
w: scala.xml.Elem = <forecast day="Thu" date="10 Nov 2011" low="37" high="58" />

These examples show howattributeandattributeswork with multiple attributes:

scala> w.attribute("day")
res0: Option[Seq[scala.xml.Node]] = Some(Thu)

scala> w.attributes("day")
res1: Seq[scala.xml.Node] = Thu

scala> w.attributes
res2: scala.xml.MetaData =  day="Thu" date="10 Nov 2011" low="37" high="58"

These examples show how to iterate over a set of attributes:

scala> for (a <- w.attributes) println(s"key: ${a.key},value: ${a.value}")
key: day,value: Thu
key: date,value: 10 Nov 2011
key: low,value: 37
key: high,value: 58

scala> w.attributes.asAttrMap
res3: Map[String,String] = Map(low -> 37,date -> 10 Nov 2011,day -> Thu,high -> 58)

Child elements

childmethod returns all child nodes of the current element. To demonstrate this,let’s create a new XML variable:

scala> val p = <person><name>Ken</name><age>23</age></person>
p: scala.xml.Elem = <person><name>Ken</name><age>23</age></person>

childmethod returns immediate child nodes:

scala> p.child
res0: Seq[scala.xml.Node] = ArrayBuffer(<name>Ken</name>,<age>23</age>)

You can usechildto iterate over all the children:

scala> for (n <- p.child) println(n)
<name>Ken</name>
<age>23</age>

Becausechildreturns a sequence,you can also access the child elements like this:

scala> p.child(0)
res1: scala.xml.Node = <name>Ken</name>

scala> p.child(0).label
res2: String = name

scala> p.child(0).text
res3: String = Ken

scala> p.child(1)
res4: scala.xml.Node = <age>23</age>

scala> p.child(1).text.toInt
res5: Int = 23

Text and strings

toStringmethod returns the XML structure as aString:

scala> p.toString
res6: String = <person><name>Ken</name><age>23</age></person>

You can improve this result withthePrettyPrinterclass.

This approach shows another way to extract the text from the elements:

scala> for (n <- p.child) yield n.text
res7: Seq[String] = ArrayBuffer(Ken,23)

There are more ways to tackle these problems using XPath methods,which will be shown in subsequent chapters.

As a word of caution,be careful with thetextmethod. It returns different results depending on how the XML is formatted,which can be a particular problem when extracting XHTML data. To demonstrate this,the following examples show the output when there is a space before the<br>tag,and when there is no space:

scala> <div><p>Hello,world,<br/>it's me.</p></div>.text
res0: String = Hello,it's me.

scala> <div><p>Hello,<br/>it's me.</p></div>.text
res1: String = Hello,it's me.

In the next examples the same XML,formatted in different ways,yields different results:

scala> <div><p>Is 2 > 1?</p><p>Why do you ask?</p></div>.text
res2: String = Is 2 > 1?Why do you ask?

scala> <div>
     | <p>Is 2 > 1?</p>
     | <p>Why do you ask?</p>
     | </div>.text
res3: String = 
"
Is 2 > 1?
Why do you ask?
"

If you need to extract text in this manner,a workaround is to extract the text components individually into a sequence,and then re-combine the sequence as desired. The following example demonstrates how to accomplish this with thechild,label,andtextmethods. Given this XML literal:

val xml = <div><p>Is 2 > 1?</p><p>Why do you ask?</p></div>

thechildmethod returns the elements as a sequence:

scala> xml.child
res0: Seq[scala.xml.Node] = 
  ArrayBuffer(<p>Is 2 > 1?</p>,<p>Why do you ask?</p>)

This lets you write the following code,which creates a sequence of strings from the<p>tags:

val strings = for {
  e <- xml.child
  if e.label == "p"
} yield e.text

The REPL shows that the resulting variable strings has the following type and data:

strings: Seq[String] = ArrayBuffer(Is 2 > 1?,Why do you ask?)

In the XPath recipes in this chapter you’ll see how to accomplish some of the same tasks using themethods.

scala> val xml = scala.xml.XML.loadFile("nasa.xml")
java.lang.OutOfMemoryError: Java heap space ...

$ scala -J-Xms256m -J-Xmx512m

$ env JAVA_OPTS="-Xms256m -Xmx512m" scala