Rust: XML Parsing with Quick-xml

There are many cases where we might want to parse an XML. It might be to ensure the XML document is indeed properly formatted. It might be for we scrapping.

But anyway!

In this article, let’s check out how we can use the quick_xml crate to parse XML files.

It provides xml reader that is

• almost zero-copy
• easy on memory allocation
• support various encoding (with encoding feature), namespaces resolution, special characters.

Start!

Create Reader

The XML Reader is the entry point for us to consume bytes and stream XML Events of an XML document.

There are 3 different way of creating a Reader

• from_file to create a reader from a file path
• from_reader to create a reader from a given reader. This can be another XML Reader, a BufReader from the std library, and etc.
• from_str to create a reader from a string slice

// from path
let path = "examples/sample.xml";
let xml_reader = Reader::from_file(path)?;

// from another reader
let buf_reader = BufReader::new(File::open(path)?);
let xml_reader = Reader::from_reader(buf_reader);

// from string
let mut contents = String::new();
File::open(path)?.read_to_string(&mut contents)?;
let xml_reader = Reader::from_str(&contents);

Just so that we can be on the same page, let’s use the following simple xml string for the rest of this article.

let xml_string = r#"
    <?xml version="1.0"?>
    <customers>
        <customer id="1">
            <name>Itsuki 1</name>
            <address>
                <state>MA</state>
                <zip>01701</zip>
            </address>
            <address>
                <state>MA</state>
                <zip>01705</zip>
            </address>
        </customer>
        <customer id="2">
            <name>Itsuki 2</name>
            <address>
                <state>MA</state>
                <zip>01702</zip>
            </address>
        </customer>
    </customers>
"#;

let mut reader = Reader::from_str(xml_string);

Modify Configuration

To configure the parser used, we can modify the Config object returned by a call to Reader::config_mut().

Here are the properties we can change.

pub struct Config {
    pub allow_unmatched_ends: bool,
    pub check_comments: bool,
    pub check_end_names: bool,
    pub expand_empty_elements: bool,
    pub trim_markup_names_in_closing_tags: bool,
    pub trim_text_start: bool,
    pub trim_text_end: bool,
}

For example, we can disable the detecting of mismatched closing tag names, disable text trimming (both start and end), and expand empty elements into an Open and a Close event like the following.

let mut reader = Reader::from_str(xml_string);
let config = reader.config_mut();
config.check_end_names = false; // default true
config.trim_text(false); // default false
config.expand_empty_elements = true; // default false

Note that the trimming text options should be set with caution. If we enable these options globally through the Config object, all text events will be trimmed which is the unwanted behavior when text events delimited by comments, processing instructions or CDATA sections.

To correctly trim data, it is better to manually applyBytesText::inplace_trim_start and BytesText::inplace_trim_end only to necessary events.

Events are what represents XML elements and we will be taking a look at those next!

Read XML

We have our reader, we have configured our parser, it is time to read our XML document!

quick_xml uses the Event structure to represent the XML element.

pub enum Event<'a> {
    Start(BytesStart<'a>),
    End(BytesEnd<'a>),
    Empty(BytesStart<'a>),
    Text(BytesText<'a>),
    CData(BytesCData<'a>),
    Comment(BytesText<'a>),
    Decl(BytesDecl<'a>),
    PI(BytesPI<'a>),
    DocType(BytesText<'a>),
    Eof,
}

For example, given the following XML, we will have a Start, a Text, an End and an Eof.

<name attr="value">Inner text</name>

As I have mentioned in the very beginning, the reader we have is a pull reader. By that means, to consume the events, we will be looping and calling read_event_into to read the next Event until Eof is reached.

Get Itsuki’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Here is how we can read through the entire XML document as well as getting the name and attributes for the tag.

// read data
let mut buf = Vec::new();

loop {
    match reader.read_event_into(&mut buf) {
        Err(e) => bail!("Error at position {}: {:?}", reader.error_position(), e),
        
        // end of file
        Ok(Event::Eof) => break,

        Ok(Event::Start(e)) => {
            let name = String::from_utf8(e.name().as_ref().to_vec())?;
            let attributes = e.attributes();
            println!("Start tag: {}", name);
            for a in attributes {
                match a {
                    Ok(a) => println!(
                        "attributes key {}; value: {}",
                        String::from_utf8(a.key.as_ref().to_vec())?,
                        String::from_utf8(a.value.to_vec())?
                    ),
                    Err(err) => println!("error getting attribute: {}", err.to_string()),
                }
            }
        }

        Ok(Event::Text(e)) => {
            println!("Text: {}", e.unescape()?);
        }

        // Not considering events such as Empty, Comment, or CData
        _ => (),
    }

    // clear the buffer to keep memory usage low
    buf.clear();
}
Ok(())

And here is the output.

Start tag: customers
Start tag: customer
attributes key id; value: 1
Start tag: name
Text: Itsuki 1
Start tag: address
Start tag: state
Text: MA
Start tag: zip
Text: 01701
Start tag: address
Start tag: state
Text: MA
Start tag: zip
Text: 01705
Start tag: customer
attributes key id; value: 2
Start tag: name
Text: Itsuki 2
Start tag: address
Start tag: state
Text: MA
Start tag: zip
Text: 01702

Couple notes here for tag name and attributes.

I have used the name function to get the tag name. This will give me the un-decoded raw tag name. If you want the local tag name excluding namespace, ie: all content up to and including the first : character is removed from the tag name, you can use local_name instead.

For getting attributes, there is also the

• attributes_raw to get the un-decoded raw string with the attributes of this tag as a &[u8], including the whitespace after the tag name if there is any.
• html_attributes for getting HTML-like attributes where quotes or = character are not required.
• try_get_attribute to try to get a specific attribute

We have used read_event_into in our example above. This is the generic case when we don’t know about the input BufRead. However, in the case where the input is string, ie: &str, or bytes, ie: &[u8], we can also use read_event instead to read an event that borrows from the input rather than a buffer.

loop {
    match reader.read_event() {
        // everything remains the same ...
    }
}

Processing Specific Tag

In our example above, we have looped through and printed out all tags.

If you have a tag of interest, we can simply read to end of the tag for the entire contents after we found the start.

To do so, we can use the read_to_end function that returns a Span, or read_text which is basically a wrapper around read_to_end to covert the Span to String.

(You can also use the same method to skip a specific tag that you are NOT interested in!)

For example, here is what we can do if we want to get all content within the customer tag at once.

let mut buf = Vec::new();

loop {
    match reader.read_event_into(&mut buf) {
        Err(e) => bail!("Error at position {}: {:?}", reader.error_position(), e),

        // end of file
        Ok(Event::Eof) => break,

        Ok(Event::Start(e)) => {
            let name = String::from_utf8(e.name().as_ref().to_vec())?;
            println!("Start tag: {}", name);
            if e.name().as_ref() == b"customer" {
                let text = reader.read_text(e.to_end().into_owned().name())?;
                println!("tag found with content: {}", text);
            }
        }
        // Not considering events such as Empty, Comment, or CData
        _ => (),
    }

    buf.clear();
}

Here is what we will get.

Start tag: customers
Start tag: customer
tag found with content: 
                <name>Itsuki 1</name>
                <address>
                    <state>MA</state>
                    <zip>01701</zip>
                </address>
                <address>
                    <state>MA</state>
                    <zip>01705</zip>
                </address>
            
Start tag: customer
tag found with content: 
                <name>Itsuki 2</name>
                <address>
                    <state>MA</state>
                    <zip>01702</zip>
                </address>

Note!

The end parameter should contain name of the end element in the reader encoding. It is good practice to always get that parameter using BytesStart::to_end() method.

Also, both functions, read_to_end and read_text, will automatically manage nested cases where parent and child elements have the literally same name for us.

What if we only want to process the interested tags but still want get every single tag within the start and end as nested tags instead of the entire contents string like above.

It is also possible to nest another read_event_into within the Event::Start when we found the tag we are interested in.

Here is a simplified version assuming that you do NOT have parent and child elements with the same name.

let mut buf = Vec::new();
    let mut inner_buf = Vec::new();
    let tag_name = b"customer";

    loop {
        match reader.read_event_into(&mut buf) {
            Err(e) => bail!("Error at position {}: {:?}", reader.error_position(), e),

            // end of file
            Ok(Event::Eof) => break,

            Ok(Event::Start(e)) => {
                let name = String::from_utf8(e.name().as_ref().to_vec())?;
                println!("Start tag outter: {}", name);
                if e.name().as_ref() == tag_name {
                    loop {
                        match reader.read_event_into(&mut inner_buf) {
                            Err(e) => {
                                bail!("Error at position {}: {:?}", reader.error_position(), e)
                            }
                            Ok(Event::Eof) => bail!("Unexpected end of xml"),

                            Ok(Event::Start(e)) => {
                                let name = String::from_utf8(e.name().as_ref().to_vec())?;
                                println!("start tag inner: {}", name);
                                let attributes = e.attributes();
                                for a in attributes {
                                    match a {
                                        Ok(a) => println!(
                                            "attributes key {}; value: {}",
                                            String::from_utf8(a.key.as_ref().to_vec())?,
                                            String::from_utf8(a.value.to_vec())?
                                        ),
                                        Err(err) => {
                                            println!("error getting attribute: {}", err.to_string())
                                        }
                                    }
                                }
                            }

                            Ok(Event::End(ref e)) if e.name().as_ref() == tag_name => break,

                            Ok(Event::Text(e)) => {
                                println!("Text: {}", e.unescape()?);
                            }

                            _ => (),
                        };
                        inner_buf.clear();
                    }
                }
            }
            // Not considering events such as Empty, Comment, or CData
            _ => (),
        }

        buf.clear();
    }

Note that in addition to the Events we had above, I have also added the End here so that we can break out of the inner loop when we reach the end of the tag that we are interested in.

And here is the output.

Start tag outter: customers
Start tag outter: customer
start tag inner: name
Text: Itsuki 1
start tag inner: address
start tag inner: state
Text: MA
start tag inner: zip
Text: 01701
start tag inner: address
start tag inner: state
Text: MA
start tag inner: zip
Text: 01705
Start tag outter: customer
start tag inner: name
Text: Itsuki 2
start tag inner: address
start tag inner: state
Text: MA
start tag inner: zip
Text: 01702

(Of course, you can achieve similar things without nesting loops but set a flag at Start and End event, and only process Text if the flag is true.)

Thank you for reading!

That’s it for this article!

You can also use quick_xml for writing as well! Give it a look yourself if you are interested!

Happy parsing!