Python 3.3: Convert XML to YAML

Question

I'm trying to convert XML files to YAML using Python 3.3. This is my code:

#! /etc/python3

test_filename_input = './reference-conversions/wikipedia-example.xml'
test_filename_output = 'wikipedia-example_xml_read-as-binary.yaml'

file_object = open( test_filename_input, 'rb')
data_in = file_object.read()
file_object.close()

from xml.dom.minidom import parseString
document_object = parseString( data_in)

import yaml
stream = open( test_filename_output, 'w')
yaml.dump( document_object, stream)
stream.close()

As a reference I used the XML-file from here:

<person>
  <firstName>John</firstName>
  <lastName>Smith</lastName>
  <age>25</age>
  <address>
    <streetAddress>21 2nd Street</streetAddress>
    <city>New York</city>
    <state>NY</state>
    <postalCode>10021</postalCode>
  </address>
  <phoneNumbers>
    <phoneNumber type="home">212 555-1234</phoneNumber>
    <phoneNumber type="fax">646 555-4567</phoneNumber>
  </phoneNumbers>
  <gender>
    <type>male</type>
  </gender>
</person>

which should result in something like this:

---
  firstName: John
  lastName: Smith
  age: 25
  address: 
        streetAddress: 21 2nd Street
        city: New York
        state: NY
        postalCode: 10021

  phoneNumber: 
        -  
            type: home
            number: 212 555-1234
        -  
            type: fax
            number: 646 555-4567
  gender: 
        type: male

However, the result is:

&id001 !!python/object/new:xml.dom.minidom.Document
state: !!python/tuple
- implementation: !!python/object:xml.dom.minidom.DOMImplementation {}
- _elem_info: {}
  _id_cache: {}
  _id_search_stack: null
  childNodes: !!python/object/new:xml.dom.minicompat.NodeList
    listitems:
    - &id039 !!python/object/new:xml.dom.minidom.Element
      state: !!python/tuple
      - null
      - _attrs: null
        _attrsNS: null
        childNodes: !!python/object/new:xml.dom.minicompat.NodeList
          listitems:
          - &id045 !!python/object/new:xml.dom.minidom.Text
            state: !!python/tuple
            - null
            - _data: "\n  "
              nextSibling: &id002 !!python/object/new:xml.dom.minidom.Element
                state: !!python/tuple
                - null
                - _attrs: null
                  _attrsNS: null
                  childNodes: !!python/object/new:xml.dom.minicompat.NodeList
                    listitems:
[...]

Any idea, how to get PyYAML filter out the object-stuff from xml.dom.minidom or any alternative to using xml.dom.minidom?

Thanks!

Answer 1

Here's an approach that addresses the problems with xml.dom and provides an approach to map the ambiguous situation where a node has both content and attributes or children. For the example input above it yields:

$ python3 yamlout.py person.xml
---
person:
  firstName: John
  lastName: Smith
  age: 25
  address:
    streetAddress: 21 2nd Street
    city: New York
    state: NY
    postalCode: 10021
  phoneNumbers:
    phoneNumber:
      _xml_node_content: 212 555-1234 
      type: home # Attribute
    phoneNumber:
      _xml_node_content: 646 555-4567 
      type: fax # Attribute
  gender:
    type: male

The implemention, yamlout.py:

import sys
import json
import xml.etree.ElementTree as ET

if len(sys.argv) != 2:
    sys.stderr.write("Usage: {0} <file>.xml".format(sys.argv[0]))

XML_NODE_CONTENT = '_xml_node_content'
ATTR_COMMENT = '# Attribute'
def yamlout(node, depth=0):
    if not depth:
        sys.stdout.write('---\n')
    # Nodes with both content AND nested nodes or attributes
    # have no valid yaml mapping. Add  'content' node for that case
    nodeattrs = node.attrib
    children = list(node)
    content = node.text.strip() if node.text else ''
    if content:
        if not (nodeattrs or children):
            # Write as just a name value, nothing else nested
            sys.stdout.write(
                '{indent}{tag}: {text}\n'.format(
                    indent=depth*'  ', tag=node.tag, text=content or ''))
            return
        else:
            # json.dumps for basic handling of multiline content
            nodeattrs[XML_NODE_CONTENT] = json.dumps(node.text)

    sys.stdout.write('{indent}{tag}:\n'.format(
        indent=depth*'  ', tag=node.tag))

    # Indicate difference node attributes and nested nodes
    depth += 1
    for n,v in nodeattrs.items():
        sys.stdout.write(
            '{indent}{n}: {v} {c}\n'.format(
                indent=depth*'  ', n=n, v=v,
                c=ATTR_COMMENT if n!=XML_NODE_CONTENT else ''))
    # Write nested nodes
    for child in children:
        yamlout(child, depth)

with open(sys.argv[1]) as xmlf:
    tree = ET.parse(xmlf)
    yamlout(tree.getroot())

Answer 2

I found an XML to YAML converter, but I had to make a minor change at about line 92:

outStr = yaml.dump(out)

changed to

outStr = yaml.safe_dump(out)

which removes any !!python/unicode tags in the output. I have tested the script via shell command line and it works fine; I'm sure it's just a simple translation to get it to work within the Python command line.

Edit

I also added my own printing method to make the output look more like what you posted originally:

def prettyPrint(node, level):
childPrint = 0
attrPrint = 0

for x in node:
    try:
        if x['attributes']:
            attrPrint = 1

            for l in range(0, level):
                sys.stdout.write("\t")

            for a in x['attributes']:
                sys.stdout.write("- %s: %s\n" % (a, x['attributes'][a]))

    except KeyError:
        try:
            if x['children']:
                childPrint = 1

                for l in range(0, level):
                    sys.stdout.write("\t")

                sys.stdout.write("%s:\n" % x['name'])
                prettyPrint(x['children'], level+1)

        except KeyError:
            pass

    finally:
        if not childPrint:
            printNextNode(x, level, attrPrint)
            attrPrint = 0

        else:
            childPrint = 0

def printNextNode(node, level, attrPrint):
    for l in range(0, level):
        sys.stdout.write("\t")

    if attrPrint:
        sys.stdout.write('  ')

    sys.stdout.write("%s: %s\n" % (node['name'], node['text']))

This is then called within the convertXml2Yaml function:

sys.stdout.write('%s:\n' % out['name'])
prettyPrint(out['children'], 1)