'Spark-xml: cannot read value of an element with attribute

I am trying to use Spark-xml to read the xml file in the link https://www.dropbox.com/s/yg66o0tfwipx3mu/PMC1249490.xml?dl=0

It is a research article and I am interested in the text in the abstract. It seems that the schema of the entire xml file is inferred correctly, but the abstract element is missing the text data. It shows the attribute value (called P1) and only the words enclosed in the brackets. Can anyone help me?

Below is the code I am using:

import pandas as pd
from pyspark.sql import SparkSession
import os
os.environ['PYSPARK_SUBMIT_ARGS'] = '--packages com.databricks:spark-xml_2.12:0.13.0 pyspark-shell'
spark = SparkSession.builder.appName("XML_Import").master("local[*]").getOrCreate()
df = spark.read.format('xml').options(rowTag="front").load('PMC1249490.xml')
df.select("article-meta.abstract").show(truncate=False)

+-------------------------------------------------+
|abstract                                         |
+-------------------------------------------------+
|{{P1, [Dictyostelium discoideum, D. discoideum]}}|
+-------------------------------------------------+
apache-spark-xml


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions