'Pyspark create schema for maptype with different value types

I need to give the correct schema to an rdd I have, but struggling with a maptype that has different valuetypes. I guess the problem is that one specific Key has a string as it's value and the others have again a dictionary as value:

{row_id -> "0", Kind -> {right -> "32", left -> "8", top -> "62", bottom -> "61"}, Tab -> {right -> "32", left -> "8", top -> "62", bottom -> "61"},...},{row_id -> "1", Kind -> {right ->...}...},..

When trying to define the schema like MapType(StringType(),MapType(StringType(),StringType())) I get a TypeError. Propably because value of row_id does not hold a dictionary.

Any ideas how to correctly define the schema?

Best!



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source