'How to convert schema from text file to scala dataframe
We have text file having below data :
String Col1
String Col2
List <Object> Col3 [
String Col3_1
String Col3_2
]
String Col4
String Col5
We need to convert this to Scala Schema as :
val Schema = new StructType()
.add("Col1",StringType)
.add("Col2",StringType)
.add("Col3",StructType)
.add("Col4",StringType)
.add("Col5",StringType)
The data would be nested json and schema will keep evolving, Hence an automated solution is required.
Option of having schemaRegistry is good but its not in the option for current implementation.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|