'Microsoft Spark - JVM method execution failed: Nonstatic method 'csv' failed for class
I try to save the result as one "csv" file on Windows Server 2019. I'm using the "Microsoft.Spark" library. An empty folder is created with no "csv" file. The query itself works.
SparkSession spark =
SparkSession
.Builder()
.Master("local[*]")
.AppName("sparkToMongo")
.Config("spark.mongodb.input.uri", authURI)
.GetOrCreate();
DataFrame df = spark.Read().Format("com.mongodb.spark.sql.DefaultSource").Load();
df.CreateOrReplaceTempView("table");
var beers = spark.Sql(sql).ToDF();
beers.Coalesce(1).Write().Option("mode", "append").Option("header", "true").Csv("result");
spark.Stop();
I get the following error:
[2021-06-10T14:17:04.3113758Z] [cdp-vm] [Exception] [JvmBridge] JVM method execution failed: Nonstatic method 'csv' failed for class '15' when called with 1 arguments ([Index=1, Type=String, Value=result], )
at Microsoft.Spark.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] args)
Unhandled exception. System.Exception: JVM method execution failed: Nonstatic method 'csv' failed for class '15' when called with 1 arguments ([Index=1, Type=String, Value=result], )
---> Microsoft.Spark.JvmException: org.apache.spark.SparkException: Job aborted.
at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:231)
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:178)
at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:108)
at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:106)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|