'How to use java runtime 11 in EMR cluster AWS
I'm creating a cluter in EMR aws and when spark runs my application I'm getting error below:
Exception in thread "main" java.lang.UnsupportedClassVersionError:
com/example/demodriver/MyClassFromJAR has been compiled by a more recent version of the Java Runtime (class file version 55.0),
this version of the Java Runtime only recognizes class file versions up to 52.0
I'm using releaseLabel emr-6.5.0
on clusters and My driver jar is built in java11
How run java11 application in EMR? Or this error is about anything else?
Solution 1:[1]
In the most recent versions of EMR, java 11 is installed. To enable it, you can provide the following configuration.
[
{
"Classification": "spark-env",
"Configurations": [
{
"Classification": "export",
"Properties": {
"JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
}
}
]
},
{
"Classification": "spark-defaults",
"Properties": {
"spark.executorEnv.JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
}
}
]
This does not seem to be documented.
The defaultJavaOptions
and extraJavaOptions
might contain incompatible options for java 11 which you still might need to adapt/update.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 |