Livy and Elasticsearch-Spark: Multiple ES-Hadoop versions detected

Question

I'm trying to read from elasticsearch in a livy job using the elastisearch-spark jar. When I upload the jar to a livy client(like the example here) I get this error and I'm not sure how to parse it.

Caused by: java.lang.RuntimeException: java.lang.Error: Multiple ES-Hadoop 
versions detected in the classpath; please use only one
jar:file:/tmp/tmp7d6epaqu/__livy__/elasticsearch-spark-20_2.11-6.2.2.jar
jar:file:/tmp/rsc-tmp3492512103399411501/__livy__/elasticsearch-spark-20_2.11-6.2.2.jar

I'm not sure what the temp directories are or why it's recognizing 2 jars when I'm only importing the one(If I remove the dependency from my pom it complains about Javasespark not existing). What am I doing wrong and what do I need to do to fix this?