I am working on a spark application . I am using a third party jar. I added it in my sbt file. The compilation was successful
> libraryDependencies += "org.springframework.security" %
> "spring-security-web" % "3.0.7.RELEASE
But when i submit spark job
spark-submit --master yarn-client --class MyClass target/scala-2.10/my-app.jar
It fails with following error:
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to
stage failure: Task 0 in stage 48.0 failed 1 times, most recent failure: Lost task 0.0 in stage 48.0 (TID 3216, localhost):
I also tried explicitly passing jar in spark-submit jar
spark-submit --master yarn-client --jars third-party-jars/spring-security-web-3.0.7.RELEASE.jar --class MyClass target/scala-2.10/my-app.jar
But then it fails with following error:
Its similar error but on another class.
Any suggestion on how to resolve this?