scala – 找不到SparkContext类错误
发布时间:2020-12-16 10:01:14 所属栏目:安全 来源:网络整理
导读:我试图使用intellij在spark上运行 scala代码. Scala代码 import scala.collection.JavaConverters._import org.apache.spark.SparkContextimport org.apache.spark.SparkConfobject WordCount { def main(args: Array[String]): Unit = { val sc = new Spark
我试图使用intellij在spark上运行
scala代码.
Scala代码 import scala.collection.JavaConverters._ import org.apache.spark.SparkContext import org.apache.spark.SparkConf object WordCount { def main(args: Array[String]): Unit = { val sc = new SparkContext(new SparkConf().setAppName("anything").setMaster("localhost")) println("Hello World!") } } POM <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.vocp</groupId> <artifactId>SparkScalaConnect</artifactId> <version>1.0-SNAPSHOT</version> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <pluginRepositories> <pluginRepository> <id>scala-tools.org</id> <name>Scala-tools Maven2 Repository</name> <url>http://scala-tools.org/repo-releases</url> </pluginRepository> </pluginRepositories> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>3.8.1</version> <scope>test</scope> </dependency> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core_2.10</artifactId> <version>1.2.0</version> <scope>provided</scope> </dependency> <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-library</artifactId> <version>2.10.5</version> </dependency> <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-compiler</artifactId> <version>2.7.5</version> <scope>compile</scope> </dependency> </dependencies> 错误信息 Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/SparkContext at WordCount$.main(WordCount.scala:9) at WordCount.main(WordCount.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkContext at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) 我试图在Scala-IDE上运行相同的代码,它运行FINE.我看到问题可能在intellij编辑器中. 长期坚持这个问题.任何建议都是有益的. 提前致谢. 注意 – 我在1.7 JAVA Oracle和2.10.5 scala SDK上运行代码. 解决方法
尝试从提供到编译的spark依赖关系的这个更改范围
issue (编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |
相关内容
- WebService 之 WSDL文件 讲解
- Bootstrap中tab标签切换demo
- node.js – npm package.json和docker(挂载……)
- yum提示Another app is currently holding the yum lock; w
- angularjs – 特定元素中的角度材料吐司
- Scala使用Java库,利用Java 8中的lambda表达式支持
- twitter-bootstrap – 使用媒体查询隐藏Bootstrap在移动设备
- shell – 捕获由Makefile启动的后台进程的PID
- 如何实现类似bash的标签完成?
- MySQL Waiting for table metadata lock故障分析