Scala对Spark安装的依赖性

项目配置放在项目/文件夹中.我的情况是：

$ls project/
build.properties  plugins.sbt  project  SparkBuild.scala  target

当您执行sbt / sbt程序集时,它会下载适当版本的Scala以及其他项目依赖项.签出文件夹目标/例如：

$ls target/
scala-2.9.2  streams

请注意,Scala版本对我来说是2.9.2.

为了在其他模式(分布式)中运行Spark,我是否需要在所有节点上安装Scala？

是.您可以按照Spark documentation中的说明创建单个装配罐

If your code depends on other projects,you will need to ensure they are also present on the slave nodes. A popular approach is to create an assembly jar (or “uber” jar) containing your code and its dependencies. Both sbt and Maven have assembly plugins. When creating assembly jars,list Spark itself as a provided dependency; it need not be bundled since it is already present on the slaves. Once you have an assembled jar,add it to the SparkContext as shown here. It is also possible to submit your dependent jars one-by-one when creating a SparkContext.