scala – SBT Assembly – 重复数据删除错误和排除错误
嘿伙计们我正在尝试使用sbt-assembly构建一个带依赖关系的JAR.但我一次又一次地遇到这个错误.我尝试过多种不同的东西,但我最终还是来了.我是SBT的新手,希望得到一些帮助.这是build.sbt& assembly.sbt文件.
build.sbt seq(assemblySettings: _*) name := "StreamTest" version := "1.0" scalaVersion := "2.10.4" libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0" libraryDependencies += "org.apache.spark" % "spark-streaming-kinesis-asl_2.10" % "1.1.0" 项目/ assembly.sbt addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2") 当我运行sbt assembly命令时,我收到以下错误. [info] Including: joda-time-2.5.jar [info] Checking every *.class/*.jar file's SHA-1. [info] Merging files... [warn] Merging 'about_files/LICENSE.txt' with strategy 'rename' [warn] Merging 'about_files/NOTICE.txt' with strategy 'rename' [warn] Merging 'META-INF/NOTICE.txt' with strategy 'rename' [warn] Merging 'META-INF/NOTICE' with strategy 'rename' [warn] Merging 'org/xerial/snappy/native/README' with strategy 'rename' [warn] Merging 'META-INF/maven/org.xerial.snappy/snappy-java/LICENSE' with strategy 'rename' [warn] Merging 'META-INF/license' with strategy 'rename' [warn] Merging 'about.html' with strategy 'rename' [warn] Merging 'META-INF/LICENSE.txt' with strategy 'rename' [warn] Merging 'META-INF/README.txt' with strategy 'rename' [warn] Merging 'LICENSE.txt' with strategy 'rename' [warn] Merging 'META-INF/LICENSE' with strategy 'rename' [warn] Merging 'META-INF/DEPENDENCIES' with strategy 'discard' java.lang.RuntimeException: deduplicate: different file contents found in the following: /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA at sbtassembly.Plugin$Assembly$.sbtassembly$Plugin$Assembly$$applyStrategy$1(Plugin.scala:253) at sbtassembly.Plugin$Assembly$$anonfun$15.apply(Plugin.scala:270) at sbtassembly.Plugin$Assembly$$anonfun$15.apply(Plugin.scala:267) at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251) at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251) at scala.collection.Iterator$class.foreach(Iterator.scala:727) at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251) at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105) at sbtassembly.Plugin$Assembly$.applyStrategies(Plugin.scala:272) at sbtassembly.Plugin$Assembly$.x$4$lzycompute$1(Plugin.scala:172) at sbtassembly.Plugin$Assembly$.x$4$1(Plugin.scala:170) at sbtassembly.Plugin$Assembly$.stratMapping$lzycompute$1(Plugin.scala:170) at sbtassembly.Plugin$Assembly$.stratMapping$1(Plugin.scala:170) at sbtassembly.Plugin$Assembly$.inputs$lzycompute$1(Plugin.scala:214) at sbtassembly.Plugin$Assembly$.inputs$1(Plugin.scala:204) at sbtassembly.Plugin$Assembly$.apply(Plugin.scala:230) at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask$1.apply(Plugin.scala:373) at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask$1.apply(Plugin.scala:370) at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47) at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40) at sbt.std.Transform$$anon$4.work(System.scala:63) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226) at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17) at sbt.Execute.work(Execute.scala:235) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226) at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159) at sbt.CompletionService$$anon$2.call(CompletionService.scala:28) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) [error] (*:assembly) deduplicate: different file contents found in the following: [error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA [error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA [error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA [error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA [error] Total time: 23 s,completed Nov 28,2014 9:32:53 PM SBT-版本 0.13.6 编辑 现在,在环顾四周后,我做了另一个更改以排除任何依赖关系,作为stackoverflow上另一个问题的一部分. 更新了build.sbt seq(assemblySettings: _*) name := "StreamTest" version := "1.0" scalaVersion := "2.10.4" libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0" libraryDependencies += "org.apache.spark" % "spark-streaming-kinesis-asl_2.10" % "1.1.0" libraryDependencies ++= Seq( exclude("org.eclipse.jetty.orbit","javax.servlet"). exclude("org.eclipse.jetty.orbit","javax.transaction"). exclude("org.eclipse.jetty.orbit","javax.mail"). exclude("org.eclipse.jetty.orbit","javax.activation"). exclude("commons-beanutils","commons-beanutils-core"). exclude("commons-collections","commons-collections"). exclude("commons-collections","commons-collections"). exclude("com.esotericsoftware.minlog","minlog") ) 当我再次运行汇编命令时,这是我得到的错误. build.sbt:14: error: not found: value exclude exclude("org.eclipse.jetty.orbit","javax.servlet"). ^ 编辑2: 更新了build.sbt import AssemblyKeys._ seq(assemblySettings: _*) name := "SparkStreamingKinesis" version := "1.0" scalaVersion := "2.10.4" libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0" libraryDependencies += "org.apache.spark" % "spark-streaming-kinesis-asl_2.10" % "1.1.0" assemblyMergeStrategy in assembly := { case PathList(ps @ _*) if ps.last endsWith ".RSA" => MergeStrategy.first case x => val oldStrategy = (assemblyMergeStrategy in assembly).value oldStrategy(x) } 进行更新后,这是我得到的错误.我尝试导入assemblyMergeStrategy,但它看起来不像我可以导入的类. build.sbt:21: error: not found: value assemblyMergeStrategy assemblyMergeStrategy in assembly := { ^ [error] Type error in expression 解决方法
试试这个只排除ECLIPSEF.RSA文件,因为排除你的库可能需要的整个依赖项,是个坏主意:)
Deduplication error in build.sbt when building a fat jar using sbt assembly 最新0.12.0版程序集插件的解决方案: assemblyMergeStrategy in assembly := { case PathList(ps @ _*) if ps.last endsWith ".RSA" => MergeStrategy.first case x => val oldStrategy = (assemblyMergeStrategy in assembly).value oldStrategy(x) } 项目/ assembly.sbt addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0") 见https://github.com/sbt/sbt-assembly#merge-strategy 附:关于exclude:你不能只在libraryDependencies中指定exclude – 你需要这样的东西:“log4j”%“log4j”%“1.2.15”exclude(“javax.jms”,“jms”).见Library Management (编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |