scala – 使用案例类编码JSON时,为什么错误“无法找到存储在数据
发布时间:2020-12-16 09:25:01 所属栏目:安全 来源:网络整理
导读:我写过火花工作: object SimpleApp { def main(args: Array[String]) { val conf = new SparkConf().setAppName("Simple Application").setMaster("local") val sc = new SparkContext(conf) val ctx = new org.apache.spark.sql.SQLContext(sc) import ctx
我写过火花工作:
object SimpleApp { def main(args: Array[String]) { val conf = new SparkConf().setAppName("Simple Application").setMaster("local") val sc = new SparkContext(conf) val ctx = new org.apache.spark.sql.SQLContext(sc) import ctx.implicits._ case class Person(age: Long,city: String,id: String,lname: String,name: String,sex: String) case class Person2(name: String,age: Long,city: String) val persons = ctx.read.json("/tmp/persons.json").as[Person] persons.printSchema() } } 在IDE中运行main函数时,发生2错误: Error:(15,67) Unable to find encoder for type stored in a Dataset. Primitive types (Int,String,etc) and Product types (case classes) are supported by importing sqlContext.implicits._ Support for serializing other types will be added in future releases. val persons = ctx.read.json("/tmp/persons.json").as[Person] ^ Error:(15,67) not enough arguments for method as: (implicit evidence$1: org.apache.spark.sql.Encoder[Person])org.apache.spark.sql.Dataset[Person]. Unspecified value parameter evidence$1. val persons = ctx.read.json("/tmp/persons.json").as[Person] ^ 但是在Spark Shell中我可以毫无错误地运行这个作业.问题是什么? 解决方法
错误消息表明Encoder无法接受Person案例类.
Error:(15,etc) and Product types (case classes) are supported by importing sqlContext.implicits._ Support for serializing other types will be added in future releases. 将case类的声明移到SimpleApp的范围之外. (编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |