scala – java.lang.RuntimeException：java.lang.String不是big

id,1,bigint
price,2,bigint
sqft,3,bigint
zip_id,4,int
name,5,string

我将解析后的数据类型映射到Spark Sql数据类型.创建数据框的代码是 –

var schemaSt = new ListBuffer[(String,String)]()
// read schema from file
for (line <- Source.fromFile("meta.txt").getLines()) {
  val word = line.split(",")
  schemaSt += ((word(0),word(2)))
}

// map datatypes
val types = Map("int" -> IntegerType,"bigint" -> LongType)
      .withDefault(_ => StringType)
val schemaChanged = schemaSt.map(x => (x._1,types(x._2))

// read data source
val lines = spark.sparkContext.textFile("data source path")

val fields = schemaChanged.map(x => StructField(x._1,x._2,nullable = true)).toList

val schema = StructType(fields)

val rowRDD = lines
  .map(_.split("t"))
  .map(attributes => Row.fromSeq(attributes))

// Apply the schema to the RDD
val new_df = spark.createDataFrame(rowRDD,schema)
new_df.show(5)
new_df.printSchema()

但以上只适用于StringType.对于IntegerType和LongType,它抛出异常 –

java.lang.RuntimeException: java.lang.String is not a valid external type for schema of int

和

java.lang.RuntimeException: java.lang.String is not a valid external type for schema of bigint.

提前致谢！