java – 使用带水印的附加输出模式时的结构化流异常

Exception in thread “main” org.apache.spark.sql.AnalysisException: Append output mode not supported when there are streaming aggregations on streaming DataFrames/DataSets without watermark;;

从我在programming guide中看到的内容,这与预期用法(和示例代码)完全匹配.有谁知道什么可能是错的？

提前致谢！

相关代码(Java 8,Spark 2.2.0)：

StructType logSchema = new StructType()
        .add("timestamp",TimestampType)
        .add("key",IntegerType)
        .add("val",IntegerType);

Dataset<Row> kafka = spark
        .readStream()
        .format("kafka")
        .option("kafka.bootstrap.servers",brokers)
        .option("subscribe",topics)
        .load();

Dataset<Row> parsed = kafka
        .select(from_json(col("value").cast("string"),logSchema).alias("parsed_value"))
        .select("parsed_value.*");

Dataset<Row> tenSecondCounts = parsed
        .withWatermark("timestamp","10 minutes")
        .groupBy(
            parsed.col("key"),window(parsed.col("timestamp"),"1 day"))
        .count();

StreamingQuery query = tenSecondCounts
        .writeStream()
        .trigger(Trigger.ProcessingTime("10 seconds"))
        .outputMode("append")
        .format("console")
        .option("truncate",false)
        .start();