scala – Spark驱动程序与主服务器解除关联并删除
发布时间:2020-12-16 08:56:00 所属栏目:安全 来源:网络整理
导读:我有一个由两个奴隶和一个主人制作的集群并设置我向火花大师(192.168.1.64)提交一个jar( scala): spark-submit --master spark://spark-master:7077 --class tests.elements target/scala-2.10/zzz-project_2.10-1.0.jar 经过一段时间运行得很好,它终止了终
我有一个由两个奴隶和一个主人制作的集群并设置我向火花大师(192.168.1.64)提交一个jar(
scala):
spark-submit --master spark://spark-master:7077 --class tests.elements target/scala-2.10/zzz-project_2.10-1.0.jar 经过一段时间运行得很好,它终止了终端上的最后几行 ... 15/08/19 17:45:24 INFO scheduler.TaskSchedulerImpl: Adding task set 411292.0 with 6 tasks 15/08/19 17:45:24 WARN scheduler.TaskSetManager: Stage 411292 contains a task of very large size (2762 KB). The maximum recommended task size is 100 KB. 15/08/19 17:45:24 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 411292.0 (TID 1832,192.168.1.64,PROCESS_LOCAL,2828792 bytes) 15/08/19 17:45:24 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 411292.0 (TID 1833,192.168.1.62,2310009 bytes) 15/08/19 17:45:24 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 411292.0 (TID 1834,2669188 bytes) 15/08/19 17:45:24 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 411292.0 (TID 1835,2295676 bytes) 15/08/19 17:45:24 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 411292.0 (TID 1836,2847786 bytes) 15/08/19 17:45:24 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 411292.0 (TID 1837,2913528 bytes) Killed 并且主日志中发生的错误如下: ... 15/08/19 16:09:49 INFO master.Master: Launching executor app-20150819160949-0001/0 on worker worker-20150819160925-192.168.1.64-51640 15/08/19 16:09:49 INFO master.Master: Launching executor app-20150819160949-0001/1 on worker worker-20150819160938-192.168.1.62-38007 15/08/19 16:15:44 INFO master.Master: akka.tcp://sparkDriver@192.168.1.64:46823 got disassociated,removing it. 15/08/19 16:15:44 INFO master.Master: Removing app app-20150819160949-0001 15/08/19 16:15:44 WARN remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkDriver@192.168.1.64:46823] has failed,address is now gated for [5000] ms. Reason is: [Disassociated]. 15/08/19 16:15:44 WARN master.Master: Application testPageRank is still in progress,it may be terminated abnormally. ... 两个工人在他们的日志中都有这样的东西 ... 15/08/19 16:15:49 INFO worker.Worker: Executor app-20150819160949-0001/0 finished with state EXITED message Command exited with code 1 exitStatus 1 15/08/19 16:15:50 WARN remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkExecutor@192.168.1.64:54799] has failed,address is now gated for [5000] ms. Reason is: [Disassociated]. 和 ... 15/08/19 16:15:43 INFO worker.Worker: Executor app-20150819160949-0001/1 finished with state EXITED message Command exited with code 1 exitStatus 1 15/08/19 16:15:43 WARN remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkExecutor@192.168.1.62:53325] has failed,address is now gated for [5000] ms. Reason is: [Disassociated]. 分别.工作/应用程序文件包含这样的内容 ... 15/08/19 16:15:41 INFO executor.Executor: Finished task 1.0 in stage 387758.0 (TID 1803). 1911 bytes result sent to driver 15/08/19 16:15:41 INFO executor.Executor: Finished task 4.0 in stage 387758.0 (TID 1806). 1911 bytes result sent to driver 15/08/19 16:15:41 INFO storage.BlockManager: Found block rdd_1206_5 locally 15/08/19 16:15:41 INFO executor.Executor: Finished task 5.0 in stage 387758.0 (TID 1807). 1911 bytes result sent to driver 15/08/19 16:15:41 INFO storage.BlockManager: Found block rdd_1206_3 locally 15/08/19 16:15:41 INFO executor.Executor: Finished task 3.0 in stage 387758.0 (TID 1805). 1911 bytes result sent to driver 15/08/19 16:15:44 ERROR executor.CoarseGrainedExecutorBackend: Driver 192.168.1.64:46823 disassociated! Shutting down. 15/08/19 16:15:44 WARN remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkDriver@192.168.1.64:46823] has failed,address is now gated for [5000] ms. Reason is: [Disassociated]. 15/08/19 16:15:45 INFO storage.DiskBlockManager: Shutdown hook called 15/08/19 16:15:46 INFO util.Utils: Shutdown hook called 和 ... 15/08/19 16:15:41 INFO storage.BlockManager: Found block rdd_1206_0 locally 15/08/19 16:15:41 INFO executor.Executor: Finished task 2.0 in stage 387758.0 (TID 1804). 1911 bytes result sent to driver 15/08/19 16:15:41 INFO executor.Executor: Finished task 0.0 in stage 387758.0 (TID 1802). 1911 bytes result sent to driver 15/08/19 16:15:42 ERROR executor.CoarseGrainedExecutorBackend: Driver 192.168.1.64:46823 disassociated! Shutting down. 15/08/19 16:15:42 INFO storage.DiskBlockManager: Shutdown hook called 15/08/19 16:15:42 WARN remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkDriver@192.168.1.64:46823] has failed,address is now gated for [5000] ms. Reason is: [Disassociated]. 15/08/19 16:15:42 INFO util.Utils: Shutdown hook called 分别. hdfs或spark似乎没有其他错误. 我怀疑错误在于主日志,第三行(15/08/19 16:15:44 INFO master.Master:akka.tcp://sparkDriver@192.168.1.64:46823已取消关联,删除它. )但我无法弄清楚为什么.我尝试将spark.akka.heartbeat.interval更改为100,如某些帖子所示,但没有运气.任何人都知道它为什么会发生以及如何解决这个问题?非常感谢. 解决方法
正如
WARN ReliableDeliverySupervisor: Association with remote system has failed,address is now gated for [5000] ms. Reason: [Disassociated]中一个非常相似的问题所述
问题可能是缺乏记忆.添加更多内存(或者在这种情况下,更多节点)应该可以解决问题. (或者,需要更少的内存当然也应该工作). (编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |