欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

spark利用yarn提交任务报:YARN application has exited unexpectedly with state UNDEFINED

程序员文章站 2022-11-06 23:06:42
spark用yarn提交任务会报ERROR cluster.YarnClientSchedulerBackend: YARN application has exited unexpectedly with state UNDEFINED! Check the YARN application lo ......

spark用yarn提交任务会报
error cluster.yarnclientschedulerbackend: yarn application has exited unexpectedly with state undefined! check the yarn application logs for more details.
error cluster.yarnclientschedulerbackend: diagnostics message: shutdown hook called before final status was reported.

只需要将hadoop集群的yarn-site.xml配置中加入如下两行即可解决:

<property>
    <name>yarn.nodemanager.pmem-check-enabled</name>
    <value>false</value>
</property>
<property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
</property>

如果你用的java8,则多会报此错误。