JobHistoryServer无法正常启动

mac2023-02-02  21

今天在测试环境启动JobHistoryServer,发现无法正常启动,报错信息如下:

2019-10-28 17:44:33,030 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal handlers for [TERM, HUP, INT] 2019-10-28 17:44:33,666 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer failed in state INITED; cause: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService not found java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2186) at org.apache.hadoop.mapreduce.v2.hs.HistoryServerStateStoreServiceFactory.getStore(HistoryServerStateStoreServiceFactory.java:39) at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.createStateStore(JobHistoryServer.java:169) at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:131) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:219) at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:229) Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2154) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2178) ... 6 more Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2060) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2152) ... 7 more 2019-10-28 17:44:33,671 FATAL org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: Error starting JobHistoryServer java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2186) at org.apache.hadoop.mapreduce.v2.hs.HistoryServerStateStoreServiceFactory.getStore(HistoryServerStateStoreServiceFactory.java:39) at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.createStateStore(JobHistoryServer.java:169) at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:131) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:219) at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:229) Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2154) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2178) ... 6 more Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2060) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2152) ... 7 more 2019-10-28 17:44:33,681 INFO org.apache.hadoop.util.ExitUtil: Exiting with status -1 2019-10-28 17:44:33,697 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: SHUTDOWN_MSG:

通过mapred-site.xml的配置发现对应报错的配置如下:

<property> <name>mapreduce.jobhistory.recovery.store.class</name> <value>org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService</value> </property>

发现确实有对应的配置项,不过对应配置的含义尚不明确,于是去官网搜索相关配置

找到如下说明:

发现mapreduce.jobhistory.recovery.enable设置为true必须在mapreduce.jobhistory.recovery.store.class的配置指定才行,而测试环境中mapreduce.jobhistory.recovery.store.class指定的还是默认值,应该是没指定成功导致启动失败,由于对JobHistoryServer的高可用需求不那么重要,于是禁用其故障恢复,设置mapreduce.jobhistory.recovery.enable为false,即可正常启动JobHistoryServer

 

最新回复(0)