Hadoop Yarn 开启jobHistory

hadoop01修改mapred-site.xml

hadoop01服务器修改mapred-site.xml，添加以下配置

cd /export/servers/hadoop-2.6.0-cdh5.14.0/etc/hadoop vim mapred-site.xml <property> <name>mapreduce.jobhistory.address</name> <value>hadoop01:10020</value> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>hadoop01:19888</value> </property>

hadoop01修改yarn-site.xml

hadoop01服务器修改yarn-site.xml，添加以下配置

cd /export/servers/hadoop-2.6.0-cdh5.14.0/etc/hadoop vim yarn-site.xml <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property> <property> <name>yarn.log-aggregation.retain-seconds</name> <value>604800</value> </property> <!--指定文件压缩类型用于压缩汇总日志--> <property> <name>yarn.nodemanager.log-aggregation.compression-type</name> <value>gz</value> </property> <!-- nodemanager本地文件存储目录--> <property> <name>yarn.nodemanager.local-dirs</name> <value>/export/servers/hadoop-2.6.0/yarn/local</value> </property> <!-- resourceManager 保存最大的任务完成个数 --> <property> <name>yarn.resourcemanager.max-completed-applications</name> <value>1000</value> </property>

修改后的文件分发到其他机器

将hadoop01修改后的mapred-site.xml和yarn-site.xml分发到其他机器上面去 hadoop01执行以下命令

cd /export/servers/hadoop-2.6.0-cdh5.14.0/etc/hadoop scp mapred-site.xml yarn-site.xml hadoop02:$PWD scp mapred-site.xml yarn-site.xml hadoop03:$PWD

重启yarn集群以及启动jobHistoryServer进程

hadoop01执行以下命令重启yarn集群

cd /export/servers/hadoop-2.6.0-cdh5.14.0/ sbin/stop-yarn.sh sbin/start-yarn.sh

启动jobhistoryserver

sbin/mr-jobhistory-daemon.sh start historyserver

页面访问jobhistoryserver

http://192.168.100.201:19888/jobhistory