前置章节：hadoop集群namenode启动ssh免密登录（hadoop完全分布式五）|11

集群启动

配置workers(3.x之前是slaves)，

删除localhost，添加102/103/104

[shaozhiqi@hadoop102 hadoop]$ pwd /opt/module/hadoop-3.1.2/etc/hadoop [shaozhiqi@hadoop102 hadoop]$ vi workers [shaozhiqi@hadoop102 hadoop]$ cat workers hadoop102 hadoop103 hadoop104 [shaozhiqi@hadoop102 hadoop]$

注：workers文件结尾不能有空格，文件中不能有空行

同步workers到其他机器

[shaozhiqi@hadoop102 hadoop]$ testxsync workers fname=workers pdir=/opt/module/hadoop-3.1.2/etc/hadoop -------hadoop103 ----------------------- sending incremental file list workers sent 113 bytes received 37 bytes 100.00 bytes/sec total size is 40 speedup is 0.27 ------hadoop104 ----------------------- sending incremental file list workers sent 113 bytes received 37 bytes 100.00 bytes/sec total size is 40 speedup is 0.27 [shaozhiqi@hadoop102 hadoop]$

验证是否同步成功

[shaozhiqi@hadoop103 hadoop]$ cat workers hadoop102 hadoop103 hadoop104 [shaozhiqi@hadoop103 hadoop]$ [shaozhiqi@hadoop104 hadoop]$ cat workers hadoop102 hadoop103 hadoop104 [shaozhiqi@hadoop104 hadoop]$

停止上节启动的namenode和datanode

[shaozhiqi@hadoop103 hadoop-3.1.2]$ sbin/hadoop-daemon.sh stop datanode shaozhiqi@hadoop104 hadoop-3.1.2]$ sbin/hadoop-daemon.sh stop datanode shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/hadoop-daemon.sh stop datanode shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/hadoop-daemon.sh stop namenode

版本提示用这个命令：hdfs --daemon stop

启动hdfs

sbin/start-dfs.sh

(hdfs包括了namenode和datanode两个节点。具体可查看大数据及hadoop简要概念|6

[shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/start-dfs.sh Starting namenodes on [hadoop102] hadoop102: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating. Starting datanodes hadoop103: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating. hadoop104: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating. Starting secondary namenodes [hadoop104] [shaozhiqi@hadoop102 hadoop-3.1.2]$

Jps发现其他都正常，但是namenode就是没有启动。。。

删除data logs，格式化namenode

[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs namenode –format

再次启动hdfs

[shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/start-dfs.sh Starting namenodes on [hadoop102] Starting datanodes hadoop104: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating. hadoop103: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating. Starting secondary namenodes [hadoop104] [shaozhiqi@hadoop102 hadoop-3.1.2]$ jps 7619 DataNode 7877 Jps 7494 NameNode [shaozhiqi@hadoop102 hadoop-3.1.2]$

验证

102

[shaozhiqi@hadoop102 hadoop-3.1.2]$ jps 7619 DataNode 7877 Jps 7494 NameNode [shaozhiqi@hadoop102 hadoop-3.1.2]$

103

[shaozhiqi@hadoop103 hadoop-3.1.2]$ jps 2931 Jps 2847 DataNode [shaozhiqi@hadoop103 hadoop-3.1.2]$

104

[shaozhiqi@hadoop104 hadoop-3.1.2]$ jps 5424 SecondaryNameNode 5467 Jps 5308 DataNode [shaozhiqi@hadoop104 hadoop-3.1.2]$

参照我们的规划配置，具体查看hadoop集群配置同步（hadoop完全分布式四）|10

配置规划

验证通过也可以看先我们的hdfs的namenodeweb,可以看到有三台机器ok

namenodeweb

 

启动yarn

根据我们在[**hadoop****集群配置同步[hadoop集群配置同步（hadoop完全分布式四）|10]的集群分配原则，我们的yarn的recourceManger在103所以yarn的启动只能在103起，否则会报错

sbin/start-yarn.sh

[shaozhiqi@hadoop103 hadoop-3.1.2]$ sbin/start-yarn.sh Starting resourcemanager Starting nodemanagers [shaozhiqi@hadoop103 hadoop-3.1.2]$ jps 3154 ResourceManager 3271 NodeManager 3611 Jps 2847 DataNode [shaozhiqi@hadoop103 hadoop-3.1.2]$

103ok

[shaozhiqi@hadoop104 hadoop-3.1.2]$ jps 5424 SecondaryNameNode 5782 Jps 5657 NodeManager 5308 DataNode [shaozhiqi@hadoop104 hadoop-3.1.2]$

104ok

[shaozhiqi@hadoop102 hadoop-3.1.2]$ jps 7619 DataNode 8052 NodeManager 8164 Jps 7494 NameNode [shaozhiqi@hadoop102 hadoop-3.1.2]$

102ok

启动成功下节测试我们的hadoop集群

由于要关闭电脑，所以停止hadoop集群启动时先起hdfs,然后yarn先停yarn 再停hdfs

停集群yarn

[shaozhiqi@hadoop103 hadoop-3.1.2]$ sbin/stop-yarn.sh Stopping nodemanagers Stopping resourcemanager [shaozhiqi@hadoop103 hadoop-3.1.2]$ jps 4145 Jps 2847 DataNode [shaozhiqi@hadoop103 hadoop-3.1.2]$

停集群hdsf

[shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/stop-dfs.sh Stopping namenodes on [hadoop102] Stopping datanodes Stopping secondary namenodes [hadoop104] [shaozhiqi@hadoop102 hadoop-3.1.2]$ jps 8941 Jps [shaozhiqi@hadoop102 hadoop-3.1.2]$

记得下次启动集群前格式namenode

转载于:https://www.cnblogs.com/shaozhiqi/p/11534727.html