1、准备三台centos7机器,这里我使用的vmware创建虚拟机,并在/etc/hosts配置host映射
192.168.8.144 master 192.168.8.145 slave1 192.168.8.146 slave22、配置ssh免密码登录 master执行
ssh_keygen -t dsa cat /root/.ssh/id_dsa.pub >> /root/.ssh/authorized_keys scp /root/.ssh/authorized_keys slave1:/root/.ssh scp /root/.ssh/authorized_keys slave2:/root/.ssh ssh slave1 ssh slave23、下载官网的jdk1.8、scala2.12.10、hadoop2.6.5、hbase1.2.2、hive1.2.2安装包并上传到centos7的master
scp -r jdk1.8 root@master:/Java scp -r scala2.12.10 root@master:/Java scp -r hadoop2.6.5 root@master:/Java scp -r hbase1.2.2 root@master:/Java4、master\slave1\slave2分别安装jdk1.8 安装vim命令
yum -y install vim* vim /etc/profile添加
export JAVA_HOME=/Java/jdk1.8 export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin export CLASSPATH=$CLASSPATH:.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib执行
source /etc/profile激活配置
5、关闭防火墙
systemctl stop firewalld.service systemctl disable firewalld.service6、修改hadoop/etc/hadoop/hadoop-env.sh和yarn-env.sh,设置为实际${JAVA_HOME}安装路径
export JAVA_HOME=/Java/jdk.187、修改core-site.xml 首先创建如下目录
mkdir -p /hadoop/hdfs/tmp添加
<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://master:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/hadoop/hdfs/tmp</value> </property> </configuration>8、修改hdfs-site.xml
<configuration> <property> <name>dfs.replication</name> <value>2</value> </property> </configuration>9、修改mapred-site.xml
复制
cp mapred-site.xml.template mapred-site.xml <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>10、修改yarn-site.xml
<configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.resourcemanager.address</name> <value>master:8040</value> </property> <property> <name>yarn.resourcemanager.scheduler.address</name> <value>master:8030</value> </property> <property> <name>yarn.resourcemanager.resource.tracker.address</name> <value>master:8025</value> </property> <property> <name>yarn.resourcemanager.admin.address</name> <value>8141</value> </property> <property> <name>yarn.resourcemanager.webapp.address</name> <value>master:8088</value> </property> </configuration>11、创建文件slaves
touch /hadoop/hadoop2.6.5/etc/hadoop/slaves添加
slave1 slave212、将修改后的hadoop安装包同步给slave1、slave2
scp -r hadoop-2.6.5 slave1:/hadoop scp -r hadoop-2.6.5 slave2:/hadoop13、修改所有节点的环境变量
vim /etc/profile添加
export HADOOP_HOME=/hadoop/hadoop-2.6.5 export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin14、在master节点格式化NameNode
hdfs namenode -format15、master节点启动
start-all.sh16、测试hadoop集群是否搭建成功 访问
master:50070 master:8088查看是否正常