1.前置条件
安装jdk
设置 ssh 免密
生成公钥/私钥对:
ssh-keygen -t rsa
拷贝公钥至authorized_keys文件:
ssh-copy-id 主机名
2.下载修改配置
core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://0.0.0.0:9000/</value>
<description>默认的HDFS端口,用于NameNode与DataNode之间到的通讯,IP为NameNode的地址</description>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/lys/hdp/data/tmp</value>
<description>存放hadoop文件系统依赖的基本配置</description>
</property>
</configuration>
hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
yarn-site.xml
<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>0.0.0.0</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
3.hdfs用户修改
要将hdfs改成root
vi $HADOOP_HOME/bin/hdfs
HADOOP_SHELL_EXECNAME="root"
否则报错:ERROR: Cannot set priority of datanode process 3340
4.启动用户 修改
vim /home/lys/hdp/hadoop-3.3.1/etc/hadoop/hadoop-env.sh
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root
export JAVA_HOME=/usr/java/jdk1.8.0_311-amd64
export HDFS_DATANODE_USER=root
export HDFS_DATANODE_SECURE_USER=root
否则启动报错
root@35b0bcbf2dca:/home/hadoop/hadoop-3.3.1/sbin# ./start-all.sh
Starting namenodes on [0.0.0.0]
ERROR: Attempting to operate on hdfs namenode as root
ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting operation.
Starting datanodes
ERROR: Attempting to operate on hdfs datanode as root
ERROR: but there is no HDFS_DATANODE_USER defined. Aborting operation.
Starting secondary namenodes [35b0bcbf2dca]
ERROR: Attempting to operate on hdfs secondarynamenode as root
ERROR: but there is no HDFS_SECONDARYNAMENODE_USER defined. Aborting operation.
2021-12-18 08:52:34,650 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting resourcemanager
ERROR: Attempting to operate on yarn resourcemanager as root
ERROR: but there is no YARN_RESOURCEMANAGER_USER defined. Aborting operation.
Starting nodemanagers
ERROR: Attempting to operate on yarn nodemanager as root
ERROR: but there is no YARN_NODEMANAGER_USER defined. Aborting operation.
5.格式化并启动
/home/lys/hdp/hadoop-3.3.1/bin/hdfs namenode -format
sh /home/lys/hdp/hadoop-3.3.1/sbin/start-all.sh