欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

Centos7安装Hadoop-- 伪分布式配置

程序员文章站 2022-07-15 18:47:40
...

1.安装jdk8

2.配置免密登陆

2.安装hadoop

(1)下载Hadoop

wget https://archive.apache.org/dist/hadoop/core/hadoop-2.7.3/hadoop-2.7.3.tar.gz

(2)解压

tar zxvf hadoop-2.7.3.tar.gz -C /app

(3)配置

cd hadoop-2.7.3/etc/hadoop

①vi hadoop-env.sh

# 修改Java环境变量

export JAVA_HOME=/app/jdk1.8.0_181

② vi hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/tmp/hadoop_data/dfs/name</value>
    </property>
    <property>
        <name>dfs.namenode.data.dir</name>
        <value>file:/tmp/hadoop_data/dfs/data</value>
    </property>
</configuration>

3. vi core-site.xml

<configuration>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/tmp/hadoop_data</value>
    </property>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://0.0.0.0:9000</value>
    </property>
</configuration>

④创建存储目录

cd tmp

mkdir hadoop_data

5.运行Hadoop

cd hadoop-2.7.3/bin

 ./hdfs namenode -format

18/08/28 19:33:54 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
18/08/28 19:33:54 INFO util.ExitUtil: Exiting with status 0
18/08/28 19:33:54 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at localhost/127.0.0.1

⑥cd /sbin

# 启动fdfs

./start-dfs.sh  

Starting namenodes on [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is 55:ae:3d:63:67:9d:66:f0:f9:c0:6c:45:e8:a8:c9:e3.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
0.0.0.0: starting namenode, logging to /app/hadoop-2.7.3/logs/hadoop-root-namenode-localhost.localdomain.out
localhost: starting datanode, logging to /app/hadoop-2.7.3/logs/hadoop-root-datanode-localhost.localdomain.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /app/hadoop-2.7.3/logs/hadoop-root-secondarynamenode-localhost.localdomain.out

jps -m

[[email protected] sbin]# jps -m
18816 SecondaryNameNode
18627 DataNode
18533 NameNode
18940 Jps -m

⑦ 测试

cd bin

# 查看hdfs目录

./hdfs dfs -ls  / 

# 创建目录

./hdfs dfs -mkdir /test

[[email protected] bin]# ./hdfs dfs -ls /       
Found 1 items
drwxr-xr-x   - root supergroup          0 2018-08-28 19:49 /test