一、Hadoop搭建

1、创建hadoop名称的用户

addset hadoop
passwd hadoop

2、切换到root用户，关闭selinux

su – root
关闭selinux：vim /etc/selinux/config
SELINUX=disabled

3、切换到hadoop用户

su – hadoop

4、hadoop集群使用免密登录

cd 进入hadoop用户的家目录
ssh-keygen -t rsa [输入完后连按4个回车]
ssh localhost [yes，输入hadoop用户的密码]
ssh-copy-id localhost [输入hadoop用户的密码]

5、使用xftp上传hadoop、hive、java安装文件

6、解压软件包到/opt/module

cd
tar -zxvf ./jdk-8u181-linux-x64.tar.gz -C /opt/module/
tar -zxvf ./hadoop-2.7.3.tar.gz -C /opt/module/
tar -zxvf ./apache-hive-3.1.1-bin.tar.gz -C /opt/module/

7、编辑环境变量

ls -a #查看隐藏文件
vim ~/.bash_profile ，在文件末尾内容添加如下：

JAVA_HOME=/opt/module/jdk1.8.0_181
HADOOP_HOME=/opt/module/hadoop-2.7.3
HIVE_HOME=/opt/module/apache-hive-3.1.1-bin
PATH= P A T H : PATH: PATH:HOME/bin: J A V A H O M E / b i n : JAVA_HOME/bin: JAVAH/span>OME/bin:HADOOP_HOME/bin: H A D O O P H O M E / s b i n : HADOOP_HOME/sbin: HADOOPH/span>OME/sbin:HIVE_HOME/bin

export JAVA_HOME
export HADOOP_HOME
export HIVE_HOME
export PATH

8、重新加载该文件，使环境变量立即生效

source ~/.bash_profile

9、修改以下文件的主机名、及路径

修改hadoop的配置文件:cd /opt/module/hadoop-2.7.3/etc/hadoop
1.vim ./hadoop-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0_181
2.vim ./mapred-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0_181
3.vim ./yarn-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0_181
4.vim ./core-site.xml

5.vim ./hdfs-site.xml

6.cp ./mapred-site.xml.template ./mapred-site.xml
vim ./mapred-site.xml

7.vim ./yarn-site.xml

8.vim ./slaves
node100

10、格式化hadoop集群

hdfs namenode -format

11、启动/关闭hadoop集群

start-all.sh
stop-all.sh

12、测试hadoop是否集群开启

查看jps，有6个进程说明hadoop开启
[hadoop@localhost ~]$ jps
21968 SecondaryNameNode
22593 Jps
22248 NodeManager
22138 ResourceManager
21788 DataNode
21646 NameNode

二、hive搭建

1、在HDFS上创建存储Hive数据的目录

hadoop fs -mkdir /tmp
hadoop fs -mkdir -p /user/hive/warehouse
hadoop fs -chmod g+w /tmp
hadoop fs -chmod g+w /user/hive/warehouse
在hive的软件目录(/opt/module/apache-hive-3.1.1-bin/)下执行初始化命令
bin/schematool -dbType derby -initSchema
初始化成功后就会在hive的安装目录下生成derby.log日志文件和metastore_db元数据库

2、查询hadoop文件系统的根目录

hadoop fs -ls /;

3、可以在web查询hadoop目录

登录192.168.5.100:50070

4、启动hive

[hadoop@localhost apache-hive-3.1.1-bin]$ bin/hive
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/hadoop/.local/bin:/home/hadoop/bin:/home/hadoop/bin:/opt/module/jdk1.8.0_181/bin:/opt/module/hadoop-2.7.3/bin:/opt/module/hadoop-2.7.3/sbin:/opt/module/apache-hive-3.1.1-bin/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/apache-hive-3.1.1-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 4e9849e6-a9b1-4101-9288-6fcf11d0871f

Logging initialized using configuration in jar:file:/opt/module/apache-hive-3.1.1-bin/lib/hive-common-3.1.1.jar!/hive-log4j2.properties Async: true
Hive Session ID = 9992744b-7764-4dd8-aa5b-70777ffc7c3f
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive>

来源：cc抱富