大数据技术
2024年1月29日

Ambari安装（一）

作者 高劲松

服务器规划

序号主机名称规格IP 
1Hdp-node18G/100G10.168.1.21Mysql ，Master
2Hdp-node28G/100G10.168.1.22 
3Hdp-node38G/100G10.168.1.23 
软件准备

公共设置

网络配置

配置固定IP，并且保证能够上外网

vi /etc/sysconfig/network-scripts/ifcfg-ens33

IPADDR=10.168.1.21

NETMASK=255.255.255.0

GATEWAY=10.168.1.1

BOOTPROTO=static

[root@localhost ~]# vi /etc/resolv.conf

# Generated by NetworkManager

nameserver 8.8.8.8

service network start

设置主机名

hostnamectl set-hostname 命令用于设置主机名。

使用方法为： hostnamectl set-hostname [new_hostname] 。

例如，要将主机名更改为 “mycomputer”，可以使用命令 hostnamectl set-hostname mycomputer 。

设置etc/hosts（所有节点）

10.168.1.21 node1

10.168.1.22 node2

10.168.1.23 node3

关闭防火墙（所有节点）

systemctl stop firewalld
systemctl status firewalld
systemctl disable firewalld

关闭防火墙：systemctl stop firewalld

查看防火墙的状态：systemctl status firewalld

设置开机不启动：systemctl disable firewalld

查看防火墙服务是否设置开机启动：systemctl is-enabled firewalld

设置关闭selinux（所有节点）

sudo sed -i ‘s/^SELINUX=enforcing$/SELINUX=disabled/’ /etc/sysconfig/selinux

安装一些系统所有必要的工具（所有节点）

yum -y install gcc gcc-c++ kernel-devel libtirpc-devel
yum -y install rpm-build yum -y install net-tools
yum -y install openssl openssl-devel patch
yum -y install wget curl
yum -y install bash-completion
yum -y install unzip zip
yum -y install libtirpc libtirpc-devel

文件句柄数配置（所有节点）

编辑配置文件：/etc/security/limits.conf，

​​vim /etc/security/limits.conf​​

添加如下配置： * soft nproc 204800
* hard nproc 204800
* soft nofile 655350
* hard nofile 655350

修改Linux系统内核内存参数（属于调优，可以不用调整）（所有节点）

# 编辑配置文件：/etc/sysctl.conf，

​​vim /etc/sysctl.conf​​

添加如下配置： vm.swappiness=1
vm.min_free_kbytes=4194304
vm.zone_reclaim_mode=0

# 使修改生效：sysctl -p

关闭透明大页THP（需重启）（所有节点）

# 查看透明大页情况：

登录后复制 cat /sys/kernel/mm/transparent_hugepage/defrag
cat /sys/kernel/mm/transparent_hugepage/enabled

# 关闭透明大页：vim /etc/rc.d/rc.local，添加如下配置：

登录后复制 if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
  echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
  echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

# 保存退出，然后赋予rc.local文件执行权限：chmod +x /etc/rc.d/rc.local

安装时间同步工具（所有节点）

#安装时间同步工具（所有节点）

#ntp或者chrony yum install -y ntp
systemctl enable ntpd

java安装 （所有节点）

1.java解压安装

cd /opt tar -zxvf jdk1.8.0_231.tar.gz

2.编辑环境变量配置

​​vim /etc/profile ​​ export JAVA_HOME=/opt/jdk1.8.0_231
export PATH=$JAVA_HOME/bin:$PATH

3.生效环境变量配置

​​source /etc/profile​​

4.检验JDK配置结果

​​jave -version​

配置SSH无密码登陆（ambari-server节点或主节点，建议集群所有集群都做SSH无密码登陆）

ssh-keygen -t rsa
ssh-copy-id node1
ssh-copy-id node2
ssh-copy-id node3

验证： ssh node1
ssh node2
ssh node3

安装mysql（Node1节点）

解压mysql-5.7的RPM安装包

​​tar -xvf mysql-5.7.20-1.el7.x86_64.rpm-bundle.tar​​

查找系统自带mysql或者maradb

rpm -qa |grep mysql
rpm -qa |grep mariadb

如果找到按下面命令删除对应包

​​rpm -e mariadb-libs --nodeps​​

先yum安装部分依赖包

yum -y install perl perl-devel autoconf libaio libaio-devel perl-Time-HiRes perl-JSON perl-Data-Dumper net-tools numactl

安装rpm包

rpm -ivh mysql-community-common-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-compat-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-embedded-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-client-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-devel-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-embedded-compat-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-embedded-devel-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-server-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-test-5.7.20-1.el7.x86_64.rpm

查看状态并启动mysqld

# 查看mysql服务 service mysqld status
#systemctl status mysqld.service

# 启动mysql服务 service mysqld start
#systemctl start mysqld.service.

# 查看密码 cat /var/log/mysqld.log | grep password

登陆修改且密码

mysql -u root -p

# 首次登录必须先修改密码（强密码） alter user ‘root’@’localhost’ identified by ‘root__123AbCdHjKl’;

设置可以配置弱密码，（这步可以不用配置，如果使用弱密码则需要配置） SHOW variables LIKE ‘validate_password%’;
set global validate_password_policy=0;
set global validate_password_length=4;

登录后复制 # 可以修改为弱密码啦
#ALTER USER ‘root’@’localhost’ IDENTIFIED BY “bigdata123”;
#flush privileges;
# 更新权限

配置root可以远程访问

use mysql;
update user set host = ‘%’ where user =’root’;
# 配置可远程登录
flush privileges;
# 更新权限
exit;
# 退出

配置mysql服务器编码

# 查看mysql服务的编码 show variables like ‘%char%’;

# 修改MySQL配置文件：/etc/my.cnf

​​vim /etc/my.cnf​​

# 在[mysqld]下添加  [mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
default-storage-engine=INNODB
#init_connect=’SET NAMES utf8mb4′

# 在[client]下添加（如果没有[client]，则创建） [client]
default-character-set = utf8mb4
[mysql]
default-character-set = utf8mb4

重启mysql服务

service mysqld restart
#systemctl restart mysqld.

设置开机启动

chkconfig mysqld on
#systemctl enable mysqld

msyql服务器安装完成。 

安装httpd 或者Nginx服务

# 安装服务 yum install httpd -y

# 启动httpd服务：systemctl start httpd

# 查看httpd状态：systemctl status httpd

# 设置开启启动：systemctl enable httpd

# 查看是否开机启动：systemctl is-enabled httpd

把HDP文件放到httpd去

# 解压Ambari压缩包
tar -zxvf /root/ambari-2.7.5.0-centos7.tar.gz -C /var/www/html/
# 解压HDP压缩包
tar -zxvf /root/HDP-3.1.5.0-centos7-rpm.tar.gz -C /var/www/html/
# 解压HDP-UTILS压缩包
tar -zxvf /root/HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/
# 解压HDP-GPL压缩包
tar -zxvf /root/HDP-GPL-3.1.5.0-centos7-gpl.tar.gz -C /var/www/html/

创建对应的HDP源

在/etc/yum.repo.d/下创建4个文件，请按照实际的ambari-server服务器IP来修改下面文件内容

YUM源配置
[root@node1 yum.repos.d]# cat HDP.repo
[HDP-3.1.5]
name=HDP-3.1.5
baseurl=http://node1/HDP/centos7/3.1.5.0-152/
path=/
enabled=1
gpgkey=http://node1/HDP/centos7/3.1.5.0-152/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
gpgcheck=1
priority=1
[root@node1 yum.repos.d]# cat HDP-GPL.repo
[HDP-3.1.5-GPL]
name=HDP-3.1.5-GPL
baseurl=http://node1/HDP-GPL/centos7/3.1.5.0-152/
path=/
enabled=1
gpgkey=http://node1/HDP-GPL/centos7/3.1.5.0-152/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
gpgcheck=1
priority=1
[root@node1 yum.repos.d]# cat HDP-UTILS.repo
[HDP-UTILS-1.1.0.22]
name=HDP-UTILS-1.1.0.22
baseurl=http://node1/HDP-UTILS/centos7/1.1.0.22/
path=/
enabled=1
gpgkey=http://node1/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
gpgcheck=1
priority=1
[root@node1 yum.repos.d]# cat ambari.repo
[ambari-2.7.5.0-72]
name=ambari-2.7.5.0-72
baseurl=http://node1/ambari/centos7/2.7.5.0-72/
path=/
enabled=1
gpgkey= http://node1/ambari/centos7/2.7.5.0-72/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
gpgcheck=1
priority=1
[root@node1 yum.repos.d]#

拷贝HDP库源文件到所有服务器

scp *.repo root@node2:/etc/yum.repos.d/
scp *.repo root@node3:/etc/yum.repos.d/

测试HDP源（所有服务器）

yum clean all
yum repolist
yum makecache

安装Ambari-Server

安装一些yum工具命令，一般系统已经安装

yum install yum-utils createrepo yum-plugin-priorities

安装ambari-server

yum -y install ambari-server

安装ambari-server的mysql的驱动

mkdir /usr/share/java
cp mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar
cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/

配置ambari-server的mysql的驱动

配置/etc/ambari-server/conf/ambari.properties，添加如下行 server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

执行ambari-server的mysql的驱动注册

# ambari-server setup –jdbc-db=mysql –jdbc-driver=/usr/share/java/mysql-connector-java.jar
Using python  /usr/bin/python
Setup ambari-server
Copying /usr/share/java/mysql-connector-java.jar to /var/lib/ambari-server/resources/mysql-connector-java.jar
If you are updating existing jdbc driver jar for mysql with mysql-connector-java.jar. Please remove the old driver jar, from all hosts. Restarting services that need the driver, will automatically copy the new jar to the hosts.
JDBC driver was successfully initialized.
Ambari Server ‘setup’ completed successfully.

初始化ambari-server

# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux…
SELinux status is ‘disabled’
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership…
Checking firewall status…
Checking JDK…
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/jdk1.8.0_231
Validating JDK on Ambari Server…done.
Check JDK version for Ambari Server…
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement…
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (y)?
Completing setup…
Configuring database…
Enter advanced database configuration [y/n] (n)? y
Configuring database…
==============================================================================
Choose one of the following options:
[1] – PostgreSQL (Embedded)
[2] – Oracle
[3] – MySQL / MariaDB
[4] – PostgreSQL
[5] – Microsoft SQL Server (Tech Preview)
[6] – SQL Anywhere
[7] – BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata): ambari123
Re-enter password:
Configuring ambari database…
Configuring remote database connection properties…
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views…
ambari-admin-2.7.5.0.72.jar
….
Ambari repo file doesn’t contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership…
Ambari Server ‘setup’ completed successfully.

ambari数据库的导入

— 创建ambari数据库
CREATE DATABASE ambari;
— 创建ambari用户
CREATE USER ‘ambari’@’%’ IDENTIFIED BY ‘ambari123’;
— 给创建的ambari赋权
GRANT ALL PRIVILEGES ON *.* TO ‘ambari’@’%’;

#CREATE USER ‘ambari’@’localhost’ IDENTIFIED BY ‘ambari123’;
#GRANT ALL PRIVILEGES ON *.* TO ‘ambari’@’localhost’;
#CREATE USER ‘ambari’@’hadoop79’ IDENTIFIED BY ‘ambari123’;
#GRANT ALL PRIVILEGES ON *.* TO ‘ambari’@’hadoop79’;
— 刷新权限
flush privileges;

登录后复制 — 导入ambari的数据库
use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

登录后复制 — 验证数据表
show tables;
— 验证用户
use mysql;
select host,user from user where user=’ambari’;

登录后复制 — #创建其它大数据组件的库
CREATE DATABASE hive;
use hive;
CREATE USER ‘hive’@’%’ IDENTIFIED BY ‘hive123’;
GRANT ALL PRIVILEGES ON *.* TO ‘hive’@’%’;
#CREATE USER ‘hive’@’localhost’ IDENTIFIED BY ‘hive123’;
#GRANT ALL PRIVILEGES ON *.* TO ‘hive’@’localhost’;
#CREATE USER ‘hive’@’hadoop79’ IDENTIFIED BY ‘hive123’;
#GRANT ALL PRIVILEGES ON *.* TO ‘hive’@’hadoop79’;
FLUSH PRIVILEGES;

CREATE DATABASE oozie;
use oozie;
CREATE USER ‘oozie’@’%’ IDENTIFIED BY ‘oozie123’;
GRANT ALL PRIVILEGES ON *.* TO ‘oozie’@’%’;
#CREATE USER ‘oozie’@’localhost’ IDENTIFIED BY ‘oozie123’;
#GRANT ALL PRIVILEGES ON *.* TO ‘oozie’@’localhost’;
#CREATE USER ‘oozie’@’hadoop79’ IDENTIFIED BY ‘oozie123’;
#GRANT ALL PRIVILEGES ON *.* TO ‘oozie’@’hadoop79’;
FLUSH PRIVILEGES;

安装ambari-agent(所有节点执行)

yum -y install ambari-agent
systemctl start ambari-agent.

启动ambari服务

启动ambari服务
#启动
ambari-server start
#查看状态
#ambari-server status
#停止
ambari-server stop

发表回复

您的电子邮箱地址不会被公开。 必填项已用*标注