centos7脚本离线化安装CDH6.3版本（ansible闪亮登场~~~）
CDH好不好，毋庸置疑，具体体现在该平台是闭源的。逻辑是这样的：既然敢闭源，那么，说明这平台确实好用，闭源都不怕挣不到钱。当然，作为大数据平台管理平台之一的CDH好像也有底气闭源啦。
软件开或者闭源，都只是它的一种运营模式，在此，不过多评价。
CDH有三种使用形式，第一，白嫖，功能砍了很多，第二，免费试用期限，免费期内和收费的版本一样功能，过期了砍成白嫖功能。第三，付费版本，所有功能都开放，如何选择，不在本文讨论范围，文章末尾会有给出详细图解。
那么，本文主要是如何自动化的脚本安装CDH6.3版本，在使用脚本前，我想，有些工作还是手动做吧，毕竟脚本不能写的太庞大，臃肿了。
基本的服务器信息概述：
说明：CDH安装的时候非常吃内存，8G是勉强可以跑，推荐是16G，虚拟机设置16G的话，宿主机可能先不行了。硬盘大一些，是怕安装的时候磁盘满造成意外停止。
文件下载地址：
链接：https://pan.baidu.com/s/12wNH03PZhqtSHNIwP8LgfA
提取码：cdhf
一，CDH脚本安装前的环境搭建工作
（1）关闭防火墙和selinux，以防止不必要的干扰。
关闭防火墙：systemctl disable firewalld && systemctl stop firewalld
关闭selinux： vim /etc/selinux/config 文件，修改成disabled
（2）ntpd时间服务器的搭建（时间服务器搭建完毕并不能立即使用，需要等待若干分钟，搭建写在脚本内并不合适）
（3）服务器之间的免密登陆
（4）本地仓库的搭建（基础软件安装仓库，一般使用centos7的系统安装ISO格式文件搭建，基础命令指得是gcc，openssl-devel这些基本的软件）
（5）编译安装MySQL5.7
编译安装MySQL也是脚本自动安装，请参照https://blog.csdn.net/alwaysbefine/article/details/118440336安装，并将后续初始化部分执行完成。
（6）进入MySQL，执行以下SQL语句，建立库和用户
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
grant all privileges on scm.* to scm@'localhost' identified by 'scm';
grant all privileges on scm.* to scm@'%' identified by 'scm';
flush privileges;
注意，我这里MySQL的scm用户设置的密码是scm，当然不建议这么简单的密码，请按实际需要更改，脚本内初始化数据库也要做相应更改哦。
（7）执行cdh安装脚本，该脚本放置在主服务器192.168.88.11上。
安装cdh的脚本：
#!/bin/bash # author zsk_john NGINX_HOME=`find / -name ansible_offline.tar.gz` tar zxf $NGINX_HOME rpm -ivh ansible_offline/* --nodeps cp -f /root/CDH-offline/ansible-hosts /etc/ansible/hosts ansible all -m copy -a'src=~/host.txt dest=~/' ansible all -m copy -a'src=~/freelogin.sh dest=~/' ansible all -m shell -a 'bash ~/freelogin.sh' JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera/ PATH=.$PATH:$JAVA_HOME/bin CLASSPATH=$JAVA_HOME/jre/lib/ext:$JAVA_HOME/lib/tools.jar export CLASSPATH PATH JAVA_HOME export PATH=./usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/usr/java/jdk1.8.0_181-cloudera//bin:/usr/local/src/cmake/bin:/usr/local/mysql/bin ansible all -m copy -a 'src=~/CDH-offline/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm dest=~/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm' ansible all -m copy -a 'src=~/CDH-offline/enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm dest=~/enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm' ansible all -m copy -a 'src=~/CDH-offline/cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm dest=~/cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm' ansible all -m copy -a 'src=~/CDH-offline/cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm dest=~/cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm' ansible all -m yum -a 'state=installed name=MySQL-python,bind-utils,psmisc,cyrus-sasl-plain,cyrus-sasl-gssapi,portmap,lsb,httpd,mod_ssl,openssl-devel,python-psycopg2' sleep 120 ansible all -m shell -a'rpm -ivh ~/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm' ansible all -m shell -a'rpm -ivh ~/enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm' ansible all -m shell -a'rpm -ivh ~/cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm' ansible all -m shell -a'rpm -ivh ~/cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm' ansible all -m shell -a 'systemctl enable cloudera-scm-agent' ansible all -m shell -a 'systemctl start cloudera-scm-agent' ansible all -m copy -a 'src=/etc/cloudera-scm-agent/config.ini dest=/etc/cloudera-scm-agent/config.ini' ansible all -m shell -a 'mkdir -p /usr/share/java' ansible all -m copy -a 'src=~/CDH-offline/mysql-connector-java-5.1.47.jar dest=/usr/share/java/mysql-connector-java.jar' rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm systemctl enable cloudera-scm-server systemctl start cloudera-scm-server ansible all -m shell -a 'systemctl restart cloudera-scm-agent' #初始化数据库，最后那个scm 是前面指定的scm用户密码，这里写成固定的，可根据需要更改哦 /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm scm systemctl restart cloudera-scm-server mv ~/CDH-offline/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel* /opt/cloudera/parcel-repo/
2021-07-10 17:43:42,768 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server. 2021-07-10 17:43:42,771 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Discovered parcel on CM server: CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel 2021-07-10 17:43:42,771 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.torrent 2021-07-10 17:43:42,778 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Creating single-file torrent for CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel... 2021-07-10 17:43:42,782 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashing data from CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel with 8 threads (3972 pieces)... 2021-07-10 17:43:43,384 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 10% complete 2021-07-10 17:43:43,803 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 20% complete 2021-07-10 17:43:44,284 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 30% complete 2021-07-10 17:43:44,687 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 40% complete 2021-07-10 17:43:45,120 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 50% complete 2021-07-10 17:43:45,532 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 60% complete 2021-07-10 17:43:45,952 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 70% complete 2021-07-10 17:43:46,370 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 80% complete 2021-07-10 17:43:46,766 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 90% complete 2021-07-10 17:43:47,203 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashed 1 file(s) (2082186246 bytes) in 3972 pieces (3972 expected) in 4417.8ms. 2021-07-10 17:43:47,210 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Single-file torrent information: 2021-07-10 17:43:47,210 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Torrent name: CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel 2021-07-10 17:43:47,210 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Announced at: Seems to be trackerless 2021-07-10 17:43:47,210 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Created on..: Sat Jul 10 17:43:42 CST 2021
通过以上步骤，我们的CDH的web端应该就已经可以正常工作了。检测是否是正常的有如下方法：
[root@master ~]# netstat -antup | grep 7180 tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 1027/java tcp 0 0 192.168.0.16:7180 192.168.0.111:54364 ESTABLISHED 1027/java [root@master ~]# jps 1027 Main 4117 Jps
可以看到服务端口已开。
mysql> use scm; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +--------------------------------+ | Tables_in_scm | +--------------------------------+ | audits | | auth_roles | | auth_roles_scopes | | client_configs | | client_configs_to_hosts | | cluster_activated_releases | | cluster_activated_releases_aud | | cluster_managed_releases | | cluster_undistributed_releases | | clusters | | clusters_aud | | cm_peers | | cm_version | | command_schedules | | commands | | commands_detail | | config_containers | | configs | | configs_aud | | credentials | | data_context_to_base_service | | data_contexts | | diagnostics_events | | external_accounts | | external_accounts_aud | | external_mapping_auth_roles | | external_mappings | | global_settings | | host_template_to_role_conf_grp | | host_templates | | hosts | | hosts_aud | | metrics | | parcel_components | | parcels | | process_active_releases | | processes | | processes_detail | | releases | | releases_aud | | revisions | | role_config_groups | | role_config_groups_aud | | role_staleness_status | | roles | | roles_aud | | schema_version | | services | | services_aud | | snapshot_policies | | user_auth_roles | | user_roles | | user_settings | | users | +--------------------------------+ 54 rows in set (0.00 sec)
可以看到scm这个数据库有54张表，如果表的数目不对，请删除库重新初始化。
可以看到端口被激活就证明没有问题了。此时，我们的agent也就是客户端并没有做任何改动，这样服务器是认不到的，也就是192.168.88.11是认不到的。因此，需要修改文件
vim /etc/cloudera-scm-agent/config.ini
[General] # Hostname of the CM server. server_host=localhost # Port that the CM server is listening on. server_port=7182
更改为：
[General] # Hostname of the CM server. server_host=192.168.88.11 # Port that the CM server is listening on. server_port=7182
然后重启客户端：
在192.168.88.11这个服务器上执行：ansible all -m shell -a 'systemctl restart cloudera-scm-agent' 这条命令即可。
二，web端的操作
经过 第一大步的操作，如果严格安装执行，应该web端已经可以进入了。
