查看“Cloudera(CDH)部署”的源代码


https://cn.cloudera.com/ 中文文档 

=pre=
<pre>
系统初始化(每个服务器都要做）
# 关闭iptables、禁用selinux

sudo systemctl disable firewalld
sudo systemctl stop firewalld

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config && setenforce 0


# 每台服务器之间设置免密认证
# CM
cat >> /etc/hosts <<EOF
192.168.88.40 cdh.master master
192.168.88.41 cdh.node1 node1
192.168.88.42 cdh.node2 node2
#192.168.1.150 cdh.node3.aniu.so node3
EOF
## 注：在每台服务器配置hosts，master和node1/2/3代表服务器的主机名

sudo hostnamectl set-hostname "master"


ssh免密码登录配置（每一台）
1. 在master 生成key 并自己配置自己的pub key 

ssh-keygen -t dsa
ssh-copy-id -i ~/.ssh/id_dsa.pub 192.168.88.40 #这里是master ip  40 
2. 主服务器与其它服务器免密码，在主节点上以下命令发送到其他对应服务器：

rsync -avz  .ssh/  192.168.88.41:~/






# 设置swap参数
echo never > /sys/kernel/mm/transparent_hugepage/defrag  #建议写到开启启动新里
sysctl -w vm.swappiness=0  # 建议写进sysctl.conf

cat >>/etc/sysct <<EOF
vm.swappiness=0
EOF

sysctl -p

# 设置ntp同步服务器时间

yum install ntp -y 

Edit the /etc/ntp.conf file to add NTP servers, as in the following example.
server 0.cn.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org

sudo systemctl start ntpd
sudo systemctl enable ntpd


#for centos6 
*/2 * * * * /usr/sbin/ntpdate 0.cn.pool.ntp.org >> /dev/null 2>&1



https://www.cloudera.com/documentation/enterprise/6/6.2/topics/install_cdh_disable_iptables.html#install_cdh_disable_iptables





Cloudera安装步骤 
参考：https://www.cloudera.com/documentation/enterprise/latest/topics/installation_installation.html

阶段1：安装JDK（忽略）
[Java SE 8 Downloads](http://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html)
export JAVA_HOME=/usr/java/jdk.1.8.0_nn
# java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
# 注 此处不用安装JDK，因为CM源有封装好的jdk，

阶段2：设置数据库
scp ~/rpm/rpmbuild2019/mysql20190506/RPMS/x86_64/ops_mysql-5.7.18-1.0.x86_64.rpm  root@192.168.88.40:~

# 使用mysql数据库，提前安装好mysql
# mysql -u root -ppassword -e "create database cmf DEFAULT CHARACTER SET utf8;"
# mysql -u root -ppassword -e "GRANT ALL PRIVILEGES ON `cmf`.* TO 'cmf'@'localhost' IDENTIFIED BY '224088' ";"



阶段3：安装Cloudera Manager服务器

dns client 改为 8.8.4.4
# 配置cloudera-cdh源和cloudera-manager源
# cloudera-manager
cd /etc/yum.repos.d/
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/cloudera-manager.repo

# cloudera-cdh
wget https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/cloudera-cdh5.repo

# 安装jdk和cloudera-manager
sudo yum install oracle-j2sdk1.7 -y 



sudo yum install cloudera-manager-daemons cloudera-manager-server -y



sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server


axel https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm


axel  https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/cloudera-manager-daemons-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm

aria2c  or 迅雷  
axel https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/cloudera-manager-server-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm



阶段4：启动CM服务并通过浏览器访问
systemctl start cloudera-scm-server

#有问题 
tailf /var/log/cloudera-scm-server/cloudera-scm-server.log

</pre>


=troubleshooting=
<pre>
安装这个就行了
mysql-connector-java


安装MySQL JDBC Driver
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
tar zxvf mysql-connector-java-5.1.46.tar.gz  && mkdir -p /usr/share/java/
cd mysql-connector-java-5.1.46 && cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar


下载 mysql-connector-java-8.0.11-1.el7.noarch.rpm
https://dev.mysql.com/downloads/connector/j/
</pre>



[https://www.cnblogs.com/spec-dog/p/4664944.html 启动CM出现 “JDBC Driver class not found: com.mysql.jdbc.Driver” 以及“Error creating bean with name 'serverLogFetcherImpl'”问题的解决方法]

[https://blog.csdn.net/hchhan/article/details/81106992 mysql-connector-java与mysql版本的对应]


[https://blog.csdn.net/zhang5476499/article/details/53791710 centos7安装mysql并jdbc测试]





[https://blog.csdn.net/baidu_36649389/article/details/53217196 centos下JDBC的安装配置]

https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-versions.html

=see also=
==normal ins==
[https://www.jianshu.com/p/f804bd587d95 Centos7.6部署CDH6.1.0大数据平台教程]

[https://blog.csdn.net/wh211212/article/details/78743191 Cloudera(CDH) 简介和在线安装]


[https://blog.csdn.net/lxbalex/article/details/83188753 从零搭建生产Hadoop集群（三）——CDH集群搭建]


[https://blog.csdn.net/weixin_39182877/article/details/79062988 CDH5-Cloudera-Server部署和启动]

[https://blog.csdn.net/qq_23660243/article/details/60870527 CDH在Centos 7离线安装]


[https://blog.csdn.net/feixeyes/article/details/38084403 适合国内网速的CDH5安装]

[https://blog.csdn.net/liangzelei/article/details/80359461 Hadoop cdh版本搭建]

==docker ins==
[https://www.jianshu.com/p/f98741f3f8fa docker安装cloudera]

[https://www.jianshu.com/p/2ecec335acbd centos安装Docker CDH+kafka]



==learn==
[https://www.zhihu.com/question/19795366 零基础学习 Hadoop 该如何下手]



[https://blog.csdn.net/yuan_xw/article/details/50003197 Hadoop教程(一) Hadoop入门教程]

https://blog.csdn.net/wh211212/article/details/78743191

[[category:ops]] [[category:bigdata]]