|
|
第1行: |
第1行: |
| | =op= |
|
| |
|
| https://cn.cloudera.com/ 中文文档
| | 去 godaddy 替换为 Cloudflare 的名称服务器 |
| | |
| =pre=
| |
| <pre>
| |
| 系统初始化(每个服务器都要做)
| |
| # 关闭iptables、禁用selinux
| |
| | |
| sudo systemctl disable firewalld
| |
| sudo systemctl stop firewalld
| |
| | |
| sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config && setenforce 0
| |
| | |
| | |
| # 每台服务器之间设置免密认证
| |
| # CM
| |
| cat >> /etc/hosts <<EOF
| |
| 192.168.88.40 cdh.master master
| |
| 192.168.88.41 cdh.node1 node1
| |
| 192.168.88.42 cdh.node2 node2
| |
| #192.168.1.150 cdh.node3.aniu.so node3
| |
| EOF
| |
| ## 注:在每台服务器配置hosts,master和node1/2/3代表服务器的主机名
| |
| | |
| sudo hostnamectl set-hostname "master"
| |
| | |
| | |
| ssh免密码登录配置(每一台)
| |
| 1. 在master 生成key 并自己配置自己的pub key
| |
| | |
| ssh-keygen -t dsa
| |
| ssh-copy-id -i ~/.ssh/id_dsa.pub 192.168.88.40 #这里是master ip 40
| |
| 2. 主服务器与其它服务器免密码,在主节点上以下命令发送到其他对应服务器:
| |
| | |
| rsync -avz .ssh/ 192.168.88.41:~/
| |
| | |
| | |
| | |
| | |
| | |
| | |
| # 设置swap参数
| |
| echo never > /sys/kernel/mm/transparent_hugepage/defrag #建议写到开启启动新里
| |
| sysctl -w vm.swappiness=0 # 建议写进sysctl.conf
| |
| | |
| cat >>/etc/sysct <<EOF
| |
| vm.swappiness=0
| |
| EOF
| |
| | |
| sysctl -p
| |
| | |
| # 设置ntp同步服务器时间
| |
| | |
| yum install ntp -y
| |
| | |
| Edit the /etc/ntp.conf file to add NTP servers, as in the following example.
| |
| server 0.cn.pool.ntp.org
| |
| server 1.pool.ntp.org
| |
| server 2.pool.ntp.org
| |
| | |
| sudo systemctl start ntpd
| |
| sudo systemctl enable ntpd
| |
| | |
| | |
| #for centos6
| |
| */2 * * * * /usr/sbin/ntpdate 0.cn.pool.ntp.org >> /dev/null 2>&1
| |
| | |
| | |
| | |
| https://www.cloudera.com/documentation/enterprise/6/6.2/topics/install_cdh_disable_iptables.html#install_cdh_disable_iptables
| |
| | |
| | |
| | |
| | |
| | |
| Cloudera安装步骤
| |
| 参考:https://www.cloudera.com/documentation/enterprise/latest/topics/installation_installation.html
| |
| | |
| 阶段1:安装JDK(忽略)
| |
| [Java SE 8 Downloads](http://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html)
| |
| export JAVA_HOME=/usr/java/jdk.1.8.0_nn
| |
| # java -version
| |
| java version "1.8.0_144"
| |
| Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
| |
| Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
| |
| # 注 此处不用安装JDK,因为CM源有封装好的jdk,
| |
| | |
| 阶段2:设置数据库
| |
| scp ~/rpm/rpmbuild2019/mysql20190506/RPMS/x86_64/ops_mysql-5.7.18-1.0.x86_64.rpm root@192.168.88.40:~
| |
| | |
| # 使用mysql数据库,提前安装好mysql
| |
| # mysql -u root -ppassword -e "create database cmf DEFAULT CHARACTER SET utf8;"
| |
| # mysql -u root -ppassword -e "GRANT ALL PRIVILEGES ON `cmf`.* TO 'cmf'@'localhost' IDENTIFIED BY '224088' ";"
| |
| | |
| | |
| | |
| 阶段3:安装Cloudera Manager服务器
| |
| | |
| dns client 改为 8.8.4.4
| |
| # 配置cloudera-cdh源和cloudera-manager源
| |
| # cloudera-manager
| |
| cd /etc/yum.repos.d/
| |
| wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/cloudera-manager.repo
| |
| | |
| # cloudera-cdh
| |
| wget https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/cloudera-cdh5.repo
| |
| | |
| # 安装jdk和cloudera-manager
| |
| sudo yum install oracle-j2sdk1.7 -y
| |
| | |
| | |
| | |
| sudo yum install cloudera-manager-daemons cloudera-manager-server -y
| |
| | |
| | |
| | |
| sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
| |
| | |
| | |
| axel https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
| |
| | |
| | |
| axel https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/cloudera-manager-daemons-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm
| |
| | |
| aria2c or 迅雷
| |
| axel https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/cloudera-manager-server-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm
| |
| | |
| | |
| | |
| 阶段4:启动CM服务并通过浏览器访问
| |
| systemctl start cloudera-scm-server
| |
| | |
| #有问题
| |
| tailf /var/log/cloudera-scm-server/cloudera-scm-server.log
| |
| | |
| </pre>
| |
| | |
| | |
| =troubleshooting=
| |
| <pre>
| |
| 安装这个就行了
| |
| mysql-connector-java
| |
| | |
| | |
| 安装MySQL JDBC Driver
| |
| wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
| |
| tar zxvf mysql-connector-java-5.1.46.tar.gz && mkdir -p /usr/share/java/
| |
| cd mysql-connector-java-5.1.46 && cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar
| |
| | |
| | |
| 下载 mysql-connector-java-8.0.11-1.el7.noarch.rpm
| |
| https://dev.mysql.com/downloads/connector/j/
| |
| </pre>
| |
| | |
| | |
| | |
| [https://www.cnblogs.com/spec-dog/p/4664944.html 启动CM出现 “JDBC Driver class not found: com.mysql.jdbc.Driver” 以及“Error creating bean with name 'serverLogFetcherImpl'”问题的解决方法]
| |
| | |
| [https://blog.csdn.net/hchhan/article/details/81106992 mysql-connector-java与mysql版本的对应]
| |
| | |
| | |
| [https://blog.csdn.net/zhang5476499/article/details/53791710 centos7安装mysql并jdbc测试]
| |
| | |
| | |
| | |
| | |
| | |
| [https://blog.csdn.net/baidu_36649389/article/details/53217196 centos下JDBC的安装配置]
| |
| | |
| https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-versions.html
| |
|
| |
|
| | network--> 关掉 http3 听说会对国内更加友好 |
| =see also= | | =see also= |
| ==normal ins==
| |
| [https://www.jianshu.com/p/f804bd587d95 Centos7.6部署CDH6.1.0大数据平台教程]
| |
|
| |
| [https://blog.csdn.net/wh211212/article/details/78743191 Cloudera(CDH) 简介和在线安装]
| |
|
| |
|
| |
| [https://blog.csdn.net/lxbalex/article/details/83188753 从零搭建生产Hadoop集群(三)——CDH集群搭建]
| |
|
| |
|
| |
| [https://blog.csdn.net/weixin_39182877/article/details/79062988 CDH5-Cloudera-Server部署和启动]
| |
|
| |
| [https://blog.csdn.net/qq_23660243/article/details/60870527 CDH在Centos 7离线安装]
| |
|
| |
|
| |
| [https://blog.csdn.net/feixeyes/article/details/38084403 适合国内网速的CDH5安装]
| |
|
| |
| [https://blog.csdn.net/liangzelei/article/details/80359461 Hadoop cdh版本搭建]
| |
|
| |
| ==docker ins==
| |
| [https://www.jianshu.com/p/f98741f3f8fa docker安装cloudera]
| |
|
| |
|
| [https://www.jianshu.com/p/2ecec335acbd centos安装Docker CDH+kafka]
| |
|
| |
|
| | [https://support.cloudflare.com/hc/zh-cn/articles/360000841472 通过自动化将多个站点添加到 Cloudflare ] |
|
| |
|
| | [https://zhuanlan.zhihu.com/p/29891330 CloudFlare免费CDN加速使用方法] |
|
| |
|
| ==learn==
| | https://support.cloudflare.com/hc/zh-cn/articles/360021801151-%E5%BC%80%E5%A7%8B%E4%BD%BF%E7%94%A8-Cloudflare-DNS |
| [https://www.zhihu.com/question/19795366 零基础学习 Hadoop 该如何下手]
| |
|
| |
|
| | [https://www.jianshu.com/p/7f38989ffa15 CloudFlare 节点优化——从入门到放弃] |
|
| |
|
| | [https://www.zhihu.com/question/57496274 用 cloudflare 的 CDN 比不用还慢,有没有其他的建议?] |
|
| |
|
| [https://blog.csdn.net/yuan_xw/article/details/50003197 Hadoop教程(一) Hadoop入门教程] | | [https://zhuanlan.zhihu.com/p/82909515 cloudflare使用入门教程,国外最好免费CDN] |
|
| |
|
| https://blog.csdn.net/wh211212/article/details/78743191
| |
|
| |
|
| [[category:ops]] [[category:bigdata]] | | [https://www.iplaysoft.com/p/cloudflare-dns 1.1.1.1 好记牛逼 - Cloudflare 推出全球平均速度最快的 DNS!] |
| | [[category:ops]] |