“Cloudera(CDH)部署”与“Cloudflare”:页面之间的差异

来自linuxsa wiki
(页面间差异)
跳转到导航 跳转到搜索
Evan留言 | 贡献
导入1个版本
 
Evan留言 | 贡献
无编辑摘要
 
第1行: 第1行:
=op=


https://cn.cloudera.com/ 中文文档
去 godaddy 替换为 Cloudflare 的名称服务器
 
=pre=
<pre>
系统初始化(每个服务器都要做)
# 关闭iptables、禁用selinux
 
sudo systemctl disable firewalld
sudo systemctl stop firewalld
 
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config && setenforce 0
 
 
# 每台服务器之间设置免密认证
# CM
cat >> /etc/hosts <<EOF
192.168.88.40 cdh.master master
192.168.88.41 cdh.node1 node1
192.168.88.42 cdh.node2 node2
#192.168.1.150 cdh.node3.aniu.so node3
EOF
## 注:在每台服务器配置hosts,master和node1/2/3代表服务器的主机名
 
sudo hostnamectl set-hostname "master"
 
 
ssh免密码登录配置(每一台)
1. 在master 生成key 并自己配置自己的pub key
 
ssh-keygen -t dsa
ssh-copy-id -i ~/.ssh/id_dsa.pub 192.168.88.40 #这里是master ip  40
2. 主服务器与其它服务器免密码,在主节点上以下命令发送到其他对应服务器:
 
rsync -avz  .ssh/  192.168.88.41:~/
 
 
 
 
 
 
# 设置swap参数
echo never > /sys/kernel/mm/transparent_hugepage/defrag  #建议写到开启启动新里
sysctl -w vm.swappiness=0  # 建议写进sysctl.conf
 
cat >>/etc/sysct <<EOF
vm.swappiness=0
EOF
 
sysctl -p
 
# 设置ntp同步服务器时间
 
yum install ntp -y
 
Edit the /etc/ntp.conf file to add NTP servers, as in the following example.
server 0.cn.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org
 
sudo systemctl start ntpd
sudo systemctl enable ntpd
 
 
#for centos6
*/2 * * * * /usr/sbin/ntpdate 0.cn.pool.ntp.org >> /dev/null 2>&1
 
 
 
https://www.cloudera.com/documentation/enterprise/6/6.2/topics/install_cdh_disable_iptables.html#install_cdh_disable_iptables
 
 
 
 
 
Cloudera安装步骤
参考:https://www.cloudera.com/documentation/enterprise/latest/topics/installation_installation.html
 
阶段1:安装JDK(忽略)
[Java SE 8 Downloads](http://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html)
export JAVA_HOME=/usr/java/jdk.1.8.0_nn
# java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
# 注 此处不用安装JDK,因为CM源有封装好的jdk,
 
阶段2:设置数据库
scp ~/rpm/rpmbuild2019/mysql20190506/RPMS/x86_64/ops_mysql-5.7.18-1.0.x86_64.rpm  root@192.168.88.40:~
 
# 使用mysql数据库,提前安装好mysql
# mysql -u root -ppassword -e "create database cmf DEFAULT CHARACTER SET utf8;"
# mysql -u root -ppassword -e "GRANT ALL PRIVILEGES ON `cmf`.* TO 'cmf'@'localhost' IDENTIFIED BY '224088' ";"
 
 
 
阶段3:安装Cloudera Manager服务器
 
dns client 改为 8.8.4.4
# 配置cloudera-cdh源和cloudera-manager源
# cloudera-manager
cd /etc/yum.repos.d/
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/cloudera-manager.repo
 
# cloudera-cdh
wget https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/cloudera-cdh5.repo
 
# 安装jdk和cloudera-manager
sudo yum install oracle-j2sdk1.7 -y
 
 
 
sudo yum install cloudera-manager-daemons cloudera-manager-server -y
 
 
 
sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
 
 
axel https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
 
 
axel  https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/cloudera-manager-daemons-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm
 
aria2c  or 迅雷 
axel https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/cloudera-manager-server-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm
 
 
 
阶段4:启动CM服务并通过浏览器访问
systemctl start cloudera-scm-server
 
#有问题
tailf /var/log/cloudera-scm-server/cloudera-scm-server.log
 
</pre>
 
 
=troubleshooting=
<pre>
安装这个就行了
mysql-connector-java
 
 
安装MySQL JDBC Driver
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
tar zxvf mysql-connector-java-5.1.46.tar.gz  && mkdir -p /usr/share/java/
cd mysql-connector-java-5.1.46 && cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar
 
 
下载 mysql-connector-java-8.0.11-1.el7.noarch.rpm
https://dev.mysql.com/downloads/connector/j/
</pre>
 
 
 
[https://www.cnblogs.com/spec-dog/p/4664944.html 启动CM出现 “JDBC Driver class not found: com.mysql.jdbc.Driver” 以及“Error creating bean with name 'serverLogFetcherImpl'”问题的解决方法]
 
[https://blog.csdn.net/hchhan/article/details/81106992 mysql-connector-java与mysql版本的对应]
 
 
[https://blog.csdn.net/zhang5476499/article/details/53791710 centos7安装mysql并jdbc测试]
 
 
 
 
 
[https://blog.csdn.net/baidu_36649389/article/details/53217196 centos下JDBC的安装配置]
 
https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-versions.html


network--> 关掉 http3 听说会对国内更加友好
=see also=
=see also=
==normal ins==
[https://www.jianshu.com/p/f804bd587d95 Centos7.6部署CDH6.1.0大数据平台教程]
[https://blog.csdn.net/wh211212/article/details/78743191 Cloudera(CDH) 简介和在线安装]
[https://blog.csdn.net/lxbalex/article/details/83188753 从零搭建生产Hadoop集群(三)——CDH集群搭建]
[https://blog.csdn.net/weixin_39182877/article/details/79062988 CDH5-Cloudera-Server部署和启动]
[https://blog.csdn.net/qq_23660243/article/details/60870527 CDH在Centos 7离线安装]
[https://blog.csdn.net/feixeyes/article/details/38084403 适合国内网速的CDH5安装]
[https://blog.csdn.net/liangzelei/article/details/80359461 Hadoop cdh版本搭建]
==docker ins==
[https://www.jianshu.com/p/f98741f3f8fa docker安装cloudera]


[https://www.jianshu.com/p/2ecec335acbd centos安装Docker CDH+kafka]


[https://support.cloudflare.com/hc/zh-cn/articles/360000841472  通过自动化将多个站点添加到 Cloudflare ]


[https://zhuanlan.zhihu.com/p/29891330 CloudFlare免费CDN加速使用方法]


==learn==
https://support.cloudflare.com/hc/zh-cn/articles/360021801151-%E5%BC%80%E5%A7%8B%E4%BD%BF%E7%94%A8-Cloudflare-DNS
[https://www.zhihu.com/question/19795366 零基础学习 Hadoop 该如何下手]


[https://www.jianshu.com/p/7f38989ffa15 CloudFlare 节点优化——从入门到放弃]


[https://www.zhihu.com/question/57496274 用 cloudflare 的 CDN 比不用还慢,有没有其他的建议?]


[https://blog.csdn.net/yuan_xw/article/details/50003197 Hadoop教程(一) Hadoop入门教程]
[https://zhuanlan.zhihu.com/p/82909515 cloudflare使用入门教程,国外最好免费CDN]


https://blog.csdn.net/wh211212/article/details/78743191


[[category:ops]] [[category:bigdata]]
[https://www.iplaysoft.com/p/cloudflare-dns 1.1.1.1 好记牛逼 - Cloudflare 推出全球平均速度最快的 DNS!]
[[category:ops]]

2023年5月11日 (四) 06:55的版本