Redis4 cluster

来自linux中国网wiki
跳到导航 跳到搜索

高级姐妹篇 Redis5 cluster

Note

参考一下其它项目配置文件有没有优化呢

 可能适合的版本  redis-4.0.11 redis3.x

7、总结:
优点:

在master节点下线后,slave节点会自动提升为master节点,保存集群持续提供服务;
     fail节点恢复后,会自动添加到集群中,变成slave节点;
缺点:

由于redis的复制使用异步机制,在自动故障转移的过程中,集群可能会丢失写命令。然而 redis 几乎是同时执行(将命令恢复发送给客户端,以及将命令复制到slave节点)这两个操作,所以实际中,命令丢失的窗口非常小。
 

ruby ins

How to Install Ruby2.3 on centos7

节点信息

角色                           主机名                                                             IP1               IP2       

7000  7001   prod-cailuw-market-java-01  172.23.11.141      172.23.11.141
7002  7003   prod-cailuw-market-java-02  172.23.11.142      172.23.11.142
7004  7005   prod-cailuw-market-java-03  172.23.11.143      172.23.11.143 


防火墙设置

第一台:
firewall-cmd --permanent --add-port=7000/tcp
firewall-cmd --permanent --add-port=17000/tcp
firewall-cmd --permanent --add-port=7001/tcp
firewall-cmd --permanent --add-port=17001/tcp
firewall-cmd --reload
firewall-cmd --list-all-zones
第二台:
firewall-cmd --permanent --add-port=7002/tcp
firewall-cmd --permanent --add-port=17002/tcp
firewall-cmd --permanent --add-port=7003/tcp
firewall-cmd --permanent --add-port=17003/tcp
firewall-cmd --reload
firewall-cmd --list-all-zones
第三台:
firewall-cmd --permanent --add-port=7004/tcp
firewall-cmd --permanent --add-port=17004/tcp
firewall-cmd --permanent --add-port=7005/tcp
firewall-cmd --permanent --add-port=17005/tcp
firewall-cmd --reload
firewall-cmd --list-all-zones

系统参数设置

vi /etc/sysctl.conf
vm.overcommit_memory = 1
net.core.somaxconn = 1024

/sbin/sysctl -p /etc/sysctl.conf

redis安装

 
wget -c http://download.redis.io/releases/redis-4.0.11.tar.gz
redis_install_dir=/home/apps/redis
redis_ver=4.0.11

THREAD=4
  tar xzf redis-${redis_ver}.tar.gz
  pushd redis-${redis_ver} > /dev/null

  make  -j ${THREAD}

make PREFIX=/home/apps/redis
make distclean
#cd src && make install #这个才行 特别是cluster ? 2019年 8月16日 星期五 23时29分41秒 CST
make install  PREFIX=/home/apps/redis  # make install #这个有时没有创建出目录来呀


复制rb 或者放到安装的bin目录 然后再加path 
cp src/redis-trib.rb /usr/local/bin

echo 'export PATH=/home/apps/redis/bin/:$PATH' >> /etc/profile && source /etc/profile

创建节点

第一台:172.23.11.141 

cd /home/apps/redis #cd /usr/local/redis
mkdir -p redis_cluster
cd redis_cluster
mkdir data 7000 7001
 cp /home/tmp/src/redis-4.0.11/redis.conf  ./    #cp /opt/redis-5.0.0/redis.conf ./
mv redis.conf 7000/redis_7000.conf
cp 7000/redis_7000.conf 7001/redis_7001.conf

第二台:172.23.11.142
cd /home/apps/redis #cd /usr/local/redis
mkdir -p redis_cluster
cd redis_cluster
mkdir data 7002 7003
 cp /home/tmp/src/redis-4.0.11/redis.conf  ./ 
mv redis.conf 7002/redis_7002.conf
cp 7002/redis_7002.conf 7003/redis_7003.conf

第三台:172.23.11.143
cd /home/apps/redis
mkdir -p redis_cluster
cd redis_cluster

mkdir data 7004 7005
cp /home/tmp/redis-4.0.11/redis.conf  .
mv redis.conf 7004/redis_7004.conf
cp 7004/redis_7004.conf 7005/redis_7005.conf


修改配置文件 三台机器6个配置

 
第一台:
mkdir -p /var/log/redis
: > 7000/redis_7000.conf
vi 7000/redis_7000.conf
port 7000
bind 10.3.10.141 172.23.11.141 127.0.0.1
cluster-enabled yes
appendfilename "appendonly-7000.aof"
cluster-config-file nodes_7000.conf 
# 生成在这里了  /home/apps/redis/redis_cluster/data/nodes_7000.conf  nodes_7001.conf
cluster-node-timeout 15000
appendonly yes
daemonize yes
dbfilename dump_7000.rdb
dir /home/apps/redis/redis_cluster/data 
logfile "/var/log/redis/7000.log"
protected-mode yes
pidfile /var/run/redis_7000.pid

:>7001/redis_7001.conf
vi 7001/redis_7001.conf
port 7001
bind 10.3.10.141 172.23.11.141 127.0.0.1
cluster-enabled yes
appendfilename "appendonly-7001.aof"
cluster-config-file nodes_7001.conf
cluster-node-timeout 15000
appendonly yes
daemonize yes
dbfilename dump_7001.rdb
dir /home/apps/redis/redis_cluster/data 
logfile "/var/log/redis/7001.log"
protected-mode yes
pidfile /var/run/redis_7001.pid


第二台:

mkdir -p /var/log/redis
:>7002/redis_7002.conf
vi 7002/redis_7002.conf
port 7002
bind 10.3.10.142  172.23.11.142 127.0.0.1
cluster-enabled yes
appendfilename "appendonly-7002.aof"
cluster-config-file nodes_7002.conf
cluster-node-timeout 15000
appendonly yes
daemonize yes
dbfilename dump_7002.rdb
dir /home/apps/redis/redis_cluster/data 
logfile "/var/log/redis/7002.log"
protected-mode yes
pidfile /var/run/redis_7002.pid

:>7003/redis_7003.conf
vi 7003/redis_7003.conf
port 7003
bind 10.3.10.142  172.23.11.142 127.0.0.1
cluster-enabled yes
appendfilename "appendonly-7003.aof"
cluster-config-file nodes_7003.conf
cluster-node-timeout 15000
appendonly yes
daemonize yes
dbfilename dump_7003.rdb
dir /home/apps/redis/redis_cluster/data 
logfile "/var/log/redis/7003.log"
protected-mode yes
pidfile /var/run/redis_7003.pid


第三台:
mkdir -p /var/log/redis
:>7004/redis_7004.conf
vi 7004/redis_7004.conf

port 7004
bind 10.3.10.143 172.23.11.143 127.0.0.1
cluster-enabled yes
appendfilename "appendonly-7004.aof"
cluster-config-file nodes_7004.conf
cluster-node-timeout 15000
appendonly yes
daemonize yes
dbfilename dump_7004.rdb
dir /home/apps/redis/redis_cluster/data 
logfile "/var/log/redis/7004.log"
protected-mode yes
pidfile /var/run/redis_7004.pid

:>7005/redis_7005.conf
vi 7005/redis_7005.conf

port 7005
bind 10.3.10.143 172.23.11.143 127.0.0.1
cluster-enabled yes
appendfilename "appendonly-7005.aof"
cluster-config-file nodes_7005.conf
cluster-node-timeout 15000
appendonly yes
daemonize yes
dbfilename dump_7005.rdb
dir /home/apps/redis/redis_cluster/data 
logfile "/var/log/redis/7005.log"
protected-mode yes
pidfile /var/run/redis_7005.pid


启动redis

第一台:
mkdir /var/log/redis
mkdir -p  /home/apps/redis/etc/redis_cluster
cd /home/apps/redis/etc/redis_cluster # 没这个路径哦 cd /usr/local/redis/etc/redis_cluster
/home/apps/redis/bin/redis-server /home/apps/redis/redis_cluster/7000/redis_7000.conf
/home/apps/redis/bin/redis-server /home/apps/redis/redis_cluster/7001/redis_7001.conf

第二台:
mkdir /var/log/redis
mkdir -p  /home/apps/redis/etc/redis_cluster
cd /home/apps/redis/etc/redis_cluster # 没这个路径哦 cd /usr/local/redis/etc/redis_cluster
/home/apps/redis/bin/redis-server /home/apps/redis/redis_cluster/7002/redis_7002.conf
/home/apps/redis/bin/redis-server /home/apps/redis/redis_cluster/7003/redis_7003.conf

第三台:
mkdir /var/log/redis
mkdir -p  /home/apps/redis/etc/redis_cluster
cd /home/apps/redis/etc/redis_cluster # 没这个路径哦 cd /usr/local/redis/etc/redis_cluster
/home/apps/redis/bin/redis-server /home/apps/redis/redis_cluster/7004/redis_7004.conf
/home/apps/redis/bin/redis-server /home/apps/redis/redis_cluster/7005/redis_7005.conf

登录redis not cluster mode

 
/home/apps/redis/bin/redis-cli -h 10.3.10.141 -p 7000
redis-cli -h 10.3.10.141 -p 7001

redis-cli -h 10.3.10.142 -p 7002
redis-cli -h 10.3.10.142 -p 7003

redis-cli -h 10.3.10.143 -p 7004
redis-cli -h 10.3.10.143 -p 7005


创建集群 redis-trib.rb

 
Note: 在任意一台上运行 不要在每台机器上都运行,一台就够了 所有节点 相对应端口是通的

 redis-trib.rb create --replicas  1  10.3.10.141:7000   10.3.10.141:7001  10.3.10.142:7002  10.3.10.142:7003  10.3.10.143:7004 10.3.10

 解说 --replicas  1  表示 自动为每一个master节点分配一个slave节点    上面有6个节点,程序会按照一定规则生成 3个master(主)3个slave(从)

执行详情

>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
10.3.10.141:7000
10.3.10.142:7002
10.3.10.143:7004
Adding replica 10.3.10.142:7003 to 10.3.10.141:7000
Adding replica 10.3.10.143:7005 to 10.3.10.142:7002
Adding replica 10.3.10.141:7001 to 10.3.10.143:7004
M: 1d5a18ca1fc1e02e03eabf256d088df29a5c6600 10.3.10.141:7000
   slots:0-5460 (5461 slots) master
S: c19b19cf632c81bcf6e5bee0da6ced1eb70c962f 10.3.10.141:7001
   replicates 4620a17d39c96707f4ffc04ffeba188b5c14a756
M: ae9050a2b8fa9770a3cb6ae55eb1a5e7362193a7 10.3.10.142:7002
   slots:5461-10922 (5462 slots) master
S: c670ed857ca6b41309568df91c21e06c247646ef 10.3.10.142:7003
   replicates 1d5a18ca1fc1e02e03eabf256d088df29a5c6600
M: 4620a17d39c96707f4ffc04ffeba188b5c14a756 10.3.10.143:7004
   slots:10923-16383 (5461 slots) master
S: 4580d628a694b5b35cd417cd745dcd9c15d24b78 10.3.10.143:7005
   replicates ae9050a2b8fa9770a3cb6ae55eb1a5e7362193a7
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join....
>>> Performing Cluster Check (using node 10.3.10.141:7000)
M: 1d5a18ca1fc1e02e03eabf256d088df29a5c6600 10.3.10.141:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: ae9050a2b8fa9770a3cb6ae55eb1a5e7362193a7 10.3.10.142:7002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: c19b19cf632c81bcf6e5bee0da6ced1eb70c962f 10.3.10.141:7001
   slots: (0 slots) slave
   replicates 4620a17d39c96707f4ffc04ffeba188b5c14a756
S: c670ed857ca6b41309568df91c21e06c247646ef 10.3.10.142:7003
   slots: (0 slots) slave
   replicates 1d5a18ca1fc1e02e03eabf256d088df29a5c6600
S: 4580d628a694b5b35cd417cd745dcd9c15d24b78 10.3.10.143:7005
   slots: (0 slots) slave
   replicates ae9050a2b8fa9770a3cb6ae55eb1a5e7362193a7
M: 4620a17d39c96707f4ffc04ffeba188b5c14a756 10.3.10.143:7004
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

查看redis集群情况

redis-cli  -c  -h 10.3.10.142 -p 7003 -a 36
#三个master 三个slave
10.3.10.142:7003> cluster nodes 

4620a17d39c96707f4ffc04ffeba188b5c14a756 10.3.10.143:7004@17004 slave c19b19cf632c81bcf6e5bee0da6ced1eb70c962f 0 1566032672093 8 connected
ae9050a2b8fa9770a3cb6ae55eb1a5e7362193a7 10.3.10.142:7002@17002 master - 0 1566032672000 3 connected 5461-10922
c19b19cf632c81bcf6e5bee0da6ced1eb70c962f 10.3.10.141:7001@17001 master - 0 1566032673094 8 connected 10923-16383
4580d628a694b5b35cd417cd745dcd9c15d24b78 10.3.10.143:7005@17005 slave ae9050a2b8fa9770a3cb6ae55eb1a5e7362193a7 0 1566032671000 6 connected
c670ed857ca6b41309568df91c21e06c247646ef 10.3.10.142:7003@17003 myself,master - 0 1566032670000 7 connected 0-5460
1d5a18ca1fc1e02e03eabf256d088df29a5c6600 10.3.10.141:7000@17000 slave c670ed857ca6b41309568df91c21e06c247646ef 0 1566032672000 7 connected


or
cluster info

cluster 完整性检查

 

redis-trib.rb check 10.3.10.141:7000
redis-trib.rb check 10.3.10.141:7001
redis-trib.rb check 10.3.10.142:7002
redis-trib.rb check 10.3.10.142:7003
redis-trib.rb check 10.3.10.143:7004
redis-trib.rb check 10.3.10.143:7005

结果如下  16384个槽都分配到了 节点 

[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.



验证集群

 

   从任一主机进入任一redis节点,

redis-cli  -c  -h 10.3.10.142 -p 7003 

 写入一个数据:
   set evan  love36

之后进入其他节点,查看redis中的数据,如登录7005节点:
redis-cli  -c  -h 10.3.10.142 -p 7002 

查看redis数据:

 要随便一个节点   你要get 一下就出来 了  不是keys * 切记
10.3.10.142:7002> get evan
-> Redirected to slot [14924] located at 10.3.10.141:7001
"love36"
 


添加redis集群密码

方式一:修改所有Redis集群中的redis.conf文件加入

 
# NOTE   还是这个办法好  不然可能有些 没加载到配置文件去呢 
masterauth passwd123 
requirepass passwd123 
说明:这种方式需要重新启动各节点

方式二:进入各个实例进行设置 就是所有节点:

 
redis-cli -c -h 10.3.10.141 -p 7001
#好像不能一起执行
config set masterauth 36DbRGvv 
config set requirepass 36DbRGvv 
auth  36DbRGvv
config rewrite 

添加redis集群密码 一开始忘记加-c 导致 143 两个都加不上了  redis本身的密码这样就有了

-a 后 再执行命令就再 auth , 而如果是进入后 auth password 要每次 都再验证一下密码 麻烦 

设置密码之后如果需要使用redis-trib.rb的各种命令


如:./redis-trib.rb check 127.0.0.1:7000,则会报错ERR] Sorry, can’t connect to node 127.0.0.1:7000 
解决办法:vim /usr/local/rvm/gems/ruby-2.3.3/gems/redis-4.0.0/lib/redis/client.rb,然后修改passord

class Client
    DEFAULTS = {
      :url => lambda { ENV["REDIS_URL"] },
      :scheme => "redis",
      :host => "127.0.0.1",
      :port => 6379,
      :path => nil,
      :timeout => 5.0,
      :password => "passwd123",
      :db => 0,
      :driver => nil,
      :id => nil,
      :tcp_keepalive => 0,
      :reconnect_attempts => 1,
      :inherit_socket => false
    }

find / -name 'client.rb'


常用命令

关闭redis集群(节点)
有开启自然有关闭,比较麻烦,一般的教程里没写,先cd到redis-cluster/,也可以把以下代码装进脚本如:vim shutdownall.sh,保存后chmod +x shutdownall.sh ,执行的时候./shutdownall.sh即可
eg 
redis01/redis-cli -p 7001 shutdown

redis02/redis-cli -p 7002 shutdown


重启redis集群(节点)
回忆我们创建redis集群的时候是create的cluster,那么重启呢?其实重启更简单!完全无需我们在create一个新的cluster!百度一会了,发现几乎没有解决方案,或者有解决方案的没有技术分享,不多说,直接说结论:

我们创建了集群之后,集群已经存在了,无需创建,仅仅重启各个节点就可以运行了!



(删除原有集群的)重启:
需要执行(形如src/redis-cli -a ds123 --cluster create 10.8.109.24:6379 10.8.109.36:6379 10.8.109.49:6379 10.8.109.49:6380 10.8.109.24:6380 10.8.109.36:6380 --cluster-replicas 1这样的)集群生成指令。

第一步:先关闭各个Redis节点。

第二步:删除各个Redis安装目录下的节点配置文件nodes.conf、数据文件dump.rdb。

说明:此方式相当于重置到刚刚创建集群的时候了,所以需要删除节点配置文件nodes.conf、还需
          要删除数据存储文件dump.rdb。

相关指令:

# 删除节点配置文件

rm nodes.conf dump.rdb

再启动所有节点 再执行集群搭建指令


shooting

t with a MOVED error, like in the following example:

GET x
-MOVED 3999 127.0.0.1:6381
The error includes the hash slot of the key (3999) and the ip:port of the instance that can serve the query. The client needs to reissue the query to the specified node's IP address and port. Note that even if the client waits a long time before reissuing the query, and in the meantime the cluster configuration changed, the destination node will reply again with a MOVED error if the hash slot 3999 is now served by another node. The same happens if the contacted node had no updated information.


redis操作key时出现以下错误

(error) MOVED 5798 127.0.0.1:7001



原因
这种情况一般是因为启动redis-cli时没有设置集群模式所导致。



解决方案
启动时使用-c参数来启动集群模式,命令如下:

redis-cli -c -p 7000


see also

https://redis.io/topics/cluster-tutorial


Redis 5 之后版本的高可用集群搭建

redis安装、测试&集群的搭建&踩过的坑

nice Redis-4.0.11集群配置

CentOs7搭建基于Redis-4.0.2的集群环境之图文详解


redis4.0.8安装并解决ruby2.2.2升级2.3.3问题

Springboot + Redis 4.x.x 搭建集群

Redis 4.0.1集群搭建

CentOs7.3 搭建 Redis-4.0.1 Cluster 集群服务

Redis 4.0集群环境部署

redis.cn Redis 集群教程

Redis Cluster集群

redis 4.0.11 集群搭建

CentOS 7 搭建redis 4.0.11集群


https://redis.io/topics/cluster-tutorial

https://redis.io/topics/cluster-spec

Redis集群的使用测试(Jedis客户端的使用)

Redis集群的关闭、启动、重启等相关指令

redis cluster 集群重新启动关闭

Redis-4.0.11集群配置


redis集群报错,(error) MOVED 15495 127.0.0.1:7003

Redis Cluster 已运行的Redis集群 设置密码 Predis 连接

redis集群密码设置

redis集群设置密码