页面“Redis5 cluster”与“Redis批量删除key”之间的差异

来自linux中国网wiki
(页面间的差异)
跳到导航 跳到搜索
 
 
第1行: 第1行:
=优化=
+
[[category:ops]] [[category:redis]] 
==remove RDB固化==
+
 
#dbfilename dump_7000.rdb
+
[[Redis-bigkeys]]
=pre 常用脚本=
+
=redis cluster 批量删除key=
 
<pre>
 
<pre>
 
+
#June 19 2020  有空结果一下原来的版本再优化一下
 
+
cat redis_del_key.sh
cat /data/evan/redisloop.sh
 
 
#!/bin/bash
 
#!/bin/bash
#usage  redisloop.sh  rr
+
# Usage
cat  $1  | while read line
+
#redis_list=("host:post" "host:post")
do
 
 
 
redis-cli    -c  -h    172.31.22.218  -p 7000  -a hell $line
 
redis-cli    -c  -h    172.31.22.218  -p 7001  -a hell  $line
 
redis-cli    -c  -h  172.31.16.135  -p 7002  -a hell  $line 
 
redis-cli    -c  -h  172.31.16.135  -p 7003  -a hell  $line
 
redis-cli    -c  -h  172.31.23.43  -p 7004  -a hell  $line
 
redis-cli    -c  -h  172.31.23.43  -p 7005  -a hell  $line
 
done
 
 
 
 
 
vi rr
 
del hello_coin_limit_045617153_2
 
del hello_coin_limit_9811460_2
 
 
 
 
 
 
 
 
 
 
 
cat  redisin.sh
 
#!/bin/bash
 
redis-cli    -c  -h    172.31.22.218  -p 7000  -a hello  `cat $1`
 
redis-cli    -c  -h    172.31.22.218  -p 7001  -a hello  `cat $1`
 
#
 
redis-cli    -c  -h  172.31.16.135  -p 7002  -a hello  `cat $1` #$1 # del hello_coin_list
 
redis-cli    -c  -h  172.31.16.135  -p 7003  -a hello `cat $1` #$1 #del hello_coin_list
 
#
 
redis-cli    -c  -h  172.31.23.43  -p 7004  -a hello  `cat $1`#$1 #del hello_coin_list
 
redis-cli    -c  -h  172.31.23.43  -p 7005  -a hello  `cat $1` #$1 #del hello_coin_list
 
 
 
</pre>
 
 
 
=docker-compose=
 
 
 
==host 网络 要手工添加集群==
 
同样要用firewall etc
 
要优化的地方    restart  已添加 
 
  
 +
redis_list=( "172.16.200.12:7003" "172.16.200.7:7004" "172.16.200.7:7005" "172.16.200.13:7000" "172.16.200.13:7001" "172.16.200.12:7002")
 +
#redis_list=( "172.16.200.13:7000" "172.16.200.13:7001" "172.16.200.12:7002")
 +
#redis_list=("172.16.200.7:7004")
 +
password="3636password"
  
 +
for info in ${redis_list[@]}
 +
    do
 +
        echo "开始执行:$info" 
 +
        ip=`echo $info | cut -d : -f 1`
 +
        port=`echo $info | cut -d : -f 2`
  
相关脚本和配置
+
      for  loop in  `cat key.txt`
https://github.com/evan886/redis/tree/master/redis-cluster-docker_host
+
        do
<pre>
+
        #echo $loop  |xargs -t -n1 redis-cli -c  -h $ip -p $port -a $password -c del
Usage:
+
        echo $loop  |xargs -t -n1 redis-cli -c  -h $ip -p $port -a $password -c keys
NOTE2020
+
        done
docker-compose up -d
 
  
bind  172.18.178.218 127.0.0.1
+
    done
 +
    echo "success done完成"
  
aliyu 不能 绑定外网NAT IP  bind 内网IP 然后 创建集群用 外网IP就可以了
+
# cat key.txt
 +
infnce:wonderfulCache
 +
ext_info
  
不然后
 
(error) CLUSTERDOWN Hash slot not served  cluster没起来
 
  
#docker exec -it  node-7000  redis-cli -p 7000  --cluster create  10.25.196.78:7000    10.25.196.78:7001  10.25.196.78:7002  10.25.196.78:7003  10.25.196.78:7004  10.25.196.78:7005  --cluster-replicas 1
 
  
IP='192.168.10.193'
+
#原来的版本
docker exec -it  node-7000  redis-cli -p 7000  --cluster create  $IP:7000    $IP:7001  $IP:7002  $IP:7003  $IP:7004  $IP:7005  --cluster-replicas 1
 
  
 +
del hello_user_*
  
#不要交互
+
redis-cli keys
echo 'masterauth passwd123d
 
requirepass passwd123d' >>redis-7000.conf
 
  
  
 +
redis-cli keys "hello_user_*" | xargs redis-cli del
  
  
 +
./redis-cli -h IP -p PORT -a PASSWORD keys 'key*' | xargs  ./redis-cli -h IP  -p PORT -a PASSWORD del
  
 +
redis-cli    -c  -h    172.31.22.218  -p 7000  -a pass keys 'hello_user_*'| xargs -i redis-cli    -c  -h    172.31.22.218  -p 7000  -a pass  del {}
 +
redis-cli    -c  -h    172.31.22.218  -p 7001  -a pass keys 'hello_user_*'| xargs -i redis-cli    -c  -h    172.31.22.218  -p 7001  -a pass  del {}
  
 +
redis-cli    -c  -h    172.31.16.135  -p 7002  -a pass keys 'hello_user_*'| xargs -i  redis-cli    -c  -h    172.31.16.135  -p 7002 -a pass  del {}
 +
redis-cli    -c  -h    172.31.16.135  -p 7003  -a pass keys 'hello_user_*'| xargs  -i redis-cli    -c  -h    172.31.16.135  -p 7003 -a pass  del {}
  
 +
redis-cli    -c  -h    172.31.23.43  -p 7004  -a pass keys 'hello_user_*'| xargs -i  redis-cli    -c  -h    172.31.23.43  -p 7004 -a pass  del {}
 +
redis-cli    -c  -h    172.31.23.43  -p 7005  -a pass keys 'hello_user_*'| xargs -i  redis-cli    -c  -h    172.31.23.43  -p 7005 -a pass  del {}
  
bind 要记得改为你要的IP  如果用0.0.0.0 那就不要127.0.0.1
 
有问题多看log 目录
 
  
#这里是母机的ip 如果是三台机器 那么就有 三个 IP
 
docker exec -it  node-7000  redis-cli -p 7000  --cluster create  ip:7000    ip:7001  ip:7002  ip:7003  ip:7004  ip:7005  --cluster-replicas 1
 
 
docker exec -it  node-7000  redis-cli -p 7000  --cluster create --cluster-replicas 1 ip:7000    ip:7001  ip:7002  ip:7003  ip:7004  ip:7005
 
 
三台母机的情形 就是每个docker-compose.yml 只启动两个 ,其它的删除就行了  201912
 
 
</pre>
 
== 数据损坏重做 2020==
 
<pre>
 
内网的193机器  PHP老是搞坏,或者强行restart os  常常会有几个节点起不来 这时 只能重新做 cluster
 
 
data 目录下的mv 走或删除掉
 
conf 目录下的 先去掉密码 
 
up -d 起所有节点
 
conf 用追加再添加回密码
 
docker-compose restart 起回所有的节点
 
 
 
 
</pre>
 
 
==laradock方法==
 
===相关文件===
 
<pre>
 
 
#注意  网络是这个, 还有就是多台机器之间 也不太可能能这个网络 那样 容器之间的redis 是不通的
 
NETWORKS_DRIVER=bridge
 
 
less  docker-compose.yml
 
### Redis Cluster evan##########################################
 
    redis-cluster:
 
      build: ./redis-cluster
 
      ports:
 
        - "${REDIS_CLUSTER_PORT_RANGE}:7000-7005"
 
      networks:
 
        - backend
 
 
 
cat  redis-cluster/Dockerfile  #注意 redis版本
 
FROM grokzen/redis-cluster:5.0.6
 
#FROM grokzen/redis-cluster:latest
 
LABEL maintainer="hareku <[email protected]>"
 
</pre>
 
 
=== 添加密码===
 
<pre>
 
添加密码 可以进入容器里面直接添加 ,不过不够好 ,有空要 放到母机出来,有如我上面
 
004  7005  redis-cluster.tmpl redis.tmpl  sentinel.tmpl
 
root@401a49002b88:/data# ls /redis-conf/700
 
7000/ 7001/ 7002/ 7003/ 7004/ 7005/
 
root@401a49002b88:/data# ls /redis-conf/7000/redis.conf
 
/redis-conf/7000/redis.conf
 
root@401a49002b88:/data# tail /redis-conf/7000/redis.conf
 
bind 0.0.0.0
 
port 7000
 
cluster-enabled yes
 
cluster-config-file nodes.conf
 
cluster-node-timeout 5000
 
appendonly yes
 
dir /redis-data/7000
 
 
 
进入redis-cluster 容器 直接非交互  这里以7005 端口为例子
 
 
redis-cli  -c  -h 127.0.0.1  -p 7005  config set masterauth 36DbRGvv66
 
redis-cli  -c  -h 127.0.0.1  -p 7005  config set requirepass 36DbRGvv66
 
redis-cli  -c  -h 127.0.0.1  -p 7005 auth  36DbRGvv66
 
 
redis-cli  -c  -h 127.0.0.1  -p 7005  -a 36DbRGvv66  config rewrite
 
redis-cli  -c  -h 127.0.0.1  -p 7005  config rewrite  # 最后这一步 有时得 加上密码
 
 
 
 
Usage:  可以直接用物理机的ip  内网或者外网
 
 
上次搞了几天 ,是忙晕了 不记得打开防火墙 7000-7005的端口
 
</pre>
 
 
=源码手工=
 
==单独安装redis-cli==
 
<pre>
 
debian 上可以单独安装 redis-cli
 
但是rhel居然不行 只能编译一下 
 
 
三步曲后 make  -j  2 之后 ,copy 走这个 bin 文件就行了
 
./src/redis-cli -c  /usr/bin/  #或者放 /usr/sbin  只有超级用户可以用到  如果其它用户要用 就不要放这里了
 
 
 
redis cluster 执行脚本 不用老是一个一个的搞 麻烦  其实只要在三个主上执行就行了 你6个全执行也没事,反正是脚本的
 
  
 
#!/bin/bash
 
#!/bin/bash
##Usage  bash redisin.sh  你的redis脚本
+
redis_comm=/usr/local/redis-5.0.3/bin/redis-cli
redis-cli    -c  -h    172.31.22.218  -p 7000  -a hello  `cat $1`
+
redis_ser01=192.168.50.175
redis-cli    -c  -h    172.31.22.218  -p 7001  -a hello  `cat $1`
+
redis_ser02=192.168.50.174
#
+
$redis_comm -c -h $redis_ser01 -p 7001 -a xxx keys $1 | xargs -i $redis_comm -c -h $redis_ser01 -p 7001 -a xxx del {}
redis-cli    -c  -h  172.31.16.135  -p 7002  -a hello  `cat $1` #$1 # del hello_list
+
$redis_comm -c -h $redis_ser02 -p 7001 -a xxx keys $1 | xargs -i $redis_comm -c -h $redis_ser02 -p 7001 -a xxx del {}
 
 
 
 
</pre>
 
 
 
==节点信息==
 
<pre>
 
角色                        主机名                                                            IP1              IP2     
 
 
 
7000  7001              01                  192.168.10.211   
 
7002  7003              02                  192.168.10.212     
 
7004  7005              03                  192.168.10.213     
 
</pre>
 
 
 
姐妹篇 [[Redis4 cluster]]
 
 
 
==防火墙设置==
 
<pre>
 
第一台:
 
firewall-cmd --permanent --add-port=7000/tcp
 
firewall-cmd --permanent --add-port=17000/tcp
 
firewall-cmd --permanent --add-port=7001/tcp
 
firewall-cmd --permanent --add-port=17001/tcp
 
firewall-cmd --reload
 
firewall-cmd --list-all-zones
 
第二台:
 
firewall-cmd --permanent --add-port=7002/tcp
 
firewall-cmd --permanent --add-port=17002/tcp
 
firewall-cmd --permanent --add-port=7003/tcp
 
firewall-cmd --permanent --add-port=17003/tcp
 
firewall-cmd --reload
 
firewall-cmd --list-all-zones
 
第三台:
 
firewall-cmd --permanent --add-port=7004/tcp
 
firewall-cmd --permanent --add-port=17004/tcp
 
firewall-cmd --permanent --add-port=7005/tcp
 
firewall-cmd --permanent --add-port=17005/tcp
 
firewall-cmd --reload
 
firewall-cmd --list-all-zones
 
</pre>
 
 
 
==系统参数设置==
 
<pre>#vi /etc/sysctl.conf
 
echo 'vm.overcommit_memory = 1
 
net.core.somaxconn = 1024' >>/etc/sysctl.conf
 
 
 
/sbin/sysctl -p /etc/sysctl.conf
 
</pre>
 
 
 
==redis 5 安装==
 
<pre>
 
#bash redisins.sh 2>&1 | tee redislog
 
 
 
wget -c http://download.redis.io/releases/redis-5.0.6.tar.gz
 
#set var
 
redis_install_dir=/data/apps/redis
 
redis_ver=5.0.6
 
 
 
THREAD=4
 
tar xzf redis-${redis_ver}.tar.gz
 
pushd redis-${redis_ver} > /dev/null
 
 
 
make  -j ${THREAD}
 
 
 
make PREFIX=${redis_install_dir}
 
make distclean
 
#cd src && make install #这个才行 特别是cluster ? 2019年 8月16日 星期五 23时29分41秒 CST
 
make install  PREFIX=${redis_install_dir} 
 
echo "export PATH=${redis_install_dir}/bin/:$PATH" >> /etc/profile && source /etc/profile
 
 
 
####**********************下面是老的
 
wget -c http://download.redis.io/releases/redis-5.0.5.tar.gz
 
redis_install_dir=/home/apps/redis
 
redis_ver=5.0.5
 
 
 
THREAD=4
 
tar xzf redis-${redis_ver}.tar.gz
 
pushd redis-${redis_ver} > /dev/null
 
make  -j ${THREAD}
 
make PREFIX=/home/apps/redis #here
 
make distclean
 
#cd src && make install #这个才行 特别是cluster ? 2019年 8月16日 星期五 23时29分41秒 CST
 
make install  PREFIX=/home/apps/redis  # make install #这个有时没有创建出目录来呀
 
 
 
#这个我没做 根本没用 哈哈
 
#复制rb 或者放到安装的bin目录 然后再加path
 
#cp /opt/redis-5.0.0/src/redis-trib.rb /usr/local/bin
 
 
 
echo 'export PATH=/home/apps/redis/bin/:$PATH' >> /etc/profile && source /etc/profile
 
##**********************上面是老的
 
NOTE
 
redis5 用redis-cli创建整个redis集群(redis5以前的版本集群是依靠ruby脚本redis-trib.rb实现
 
</pre>
 
 
 
==创建节点 ==
 
<pre>
 
#第一台:192.168.10.211 #here
 
redis_install_dir=/data/apps/redis
 
  mkdir -p ${redis_install_dir}
 
cd ${redis_install_dir}
 
mkdir -p redis_cluster
 
cd redis_cluster
 
mkdir data 7000 7001
 
 
 
#cp /root/src/redis-5.0.5/redis.conf ${redis_install_dir}/redis_cluster
 
#cd  /home/apps/redis/redis_cluster
 
#cp redis.conf 7000/redis_7000.conf
 
#mv redis.conf  7001/redis_7001.conf
 
 
 
 
 
#第2台:192.168.10.212
 
redis_install_dir=/data/apps/redis
 
mkdir -p ${redis_install_dir}
 
cd ${redis_install_dir}
 
mkdir -p redis_cluster
 
cd redis_cluster
 
mkdir data 7002 7003
 
 
 
#cp /root/src/redis-5.0.5/redis.conf  .
 
#cp  redis.conf 7002/redis_7002.conf
 
#mv redis.conf  7003/redis_7003.conf
 
 
 
 
 
#第3台:192.168.10.213
 
redis_install_dir=/data/apps/redis
 
mkdir -p ${redis_install_dir}
 
cd ${redis_install_dir}
 
mkdir -p redis_cluster
 
cd redis_cluster
 
mkdir data 7004 7005
 
#cp /root/src/redis-5.0.5/redis.conf  .
 
#cp redis.conf 7004/redis_7004.conf
 
#mv redis.conf 7005/redis_7005.conf
 
 
 
 
 
</pre>
 
 
 
==修改配置文件(三台机器6个配置)==
 
<pre>
 
note  日志在系统那  因为我们配置文件没指定 /var/log/redis/
 
 
 
ls /home/apps/redis/redis_cluster/data/
 
appendonly-7000.aof  dump_7000.rdb  nodes_7000.conf
 
appendonly-7001.aof  dump_7001.rdb  nodes_7001.conf
 
# 生成在这里了  /home/apps/redis/redis_cluster/data/nodes_7000.conf  nodes_7001.conf
 
 
 
配置建议加上这个  下面这些配置还加上的
 
# appendfsync always
 
appendfsync everysec
 
# appendfsync no
 
 
 
第一台:
 
  pushd /home/apps/redis/redis_cluster
 
mkdir -p /var/log/redis
 
: > 7000/redis_7000.conf
 
#vi 7000/redis_7000.conf
 
#非交互进行
 
echo 'port 7000
 
bind 192.168.10.211 127.0.0.1
 
cluster-enabled yes
 
appendfilename "appendonly-7000.aof"
 
cluster-config-file nodes_7000.conf
 
cluster-node-timeout 15000
 
appendonly yes
 
appendfsync everysec
 
daemonize yes
 
dbfilename dump_7000.rdb
 
dir /data/apps/redis/redis_cluster/data
 
logfile "/var/log/redis/7000.log"
 
protected-mode yes
 
pidfile /var/run/redis_7000.pid'>7000/redis_7000.conf
 
 
 
 
 
:>7001/redis_7001.conf
 
echo 'port 7001
 
bind 192.168.10.211 127.0.0.1
 
cluster-enabled yes
 
appendfilename "appendonly-7001.aof"
 
cluster-config-file nodes_7001.conf
 
cluster-node-timeout 15000
 
appendonly yes
 
appendfsync everysec
 
daemonize yes
 
dbfilename dump_7001.rdb
 
dir /home/apps/redis/redis_cluster/data
 
logfile "/var/log/redis/7001.log"
 
protected-mode yes
 
pidfile /var/run/redis_7001.pid'>7001/redis_7001.conf
 
 
 
 
 
第二台
 
pushd /home/apps/redis/redis_cluster
 
mkdir -p /var/log/redis
 
:>7002/redis_7002.conf
 
echo 'port 7002
 
bind 192.168.10.212 127.0.0.1
 
cluster-enabled yes
 
appendfilename "appendonly-7002.aof"
 
cluster-config-file nodes_7002.conf
 
cluster-node-timeout 15000
 
appendonly yes
 
appendfsync everysec
 
daemonize yes
 
dbfilename dump_7002.rdb
 
dir /home/apps/redis/redis_cluster/data
 
logfile "/var/log/redis/7002.log"
 
protected-mode yes
 
pidfile /var/run/redis_7002.pid' >7002/redis_7002.conf
 
 
 
:>7003/redis_7003.conf
 
echo 'port 7003
 
bind 192.168.10.212 127.0.0.1
 
cluster-enabled yes
 
appendfilename "appendonly-7003.aof"
 
cluster-config-file nodes_7003.conf
 
cluster-node-timeout 15000
 
appendonly yes
 
appendfsync everysec
 
daemonize yes
 
dbfilename dump_7003.rdb
 
dir /home/apps/redis/redis_cluster/data
 
logfile "/var/log/redis/7003.log"
 
protected-mode yes
 
pidfile /var/run/redis_7003.pid'>7003/redis_7003.conf
 
 
 
 
 
第3台
 
pushd /home/apps/redis/redis_cluster
 
mkdir -p /var/log/redis
 
:>7004/redis_7004.conf
 
echo 'port 7004
 
bind 192.168.10.213 127.0.0.1
 
cluster-enabled yes
 
appendfilename "appendonly-7004.aof"
 
cluster-config-file nodes_7004.conf
 
cluster-node-timeout 15000
 
appendonly yes
 
appendfsync everysec
 
daemonize yes
 
dbfilename dump_7004.rdb
 
dir /home/apps/redis/redis_cluster/data
 
logfile "/var/log/redis/7004.log"
 
protected-mode yes
 
pidfile /var/run/redis_7004.pid ' > 7004/redis_7004.conf
 
 
 
:>7005/redis_7005.conf
 
echo 'port 7005
 
bind 192.168.10.213 127.0.0.1
 
cluster-enabled yes
 
appendfilename "appendonly-7005.aof"
 
cluster-config-file nodes_7005.conf
 
cluster-node-timeout 15000
 
appendonly yes
 
daemonize yes
 
dbfilename dump_7005.rdb
 
dir /home/apps/redis/redis_cluster/data
 
logfile "/var/log/redis/7005.log"
 
protected-mode yes
 
pidfile /var/run/redis_7005.pid' >7005/redis_7005.conf
 
 
 
</pre>
 
 
 
==启动redis==
 
<pre>
 
#cd /usr/local/redis/etc/redis_cluster #on freebsd
 
第一台: #注意用 "" 不要'' 要替换变量的值
 
mkdir /var/log/redis
 
redis_install_dir=/data/apps/redis
 
echo  "redis_install_dir=/data/apps/redis
 
${redis_install_dir}/bin/redis-server ${redis_install_dir}/redis_cluster/7000/redis_7000.conf
 
${redis_install_dir}/bin/redis-server ${redis_install_dir}/redis_cluster/7001/redis_7001.conf">${redis_install_dir}/redis_cluster/run
 
 
 
chmod +x ${redis_install_dir}/redis_cluster/run && bash ${redis_install_dir}/redis_cluster/run
 
chmod +x /etc/rc.d/rc.local
 
echo "${redis_install_dir}/redis_cluster/run" > >/etc/rc.d/rc.local #把变量替换为值,不建议用''
 
 
 
第二台
 
mkdir /var/log/redis
 
redis_install_dir=/data/apps/redis
 
echo "${redis_install_dir}/bin/redis-server ${redis_install_dir}/redis_cluster/7002/redis_7002.conf
 
${redis_install_dir}/bin/redis-server ${redis_install_dir}/redis_cluster/7003/redis_7003.conf">${redis_install_dir}/redis_cluster/run
 
 
 
chmod +x ${redis_install_dir}/redis_cluster/run && bash ${redis_install_dir}/redis_cluster/run
 
chmod +x /etc/rc.d/rc.local
 
echo "${redis_install_dir}/redis_cluster/run" > >/etc/rc.d/rc.local #把变量替换为值
 
 
 
 
 
第三台:
 
mkdir /var/log/redis
 
redis_install_dir=/data/apps/redis
 
echo "${redis_install_dir}/bin/redis-server ${redis_install_dir}/redis_cluster/7004/redis_7004.conf
 
${redis_install_dir}/bin/redis-server ${redis_install_dir}/redis_cluster/7005/redis_7005.conf">${redis_install_dir}/redis_cluster/run
 
 
 
chmod +x ${redis_install_dir}/redis_cluster/run && bash ${redis_install_dir}/redis_cluster/run
 
chmod  +x /etc/rc.d/rc.local
 
echo "${redis_install_dir}/redis_cluster/run"  >>/etc/rc.d/rc.local #把变量替换为值
 
 
 
</pre>
 
 
 
=登陆redis=
 
<pre>
 
/home/apps/redis/bin/redis-cli  -c -h  192.168.10.211 -p 7000
 
redis-cli -h 10.3.10.141 -p 7001
 
 
 
redis-cli -h 10.3.10.142 -p 7002
 
redis-cli -h 10.3.10.142 -p 7003
 
  
redis-cli -h 10.3.10.143 -p 7004
+
运行:
redis-cli -h 10.3.10.143 -p 7005
 
</pre>
 
==创建集群==
 
<pre>注意:在任意一台上运行 不要在每台机器上都运行,一台就够了
 
#要开通端口  firewalld
 
#在geany 有: 格式可能有点不对 要注意
 
redis-cli --cluster create --cluster-replicas 1 192.168.10.211:7000 192.168.10.211:7001 192.168.10.212:7002 192.168.10.212:7003 192.168.10.213:7004  192.168.10.213:7005
 
#--replicas  1  表示 自动为每一个master节点分配一个slave节点    上面有6个节点,程序会按照一定规则生成 3个master(主)3个slave(从)
 
</pre>
 
===执行详情===
 
<pre>
 
[root@dev-hello-1 redis_cluster]# redis-cli --cluster create --cluster-replicas 1 192.168.10.201:7000 192.168.10.201:7001 192.168.10.202:7002 192.168.10.202:7003 192.168.10.203:7004  192.168.10.203:7005
 
>>> Performing hash slots allocation on 6 nodes...
 
Master[0] -> Slots 0 - 5460
 
Master[1] -> Slots 5461 - 10922
 
Master[2] -> Slots 10923 - 16383
 
Adding replica 192.168.10.202:7003 to 192.168.10.201:7000
 
Adding replica 192.168.10.203:7005 to 192.168.10.202:7002
 
Adding replica 192.168.10.201:7001 to 192.168.10.203:7004
 
M: caf3a2c83f4f34498d9152161d3d62ca499094eb 192.168.10.201:7000
 
  slots:[0-5460] (5461 slots) master
 
S: ecbc0c648ff15c4ff2c176563afa580c7c8b7f3b 192.168.10.201:7001
 
  replicates 3792aae886d957e1a0272bcd198a06798f05c26d
 
M: 9c2f0b7827180d93371b493e8ca59b6eea776b0c 192.168.10.202:7002
 
  slots:[5461-10922] (5462 slots) master
 
S: b66f09b38e321ce3f8ee000b7cb40fe804f3a450 192.168.10.202:7003
 
  replicates caf3a2c83f4f34498d9152161d3d62ca499094eb
 
M: 3792aae886d957e1a0272bcd198a06798f05c26d 192.168.10.203:7004
 
  slots:[10923-16383] (5461 slots) master
 
S: a027592b56eb7477fe039786490318efdda8a0d6 192.168.10.203:7005
 
  replicates 9c2f0b7827180d93371b493e8ca59b6eea776b0c
 
Can I set the above configuration? (type 'yes' to accept): yes
 
>>> Nodes configuration updated
 
>>> Assign a different config epoch to each node
 
>>> Sending CLUSTER MEET messages to join the cluster
 
Waiting for the cluster to join
 
....
 
>>> Performing Cluster Check (using node 192.168.10.201:7000)
 
M: caf3a2c83f4f34498d9152161d3d62ca499094eb 192.168.10.201:7000
 
  slots:[0-5460] (5461 slots) master
 
  1 additional replica(s)
 
S: a027592b56eb7477fe039786490318efdda8a0d6 192.168.10.203:7005
 
  slots: (0 slots) slave
 
  replicates 9c2f0b7827180d93371b493e8ca59b6eea776b0c
 
M: 3792aae886d957e1a0272bcd198a06798f05c26d 192.168.10.203:7004
 
  slots:[10923-16383] (5461 slots) master
 
  1 additional replica(s)
 
M: 9c2f0b7827180d93371b493e8ca59b6eea776b0c 192.168.10.202:7002
 
  slots:[5461-10922] (5462 slots) master
 
  1 additional replica(s)
 
S: ecbc0c648ff15c4ff2c176563afa580c7c8b7f3b 192.168.10.201:7001
 
  slots: (0 slots) slave
 
  replicates 3792aae886d957e1a0272bcd198a06798f05c26d
 
S: b66f09b38e321ce3f8ee000b7cb40fe804f3a450 192.168.10.202:7003
 
  slots: (0 slots) slave
 
  replicates caf3a2c83f4f34498d9152161d3d62ca499094eb
 
[OK] All nodes agree about slots configuration.
 
>>> Check for open slots...
 
>>> Check slots coverage...
 
[OK] All 16384 slots covered.
 
</pre>
 
  
=查看redis集群情况=
+
./redis_del_key.sh  匹配的key*
<pre>
 
redis-cli -h 192.168.10.211 -p 7000 cluster nodes
 
  
a027592b56eb7477fe039786490318efdda8a0d6 192.168.10.203:7005@17005 slave 9c2f0b7827180d93371b493e8ca59b6eea776b0c 0 1567502665000 6 connected
+
err 
3792aae886d957e1a0272bcd198a06798f05c26d 192.168.10.203:7004@17004 master - 0 1567502666171 5 connected 10923-16383
 
9c2f0b7827180d93371b493e8ca59b6eea776b0c 192.168.10.202:7002@17002 master - 0 1567502664166 3 connected 5461-10922
 
ecbc0c648ff15c4ff2c176563afa580c7c8b7f3b 192.168.10.201:7001@17001 slave 3792aae886d957e1a0272bcd198a06798f05c26d 0 1567502664000 5 connected
 
caf3a2c83f4f34498d9152161d3d62ca499094eb 192.168.10.201:7000@17000 myself,master - 0 1567502663000 1 connected 0-5460
 
b66f09b38e321ce3f8ee000b7cb40fe804f3a450 192.168.10.202:7003@17003 slave caf3a2c83f4f34498d9152161d3d62ca499094eb 0 1567502665169 4 connected
 
  
</pre>
+
-i  {} 都是一定要的
==cluster 完整性检查==
 
<pre>
 
  
redis-trib.rb check 10.3.10.140:7000
+
redis-cli -n 0 keys "test*" | xargs redis-cli -n 0 del
redis-trib.rb check 10.3.10.140:7001
+
(error) CROSSSLOT Keys in request don't hash to the same slot
redis-trib.rb check 10.3.10.140:7002
 
redis-trib.rb check 10.3.10.140:7003
 
redis-trib.rb check 10.3.10.140:7004
 
redis-trib.rb check 10.3.10.140:7005
 
  
结果如下  16384个槽都分配到了 节点
 
 
[OK] All nodes agree about slots configuration.
 
>>> Check for open slots...
 
>>> Check slots coverage...
 
[OK] All 16384 slots covered.
 
 
</pre>
 
</pre>
 
==测试 ==
 
<pre>
 
 
 
redis-cli -c -h  192.168.10.201 -p 7000
 
 
[root@dev-hello-1 redis_cluster]# redis-cli -c -h  192.168.10.201 -p 7000
 
192.168.10.201:7000> set evan 36
 
-> Redirected to slot [14924] located at 192.168.10.203:7004
 
OK
 
192.168.10.203:7004>
 
 
[root@dev-hello-2 redis_cluster]# redis-cli -c -h  192.168.10.202 -p 7002
 
192.168.10.202:7002> get evan
 
-> Redirected to slot [14924] located at 192.168.10.203:7004
 
"36"
 
192.168.10.203:7004>
 
 
dev-hello-3 redis_cluster]#  redis-cli -c -h  192.168.10.203  -p 7005
 
192.168.10.203:7005> get evan
 
-> Redirected to slot [14924] located at 192.168.10.203:7004
 
"36"
 
192.168.10.203:7004>
 
 
测试通过了
 
 
 
 
 
#20191113 
 
要加 -a 密码访问  不然老是提示要密码
 
[root@prod-02 ~]# redis-cli  -c -h 172.16.200.12 -p 7002
 
172.16.200.12:7002> auth ci1723
 
OK
 
172.16.200.12:7002> get evan
 
-> Redirected to slot [14924] located at 172.16.200.7:7004
 
(error) NOAUTH Authentication required.
 
172.16.200.7:7004> auth ci1723
 
OK
 
172.16.200.7:7004> get evan
 
"56;"
 
 
 
 
 
</pre>
 
 
=添加redis集群密码=
 
==方式一:修改所有Redis集群中的redis.conf文件加入==
 
<pre>
 
#不要交互
 
echo 'masterauth passwd123d
 
requirepass passwd123d' >>redis-7000.conf
 
 
 
 
# NOTE  还是这个办法好  不然可能有些 没加载到配置文件去呢
 
masterauth passwd123
 
requirepass passwd123
 
说明:这种方式需要重新启动各节点
 
</pre>
 
 
==方式二:进入各个实例进行设置  就是所有节点:不建议==
 
<pre>
 
redis-cli -c -h 10.3.10.141 -p 7001
 
#好像不能一起执行
 
config set masterauth 36DbRGvv
 
config set requirepass 36DbRGvv
 
auth  36DbRGvv
 
config rewrite
 
 
添加redis集群密码 一开始忘记加-c 导致 143 两个都加不上了  redis本身的密码这样就有了
 
 
-a 后 再执行命令就再 auth , 而如果是进入后 auth password 要每次 都再验证一下密码 麻烦
 
 
</pre>
 
 
=开机自己启动=
 
<pre>
 
#非交互
 
redis_install_dir=/data/apps/redis
 
echo "${redis_install_dir}/bin/redis-server ${redis_install_dir}/redis_cluster/7004/redis_7004.conf
 
${redis_install_dir}/bin/redis-server ${redis_install_dir}/redis_cluster/7005/redis_7005.conf">${redis_install_dir}/redis_cluster/run
 
 
chmod +x ${redis_install_dir}/redis_cluster/run && bash ${redis_install_dir}/redis_cluster/run
 
chmod  +x /etc/rc.d/rc.local
 
echo "${redis_install_dir}/redis_cluster/run"  >>/etc/rc.d/rc.local #把变量替换为值
 
 
#交互
 
vi /apps/redis/redis_cluster/run
 
 
/apps/redis/bin/redis-server /home/apps/redis/redis_cluster/7004/redis_7004.conf
 
/apps/redis/bin/redis-server /home/apps/redis/redis_cluster/7005/redis_7005.conf
 
 
chmod  +x  /apps/redis/redis_cluster/run
 
chmod +x /etc/rc.d/rc.local
 
 
vi  /etc/rc.d/rc.local
 
/apps/redis/redis_cluster/run
 
 
 
</pre>
 
 
=开启关闭=
 
<pre>
 
redis-cli -a xxx -c -h 192.168.0.60 -p 8001 shutdown  #集群cluster_state变成了fail状态?  不过要服务正常的情况下才行
 
 
</pre>
 
 
=troubleshooting=
 
<pre>
 
收到报警
 
 
Labels
 
alertname = vm_host_down
 
instance = redis://10.3.10.141:7000
 
job = redis_cluster
 
 
severity = page
 
Annotations
 
description = redis://10.3.10.141:7000 of job market_redis_cluster has been down for more than 1 minutes..
 
summary = Instance redis://10.3.10.141:7000 down
 
 
 
 
7000 进程在但是 登录不进去
 
 
日志如下
 
03:01:16.166 # Unable to connect to MASTER: No such file or directory
 
22283:S 16 Jun 03:01:17.167 * Connecting to MASTER ?:7003
 
22283:S 16 Jun 03:01:17.175 # Unable to connect to MASTER: No such file or directory
 
22283:S 16 Jun 03:01:18.177 * Connecting to MASTER ?:7003
 
 
 
处理办法 
 
kill -9 7000的 再启动  暂时正常
 
 
 
 
 
 
 
ISCONF Errors writing to the AOF file: No space left on device; nested exception is redis.clients.jedis.exceptions.JedisDataException: MISCONF Errors writing to the AOF file: No space left on device
 
 
org.springframework.dao.InvalidDataAccessApiUsageException: MISCONF Errors writing to the AOF file: No space left on device; nested exception is redis.clients.jedis.exceptions.JedisDataException: MISCONF Errors writing to the AOF file: No space left on device
 
 
原因 有一个台redis的机器 空间100%
 
 
 
 
tail  boot.log
 
[  OK  ] Started Network Time Service.
 
[FAILED] Failed to start /etc/rc.d/rc.local Compatibility.
 
See 'systemctl status rc-local.service' for details.
 
 
#启动rc-local 服务  on aliyun 不要不小心 不清空了  /etc/rc.d/rc.local
 
systemctl enable  rc-local.service
 
 
redis 安装报错 jemalloc/jemalloc.h: No such file or directory
 
 
理上次编译残留文件,重新编译
 
 
make distclean  && make
 
 
导致出现这个错误的原因
 
 
  错误的本质是我们在开始执行make 时遇到了错误(大部分是由于gcc未安装),然后我们安装好了gcc 后,我们再执行make ,这时就出现了jemalloc/jemalloc.h: No such file or directory。这是因为上次的
 
 
 
ERROR: for redis-cluster-8005  Cannot start service redis-cluster-8005: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"/home/redis-cluster-docker_host/conf/redis-8005.conf\\\" to rootfs \\\"/var/lib/docker/overlay2/f80340e6f1abc4aa43cde08ba4c5aa48787b092cbbbb64f01c5f0ad213d71368/merged\\\" at \\\"/var/lib/docker/overlay2/f80340e6f1abc4aa43cde08ba4c5aa48787b092cbbbb64f01c5f0ad213d71368/merged/usr/local/etc/redis/redis.conf\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
 
ERROR: Encountered errors while bringing up the project.
 
[root@stage-secret-im-rabbitmq03 redis-cluster-docker_host]# docker -V
 
^C
 
[root@stage-secret-im-rabbitmq03 redis-cluster-docker_host]# docker -v
 
Docker version 19.03.5, build 633a0ea
 
[root@stage-secret-im-rabbitmq03 redis-cluster-docker_host]# docker-compose  down
 
Removing node-8004 ... done
 
Removing node-8005 ... done
 
Removing node-8003 ... done
 
Removing node-8001 ... done
 
Removing node-8002 ... done
 
Removing node-8006 ... done
 
 
 
 
 
</pre>
 
 
[https://www.bbsmax.com/A/xl5613k1Jr/ redis 安装报错 jemalloc/jemalloc.h: No such file or directory。]
 
  
 
=see also=
 
=see also=
  
  
https://redis.io/topics/cluster-tutorial
+
[https://blog.csdn.net/yangyangye/article/details/100523387  redis集群批量删除模糊key shell脚本]
 
 
http://redisdoc.com/topic/cluster-tutorial.html
 
 
 
[https://blog.csdn.net/truelove12358/article/details/79612954 Redis cluster集群:原理及搭建]
 
 
 
[https://blog.csdn.net/yaomingyang/article/details/79057973 (八)Redis集群常用命令、集群节点新增删除]
 
 
 
[https://www.cnblogs.com/xmxkkk/p/8038190.html redis集群,添加删除槽]
 
 
 
[https://blog.51cto.com/kerry/2316700 redis-5.0 cluster带认证及客户端连接]
 
 
 
[https://www.jianshu.com/p/8045b92fafb2 Redis 5 之后版本的高可用集群搭建]
 
  
[https://blog.csdn.net/zhengzhaoyang122/article/details/94321292 Redis——Cluster集群搭建及节点的添加、删除]
+
[https://www.cnblogs.com/feng0520/p/11067025.html  redis cluster集群批量删除中的key]
  
 +
[https://my.oschina.net/u/1255588/blog/1563672 Redis集群批量删除key]
  
[https://www.cnblogs.com/amei0/p/8177076.html redis(二)集群 redis-cluster & redis主从同步 ]
+
[https://blog.csdn.net/zj20142213/article/details/80879744 redis集群批量删除指定的key]
  
[https://www.cnblogs.com/netonline/p/7875067.html Redis 4集群配置]
+
https://blog.csdn.net/qq_36090419/article/details/80537684
[[category:ops]] [[category:redis]]
 

2020年6月22日 (一) 11:37的版本


Redis-bigkeys

redis cluster 批量删除key

#June 19 2020   有空结果一下原来的版本再优化一下 
cat redis_del_key.sh
#!/bin/bash
# Usage 
#redis_list=("host:post" "host:post")

redis_list=( "172.16.200.12:7003" "172.16.200.7:7004" "172.16.200.7:7005" "172.16.200.13:7000" "172.16.200.13:7001" "172.16.200.12:7002")
#redis_list=( "172.16.200.13:7000" "172.16.200.13:7001" "172.16.200.12:7002")
#redis_list=("172.16.200.7:7004")
password="3636password"

for info in ${redis_list[@]}
    do
        echo "开始执行:$info"  
        ip=`echo $info | cut -d : -f 1`
        port=`echo $info | cut -d : -f 2`

      for  loop in  `cat key.txt`
         do
         #echo $loop  |xargs -t -n1 redis-cli -c  -h $ip -p $port -a $password -c del
         echo $loop  |xargs -t -n1 redis-cli -c  -h $ip -p $port -a $password -c keys 
         done

    done
    echo "success done完成"

# cat key.txt 
infnce:wonderfulCache
ext_info



#原来的版本

del hello_user_* 

redis-cli keys


 redis-cli keys "hello_user_*" | xargs redis-cli del


./redis-cli -h IP -p PORT -a PASSWORD keys 'key*' | xargs  ./redis-cli -h IP  -p PORT -a PASSWORD del

redis-cli    -c  -h    172.31.22.218   -p 7000  -a pass keys 'hello_user_*'| xargs -i redis-cli    -c  -h    172.31.22.218   -p 7000  -a pass  del {}
redis-cli    -c  -h    172.31.22.218   -p 7001  -a pass keys 'hello_user_*'| xargs -i redis-cli    -c  -h    172.31.22.218   -p 7001  -a pass  del {}

redis-cli    -c  -h    172.31.16.135  -p 7002  -a pass keys 'hello_user_*'| xargs -i  redis-cli    -c  -h    172.31.16.135   -p 7002 -a pass  del {}
redis-cli    -c  -h    172.31.16.135  -p 7003  -a pass keys 'hello_user_*'| xargs  -i redis-cli    -c  -h    172.31.16.135   -p 7003 -a pass  del {}

redis-cli    -c  -h     172.31.23.43   -p 7004  -a pass keys 'hello_user_*'| xargs -i  redis-cli    -c  -h    172.31.23.43   -p 7004 -a pass  del {}
redis-cli    -c  -h     172.31.23.43   -p 7005  -a pass keys 'hello_user_*'| xargs -i  redis-cli    -c  -h    172.31.23.43   -p 7005 -a pass  del {}



#!/bin/bash
redis_comm=/usr/local/redis-5.0.3/bin/redis-cli
redis_ser01=192.168.50.175
redis_ser02=192.168.50.174
$redis_comm -c -h $redis_ser01  -p 7001 -a xxx  keys $1 | xargs -i $redis_comm -c -h $redis_ser01  -p 7001 -a xxx del {}
$redis_comm -c -h $redis_ser02  -p 7001 -a xxx  keys $1 | xargs -i $redis_comm -c -h $redis_ser02  -p 7001 -a xxx del {}

运行:

./redis_del_key.sh  匹配的key*

err  

-i  {} 都是一定要的 

redis-cli -n 0 keys "test*" | xargs redis-cli -n 0 del
(error) CROSSSLOT Keys in request don't hash to the same slot

see also

redis集群批量删除模糊key shell脚本

redis cluster集群批量删除中的key

Redis集群批量删除key

redis集群批量删除指定的key

https://blog.csdn.net/qq_36090419/article/details/80537684