Rabbitmq cluster on docker-compose

来自linux中国网wiki
Evan（讨论 | 贡献）2020年8月11日 (二) 08:09的版本 （→‎troubleshooting）
跳到导航 跳到搜索

站内资源

Rabbitmq常用命令

docker简易搭建RabbitMQ集群HA-good

ins rabbitmq

info

 适合于 3.610 and 3.7.5 版本
-p 15672:15672 http访问端口
-p 5672:5672 amqp访问端口
-p 4369 #unable to connect to epmd (port 4369) on rabbitmq01: address (cannot connect to host/port)

mq1

Note

有一种是 node1 不加别 , node2  add node1  ; node3  add node1 and node2  但是  rabbitmqctl cluster_status 可能看起来有些node alarms是down的 不过通过浏览器的不会 


mkdir -p /home/data/docker-compose-mqcluster
cd /home/data/docker-compose-mqcluster
##"登录容器请执行:rabbitmq-plugins enable rabbitmq_management 才可>使用WEB管理页面"
vi docker-compose.yml

version: '3.1'  
services:
  rabbitmq:
    image: rabbitmq:3.6.10-management
    ports:
      - "5672:5672"
      - "4369:4369"
      #- "1883:1883"
      - "15672:15672"
      - "25672:25672"

    container_name: rabbitmqCluster01

    hostname: rabbitmq01
    extra_hosts:
      #- "rabbitmq01:192.168.10.211"#不要加上自己，不然会老是重启的
      - "rabbitmq02:192.168.10.212" 
      - "rabbitmq03:192.168.10.213"
    environment:

      RABBITMQ_ERLANG_COOKIE: 'rabbitmqCookie'   
      RABBITMQ_DEFAULT_VHOST: /

      RABBITMQ_DEFAULT_USER: lxtxadmin

      RABBITMQ_DEFAULT_PASS: tSNTTo886Gq

      RABBITMQ_LOGS: /var/lib/rabbitmq/rabbitmq.log

      RABBITMQ_SASL_LOGS: /var/lib/rabbitmq/rabbitmq-sasl.log

    volumes:
      - ./data/rabbitmq:/var/lib/rabbitmq

    restart: always  
# docker-compose  up -d

mq2

mkdir -p /home/data/docker-compose-mqcluster
cd /home/data/docker-compose-mqcluster

vi docker-compose.yml
 
version: '3.1'  
services:
  rabbitmq:
    image: rabbitmq:3.6.10-management

    ports:
      - "5672:5672"
      - "4369:4369"
      #- "1883:1883"
      - "15672:15672"
      - "25672:25672"

    container_name: rabbitmqCluster02
    hostname: rabbitmq02
    extra_hosts:
      - "rabbitmq01:192.168.10.211"
      #- "rabbitmq02:192.168.10.212" #不要加上自己，不然会老是重启的
      - "rabbitmq03:192.168.10.213"
    environment:
      RABBITMQ_ERLANG_COOKIE: 'rabbitmqCookie'   
      RABBITMQ_DEFAULT_VHOST: /
      RABBITMQ_DEFAULT_USER: lxtxadmin
      RABBITMQ_DEFAULT_PASS: tSNTTo886Gq
      RABBITMQ_LOGS: /var/lib/rabbitmq/rabbitmq.log
      RABBITMQ_SASL_LOGS: /var/lib/rabbitmq/rabbitmq-sasl.log
      CLUSTERED: 'true'
      CLUSTER_WITH: rabbit1
      RAM_NODE: 'true'

    volumes:
      - ./data/rabbitmq:/var/lib/rabbitmq
    restart: always

mq3

mkdir -p /home/data/docker-compose-mqcluster
cd /home/data/docker-compose-mqcluster

vi docker-compose.yml

 
version: '3.1'  
services:
  rabbitmq:
    image: rabbitmq:3.6.10-management

    ports:
      - "5672:5672"
      - "4369:4369"
      #- "1883:1883"
      - "15672:15672"
      - "25672:25672"

    container_name: rabbitmqCluster03

    hostname: rabbitmq03
    extra_hosts:
      - "rabbitmq01:10.3.10.141"
      - "rabbitmq02:10.3.10.142"
    environment:

      RABBITMQ_ERLANG_COOKIE: 'rabbitmqCookie'   
      RABBITMQ_DEFAULT_VHOST: /

      RABBITMQ_DEFAULT_USER: lxtxadmin

      RABBITMQ_DEFAULT_PASS: tSNTTo886Gq

      RABBITMQ_LOGS: /var/lib/rabbitmq/rabbitmq.log

      RABBITMQ_SASL_LOGS: /var/lib/rabbitmq/rabbitmq-sasl.log
      CLUSTERED: 'true'
      CLUSTER_WITH: rabbit1
      RAM_NODE: 'true'

    volumes:

      - ./data/rabbitmq:/var/lib/rabbitmq

    restart: always

config

将节点2,3加入集群

#在rabbit2机器进入容器的命令行
sudo docker exec -it rabbitmqCluster02 /bin/bash

#加入集群
rabbitmqctl stop_app  && rabbitmqctl join_cluster rabbit@rabbitmq01  &&rabbitmqctl start_app

rabbit3执行相同的命令
docker exec -it rabbitmqCluster03 /bin/bash
rabbitmqctl stop_app && rabbitmqctl join_cluster rabbit@rabbitmq01  &&rabbitmqctl start_app


查询集群状态
rabbitmqctl cluster_status

nginx 配置


#如果是yum nginx  在放在 conf.d/mq.conf
   server {
        listen       80;
        server_name  mq.com;
 

        location / {
            proxy_pass http://10.3.10.141:15672;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
           
        }
     }


NOTE  selinux  记得关掉  不然可能是502哦 
#SELinux设置
getenforce   #查看SELINUX工作模式

sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config #禁用SELINUX
 setenforce 0 #临时禁用SELINUX，无需重启
 
 
 SELINUX=enforcing
#此项定义selinux状态。
#enforcing—是强制模式系统受selinux保护。就是你违反了策略，你就无法继续操作下去
#permissive—是提示模式系统不会受到selinux保护，只是收到警告信息。permissive就是Selinux有效，但是即使你违反了策略的话它让你继续操作，但是把你的违反的内容记录下来(警告信息)
#disabled—禁用selinux。

镜像模式

开启镜像模式

troubleshooting


1.docker-compose.yml文件报错
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.rabbitmq.environment.CLUSTERED contains true, which is an invalid type, it should be a string, number, or a null


CLUSTERED: true -->CLUSTERED: 'true'


绑hosts的前进 不要全局FQ  SSR常常坑我们 



15672 端口不通 也就是web界面打不开 

问题 镜像用了 rabbitmq 而不是rabbitmq-manager

发现过程 web打不开   15672 端口不能 ，而 5672是通的 进入容器发现是ubuntu 而我记得以前是debian呀 问了下sy才知道 他换了，没全用我的，晕死 ，坑自己人啊

RabbitMQ镜像集群脑裂


现象：
RabbitMQ GUI上显示

Network partition detected
Mnesia reports that this RabbitMQ cluster has experienced a network partition. There is a risk of losing data. Please read RabbitMQ documentation about network partitions and the possible solutions.


节点3 是挂，以节点1 为准  在 node2  node3 上 分别执行
rabbitmqctl stop_app
rabbitmqctl start_app

就行

端口在 进入容器 cannot exec in a stopped state: unknown

今天是端口在 docker restart id 都不行，进程也还在的 重启了 docker 服务才行了

rabbitmq node not running


行情 重启了一下 docker  mq集群就好了 

试过 telnet  15276 有点不太对 一下就跳出来了  

也登录不了容器  一开始还以为是防火墙  



 docker exec  -it af1934d27e21 /bin/bash 
root@rabbitmq02:/# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbitmq02
[{nodes,[{disc,[rabbit@rabbitmq01,rabbit@rabbitmq02,rabbit@rabbitmq03]}]},
 {alarms,[]}]



再想加入 报错
:/# rabbitmqctl stop_app
Stopping rabbit application on node rabbit@rabbitmq02

root@rabbitmq02:/# rabbitmqctl join_cluster rabbit@rabbitmq01
Clustering node rabbit@rabbitmq02 with rabbit@rabbitmq01
Error: unable to connect to node rabbit@rabbitmq02: nodedown

DIAGNOSTICS
===========

attempted to contact: [rabbit@rabbitmq02]

rabbit@rabbitmq02:
  * connected to epmd (port 4369) on rabbitmq02
  * epmd reports node 'rabbit' running on port 25672
  * TCP connection succeeded but Erlang distribution failed

  * Distribution failed unexpectedly while waiting for challenge: {error,
                                                                   closed}


current node details:
- node name: 'rabbitmq-cli-69@rabbitmq02'
- home dir: /var/lib/rabbitmq
- cookie hash: /HHYeuND8Y2blMJ8WoOg1g==

root@rabbitmq02:/# rabbitmqctl start_app
Starting node rabbit@rabbitmq02
Error: unable to connect to node rabbit@rabbitmq02: nodedown


DIAGNOSTICS
===========

attempted to contact: [rabbit@rabbitmq02]

rabbit@rabbitmq02:
current node details:
- node name: 'rabbitmq-cli-69@rabbitmq02'
- home dir: /var/lib/rabbitmq
- cookie hash: /HHYeuND8Y2blMJ8WoOg1g==

root@rabbitmq02:/# rabbitmqctl start_app
Starting node rabbit@rabbitmq02
Error: unable to connect to node rabbit@rabbitmq02: nodedown


DIAGNOSTICS
===========

attempted to contact: [rabbit@rabbitmq02]

rabbit@rabbitmq02:
  * connected to epmd (port 4369) on rabbitmq02
  * epmd reports node 'rabbit' running on port 25672
  * TCP connection succeeded but Erlang distribution failed

  * TCP connection to remote host has timed out. Is the Erlang distribution using TLS?


current node details:
- node name: 'rabbitmq-cli-97@rabbitmq02'
- home dir: /var/lib/rabbitmq
- cookie hash: /HHYeuND8Y2blMJ8WoOg1g==

重启 docker 服务  再 如上添加 不过提示 The node is already a member of this cluster  所以应该是重启docker 就好了

单机版集群

version: '3.1'  
services:
  rabbitmq01:
    image: rabbitmq:3.6.10-management
    ports:
      - "5672:5672"
        #- "4369:4369"
      - "15672:15672"
        #- "25672:25672"
    container_name: rabbitmqCluster01
    hostname: rabbitmq01
    environment:
      RABBITMQ_ERLANG_COOKIE: 'rabbitmqCookie'   
      RABBITMQ_DEFAULT_VHOST: /
      RABBITMQ_DEFAULT_USER: lxtxadmin
      RABBITMQ_DEFAULT_PASS: tSNTTo886Gq
      RABBITMQ_LOGS: /var/lib/rabbitmq/rabbitmq.log
      RABBITMQ_SASL_LOGS: /var/lib/rabbitmq/rabbitmq-sasl.log
    volumes:
      - ./data/rabbitmq:/var/lib/rabbitmq
    restart: always  


  rabbitmq02:
    image: rabbitmq:3.6.10-management
    ports:
      - "5673:5672"
        #- "4369:4369"
      - "15673:15672"
        #- "25672:25672"
    container_name: rabbitmqCluster02
    hostname: rabbitmq02
    links:
      - rabbitmq01
    environment:
      RABBITMQ_ERLANG_COOKIE: 'rabbitmqCookie'   
      RABBITMQ_DEFAULT_VHOST: /
      RABBITMQ_DEFAULT_USER: lxtxadmin
      RABBITMQ_DEFAULT_PASS: tSNTTo886Gq
      RABBITMQ_LOGS: /var/lib/rabbitmq/rabbitmq.log
      RABBITMQ_SASL_LOGS: /var/lib/rabbitmq/rabbitmq-sasl.log
    volumes:
      - ./data/rabbitmq02:/var/lib/rabbitmq
    restart: always  


  rabbitmq03:
    image: rabbitmq:3.6.10-management
    ports:
      - "5674:5672"
        #- "4369:4369"
      - "15674:15672"
        #- "25672:25672"
    container_name: rabbitmqCluster03
    hostname: rabbitmq03
    links:
      - rabbitmq01
    environment:
      RABBITMQ_ERLANG_COOKIE: 'rabbitmqCookie'   
      RABBITMQ_DEFAULT_VHOST: /
      RABBITMQ_DEFAULT_USER: lxtxadmin
      RABBITMQ_DEFAULT_PASS: tSNTTo886Gq
      RABBITMQ_LOGS: /var/lib/rabbitmq/rabbitmq.log
      RABBITMQ_SASL_LOGS: /var/lib/rabbitmq/rabbitmq-sasl.log
    volumes:
      - ./data/rabbitmq03:/var/lib/rabbitmq
    restart: always  





然后再加入集群 同上

see also

https://www.rabbitmq.com/clustering.html#peer-discovery-how-does-it-work

Setting up a RabbitMQ Cluster on Docker

Docker分布式部署RabbitMQ集群

多台机器 值得看 Docker：搭建RabbitMQ集群

docker-compose 安装搭建 RabbitMQ 集群

单一台机器母机docker简易搭建RabbitMQ集群

RabbitMQ之集群搭建

也是单物理机么 ？ https://www.cnblogs.com/vipstone/p/9362388.html

docker简易搭建RabbitMQ集群

取自“http://wiki.linuxchina.net/index.php?title=Rabbitmq_cluster_on_docker-compose&oldid=4427”