|
|
| 第1行： |
第1行： |
| | =re= |
| | 平时的 mv /dat/*.war 在rsync 里面是不可用的 并不是写错 要**么 |
|
| |
|
| == 什么是rsync==
| | [http://blog.51cto.com/share/567578 rsync三：过滤规则] |
| rsync，remote synchronize顾名思意就知道它是一款实现远程同步功能的软件，它在同步文件的同时，可以保持原来文件的权限、时间、软硬链接等附加信息。 rsync是用 “rsync 算法”提供了一个客户机和远程文件服务器的文件同步的快速方法，而且可以通过ssh方式来传输文件，这样其保密性也非常好，另外它还是免费的软件。
| |
|
| |
|
| rsync 包括如下的一些特性：
| |
|
| |
|
| 能更新整个目录和树和文件系统；
| | =References= |
| | | [https://blog.csdn.net/wulong710/article/details/50403974 实现类似rsync中regex正则表达式] |
| 有选择性的保持符号链链、硬链接、文件属于、权限、设备以及时间等；
| | [[Category:shell]] [[category:ops]] |
| | |
| 对于安装来说，无任何特殊权限要求；
| |
| | |
| 对于多个文件来说，内部流水线减少文件等待的延时；
| |
| | |
| 能用rsh、ssh 或直接端口做为传输入端口；
| |
| | |
| 支持匿名rsync 同步文件，是理想的镜像工具；
| |
| | |
| rsync 是一个快速增量文件传输工具，它可以用于在同一主机备份内部的备分，我们还可以把它作为不同主机网络备份工具之用。本文主要讲述的是如何自架rsync服务器，以实现文件传输、备份和镜像。
| |
| | |
| ==1.服务器端==
| |
| <pre>
| |
| mkdir -p /data/allbackdata/
| |
| # uid git 可以改成你所想要的
| |
| echo '
| |
| ######################################################################################################
| |
| # ******进程相关全局配置******
| |
| ######################################################################################################
| |
| # = 后面的值可根据自己的实际情况更改
| |
| # pid file 守护进程pid文件
| |
| # port 守护进程监听端口，可更改，由xinetd允许rsyncd时忽略此参数
| |
| # address 守护进程监听ip，由xinetd允许rsyncd时忽略此参数
| |
| port = 873
| |
| #address = 192.168.1.2 #很多时候这个不要
| |
| uid=root
| |
| gid=root
| |
| | |
| #hosts allow = 192.168.1.130 # \\ 允许同步的机器，可以是一个网段 很多时候这个不要
| |
| #hosts deny = 0.0.0.0/0 #\\ 拒绝同步的机器，这里是只允许上面指定的机器 很多时候这个不要
| |
| use chroot = yes # 这个很多时候不敢要
| |
| read only = no
| |
| | |
| max connections = 80
| |
| timeout = 300
| |
| | |
| # read only = false 允许client上传
| |
| # wirte only = false 允许 client 下载
| |
| | |
| max connections = 20
| |
| timeout = 300
| |
| #欢迎文件路径，可选的
| |
| motd file = /etc/rsyncd.motd
| |
| pid file = /var/run/rsyncd.pid
| |
| log file = /data/logs/rsync.log
| |
| lock file = /var/run/rsync.lock
| |
| | |
| secrets file = /etc/pass.crt
| |
| | |
| [data]
| |
| path = /data/allbackdata/
| |
| #是否允许列出模块里的内容
| |
| list=yes
| |
| #忽略错误
| |
| #ignore errors
| |
| #排除目录，多个之间使用空格隔开
| |
| exclude = test1/ test2
| |
| auth users = ops ' > /etc/rsyncd.conf
| |
| | |
| #Note client 只要密码 不用写上用户哦 Nov 25 2021,
| |
| #但是这是server端 两个全要
| |
| echo 'ops:3636' > /etc/pass.crt
| |
| chmod 600 /etc/pass.crt #注意 这个一定得是 600 不然权限太大 失败的 Oct 16
| |
| | |
| cat /var/jenkins_home/lcpass
| |
| youpassword
| |
| | |
| | |
| | |
| | |
| #on service
| |
| iptables -A INPUT -s 47.8.16.30 -p tcp -m tcp --dport 873 -j ACCEPT
| |
| ##*******************
| |
| | |
| #自启动
| |
| chkconfig --level 345 rsyncd on
| |
|
| |
| </pre>
| |
| | |
| == 2. client ==
| |
| <pre>
| |
| iptables -A INPUT -s 10.45.241.123/32 -p tcp -m tcp --dport 873 -j ACCEPT
| |
| | |
| #pull
| |
| env RSYNC_PASSWORD=3636 rsync -avz s_20160928.sql.gz ops@10.4.265.234::data
| |
| | |
| rsync -avz s_20160928.sql.gz --delete --password-file=rsyncd.secrets ops@10.4.265.234::data
| |
| | |
| /usr/bin/rsync -al -q --timeout=50 ${DB_BAK_ROOT} root@${BAK_SERVER}::${MODULE}/${IPADDR}
| |
| | |
| </pre>
| |
| | |
| == 3.rsync daemon 启动脚本==
| |
| ===sysd centos7===
| |
| <pre>
| |
| cat /usr/lib/systemd/system/rsyncd.service
| |
| [Unit]
| |
| Description=fast remote file copy program daemon
| |
| ConditionPathExists=/etc/rsyncd.conf
| |
| | |
| [Service]
| |
| EnvironmentFile=/etc/sysconfig/rsyncd
| |
| ExecStart=/usr/bin/rsync --daemon --no-detach "$OPTIONS"
| |
| | |
| [Install]
| |
| WantedBy=multi-user.target
| |
| | |
| </pre>
| |
| === sysv centos6.x===
| |
| <pre>
| |
| #!/bin/bash
| |
| #
| |
| # rsyncd This shell script takes care of starting and stopping
| |
| # standalone rsync.
| |
| #
| |
| # chkconfig: - 99 50
| |
| # description: rsync is a file transport daemon
| |
| # processname: rsync
| |
| # config: /etc/rsyncd.conf
| |
|
| |
| # Source function library
| |
| . /etc/rc.d/init.d/functions
| |
|
| |
| RETVAL=0
| |
| rsync="/usr/bin/rsync"
| |
| prog="rsync"
| |
| CFILE="/etc/rsyncd.conf"
| |
| | |
| start() {
| |
| # Start daemons.
| |
| [ -x $rsync ] || \
| |
| { echo "FATAL: No such programme";exit 4; }
| |
| [ -f $CFILE ] || \
| |
| { echo "FATAL: config file does not exist";exit 6; }
| |
| echo -n $"Starting $prog: "
| |
| daemon $rsync --daemon --config=$CFILE
| |
| RETVAL=$?
| |
| [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
| |
| echo
| |
| return $RETVAL
| |
| }
| |
|
| |
| stop() {
| |
| # Stop daemons.
| |
| echo -n $"Stopping $prog: "
| |
| killproc $prog -QUIT
| |
| RETVAL=$?
| |
| echo
| |
| [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$prog
| |
| # [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/rsync && rm -f /var/lock/subsys/$prog && rm -f /var/run/rsyncd.pid /var/lock/subsys/rsync
| |
| | |
| return $RETVAL
| |
| }
| |
|
| |
| # call the function we defined
| |
| case "$1" in
| |
| start)
| |
| start
| |
| ;;
| |
| stop)
| |
| stop
| |
| ;;
| |
| restart|reload)
| |
| stop
| |
| sleep 2
| |
| start
| |
| RETVAL=$?
| |
| ;;
| |
| status)
| |
| status $prog
| |
| RETVAL=$?
| |
| ;;
| |
| *)
| |
| echo $"Usage: $0 {start|stop|restart|reload|status}"
| |
| exit 2
| |
| esac
| |
|
| |
| exit $RETVAL
| |
| </pre>
| |
| | |
| == trouble shooting ==
| |
| <pre>1.
| |
| uid=root
| |
| gid=root
| |
| | |
| 配置文件和 模块的目录属性要一致 不然传输会报错
| |
| rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1505)
| |
| | |
| 2. client 密码文件 也得是600
| |
| | |
| 3. address ip写得不对 启动不了
| |
| | |
| tail /var/log/message
| |
| 23 17:02:44 localhost rsyncd[5345]: rsyncd version 3.0.6 starting, listening on port 873
| |
| Mar 23 17:02:44 localhost rsyncd[5345]: bind() failed: Cannot assign requested address (address-family 2)
| |
| Mar 23 17:02:44 localhost rsyncd[5345]: unable to bind any inbound sockets on port 873
| |
| Mar 23 17:02:44 localhost rsyncd[5345]: rsync error: error in socket IO (code 10) at socket.c(541) [receiver=3.0.6]
| |
| | |
| 4. 启动脚本添加 rm -f /var/run/rsyncd.pid
| |
| | |
| 5.：
| |
| @ERROR: chroot failed
| |
| rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
| |
| | |
| 原因：
| |
| 服务器端的目录不存在或无权限。创建目录并修正权限可解决问题
| |
| | |
| 6.restart 时起不来，log如下
| |
| | |
| [root@ ~]# cat /data/logs/rsync.log
| |
| 2017/07/27 11:16:34 [21027] rsyncd version 3.0.6 starting, listening on port 873
| |
| 2017/07/27 11:16:34 [21027] bind() failed: Address already in use (address-family 2)
| |
| 2017/07/27 11:16:34 [21027] socket(10,1,6) failed: Address family not supported by protocol
| |
| 2017/07/27 11:16:34 [21027] unable to bind any inbound sockets on port 873
| |
| 2017/07/27 11:16:34 [21027] rsync error: error in socket IO (code 10) at socket.c(541) [receiver=3.0.6]
| |
| | |
| 改为start 成功
| |
| 最终的解决方案是 sleep 2
| |
| restart() {
| |
| stop
| |
| sleep 2
| |
| start
| |
| }
| |
| | |
| 7. 问题
| |
| restart后 其实有时进程不在
| |
| 2017/08/02 18:09:49 [6875] bind() failed: Address already in use (address-family 2)
| |
| 2017/08/02 18:09:49 [6875] unable to bind any inbound sockets on port 873
| |
| 2017/08/02 18:09:49 [6875] rsync error: error in socket IO (code 10) at socket.c(541) [receiver=3.0.6]
| |
| | |
| 为空
| |
| lsof -i:873
| |
| | |
| 解决 查看了nginx的启动脚本后 得到启发 在启动脚本上加 sleep 2
| |
| | |
| 8.查看log得知
| |
| Aug 2 17:30:13 localhost rsyncd[3232]: Badly formed boolean in configuration file: "on".
| |
| Aug 2 17:30:13 localhost rsyncd[3232]: Unknown Parameter encountered: "pid flie"
| |
| Aug 2 17:30:13 localhost rsyncd[3232]: IGNORING unknown parameter "pid flie"
| |
| | |
| | |
| 小伙伴no 写成了 on ,file 写成了 flie 哈哈哈哈
| |
| | |
| | |
| 期间还有 rsyslog 重启什么的
| |
| | |
| | |
| 直接手工启动
| |
| rsync --daemon --config=$CFILE
| |
| | |
| | |
| 9. rsync报错rsync: failed to set times on "." (in backup): Permission denied (13) 原创
| |
| | |
| 还有 对应的模块 目录一定得是运行是apapche
| |
| 因为rsyncd 运行用户是apapche
| |
| 不行 rsync报错rsync: failed to set times on "." (in backup): Permission denied (13) 原创
| |
| | |
| 10. on alpine docker
| |
| + env 'RSYNC_PASSWORD=7jKSNcSD2zN6AjJK' rsync -az '--port=873' '--password-file=/etc/lcpass' /var/jenkins_home/workspace/l/vue-wap/ rsync@172.89.16.7::lc-temp
| |
| rsync: [sender] could not open password file /etc/pass: Permission denied (13)
| |
| rsync error: syntax or usage error (code 1) at authenticate.c(188) [sender=3.2.3]
| |
| Post stage
| |
| [Pipeline]
| |
| | |
| 11. ERROR: password file must not be other-accessible
| |
| 密码文件得当前用户的权限的同时 600 所以在 alpine dokcer里 先用普通用户创建 再用root 去chmod 600 passwordfile
| |
| | |
| 12.
| |
| 2022/01/12 10:42:43 [22994] rsync: failed to write xattr user.rsync.%stat for "." (in sync-task): Permission denied (13)
| |
| 2022/01/12 10:42:43 [22994] rsync: failed to set times on "." (in core-sync-task): Operation not permitted (1)
| |
| 2022/01/12 10:42:43 [22994] ./
| |
| | |
| 整个/data/nginx 全改为 apache用户权限
| |
| | |
| </pre>
| |
| | |
| [https://www.jb51.net/article/60194.htm rsync @ERROR: chdir failed 的解决方法] | |
| | |
| [https://blog.51cto.com/u_9598767/1844669 rsync报错rsync: failed to set times on "." (in backup): Permission denied (13) ]
| |
| | |
| ==参考==
| |
| | |
| [http://wjw7702.blog.51cto.com/5210820/1148808 Rsync常见错误及命令详细参数] | |
| | |
| | |
| https://www.centos.bz/2011/06/rsync-server-setup/
| |
| | |
| https://segmentfault.com/a/1190000000444614
| |
| | |
| [http://www.cnblogs.com/itech/archive/2009/08/10/1542945.html RSync实现文件备份同步]
| |
| | |
| rsync一：工作模式及语法
| |
| http://share.blog.51cto.com/278008/560742
| |
| | |
| rsync二：daemon端配置文件
| |
| http://share.blog.51cto.com/278008/560761
| |
| | |
| rsync三：过滤规则
| |
| http://share.blog.51cto.com/278008/567578
| |
| | |
| https://rsync.samba.org/ftp/rsync/rsyncd.conf.html
| |
| | |
| | |
| http://jedy82.blog.51cto.com/425872/891341
| |
| | |
| [[category:ops]] | |