合 利用Docker搭建OceanBase 3.1.1社区版的集群
Tags： 高可用Docker安装部署环境搭建OceanBase集群部署
👉 本文共约2122个字，系统预计阅读时间或需8分钟。
环境规划
环境是这样的，只有一台虚拟机，配置16c80G，准备使用docker来模拟出4台CentOS主机，搭建OceanBase 3.1.1社区版的集群。OBServer每台至少需要8G内存，否则不能启动。
具体配置如下所示：
| 主机名 | IP | 端口 | 主机映射端口 | Zone | 作用 |
|---|---|---|---|---|---|
| lhrob1 | 172.72.8.11 | 2881 | 28811 | zone1 | OB Server1 |
| lhrob2 | 172.72.8.12 | 2881 | 28812 | zone2 | OB Server2 |
| lhrob3 | 172.72.8.13 | 2881 | 28813 | zone3 | OB Server3 |
| lhrobproxy | 172.72.8.14 | 2883 | 28814 | 用于OBD、OBProxy、OBClient、mysql客户端、时间服务器等 |
初始化准备
申请环境
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | docker network create --subnet=172.72.8.0/24 lhrob-network docker network inspect lhrob-network docker rm -f lhrob1 lhrob2 lhrob3 lhrobproxy docker run -d --name lhrob1 -h lhrob1 \ --net=lhrob-network --ip 172.72.8.11 \ -p 28811:2881 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:8.5 \ /usr/sbin/init docker run -d --name lhrob2 -h lhrob2 \ --net=lhrob-network --ip 172.72.8.12 \ -p 28812:2881 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:8.5 \ /usr/sbin/init docker run -d --name lhrob3 -h lhrob3 \ --net=lhrob-network --ip 172.72.8.13 \ -p 28813:2881 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:8.5 \ /usr/sbin/init docker run -d --name lhrobproxy -h lhrobproxy \ --net=lhrob-network --ip 172.72.8.14 \ -p 28814:2883 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:8.5 \ /usr/sbin/init docker exec -it lhrobproxy bash |
配置时钟源
如果您使用集群安装 OceanBase，则需要保证集群内各机器的时间同步。否则集群无法启动，服务在运行时也会出现异常。如果您已配置 NTP 时钟同步，则无需重新配置。
OceanBase 集群中的服务器时间必须保持一致，否则会导致 OceanBase 集群无法启动，运行时也会出现故障。物理机与时钟服务器的误差在 50ms 以下可认为时钟是同步状态，OceanBase 集群最大容忍误差不能超过 100ms。当超过 100ms 时，会出现无主情况。恢复时钟同步后。重启 OceanBase 集群, 可以恢复正常。
部署 OceanBase 集群时，各个 OBServer 的 RPC 允许的时钟偏差最大是100ms。
这里以“172.72.8.14”为时间服务器，其它3台OBServer同步该机器的时间：
1 2 3 4 | yum install ntp ntpdate -y ntpq -4p ntpstat timedatectl |
1、修改“172.72.8.14”为时间服务器/etc/ntp.conf
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | # For more information about this file, see the man pages # ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5). driftfile /var/lib/ntp/drift #新增:日志目录 logfile /var/log/ntpd.log # Permit time synchronization with our time source, but do not # permit the source to query or modify the service on this system. restrict default nomodify notrap nopeer noquery # Permit all access over the loopback interface. This could # be tightened as well, but to do so would effect some of # the administrative functions. restrict 127.0.0.1 restrict ::1 #新增:这一行的含义是授权172.72.8.0网段上的所有机器可以从这台机器上查询和同步时间. restrict 172.72.8.0 mask 255.255.255.0 nomodify notrap # Hosts on local network are less restricted. #restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst #新增:时间服务器列表. server 0.cn.pool.ntp.org iburst server 1.cn.pool.ntp.org iburst server 2.cn.pool.ntp.org iburst server 3.cn.pool.ntp.org iburst #新增:当外部时间不可用时，使用本地时间 server 127.0.0.1 iburst fudge 127.0.0.1 stratum 10 #broadcast 192.168.1.255 autokey # broadcast server #broadcastclient # broadcast client #broadcast 224.0.1.1 autokey # multicast server #multicastclient 224.0.1.1 # multicast client #manycastserver 239.255.254.254 # manycast server #manycastclient 239.255.254.254 autokey # manycast client # Enable public key cryptography. #crypto includefile /etc/ntp/crypto/pw # Key file containing the keys and key identifiers used when operating # with symmetric key cryptography. keys /etc/ntp/keys # Specify the key identifiers which are trusted. #trustedkey 4 8 42 # Specify the key identifier to use with the ntpdc utility. #requestkey 8 # Specify the key identifier to use with the ntpq utility. #controlkey 8 # Enable writing of statistics records. #statistics clockstats cryptostats loopstats peerstats # Disable the monitoring facility to prevent amplification attacks using ntpdc # monlist command when default restrict does not include the noquery flag. See # CVE-2013-5211 for more details. # Note: Monitoring will not be disabled with the limited restriction flag. disable monitor |
配置开机启动：
1 2 3 4 5 6 7 8 9 10 | systemctl enable ntpd systemctl is-enabled ntpd ntpdate -u 1.cn.pool.ntp.org systemctl restart ntpd [root@lhrobproxy /]# ntpstat synchronised to NTP server (84.16.73.33) at stratum 2 time correct to within 98 ms polling server every 64 s |
其它客户端，修改“/etc/ntp.conf”，注释server开头的行，并添加如下行：
1 2 3 4 5 6 7 | server 172.72.8.14 restrict 172.72.8.14 nomodify notrap noquery server 127.0.0.1 fudge 127.0.0.1 stratum 10 |
配置开机启动：
1 2 | systemctl enable ntpd systemctl restart ntpd |
客户端配置自动同步：
1 2 | crontab -e * * * * * /usr/sbin/ntpdate -u 172.72.8.14 & > /dev/null |
配置内核参数
4个节点都运行：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | cat >> /etc/security/limits.conf <<"EOF" root soft nofile 655350 root hard nofile 655350 * soft nofile 655350 * hard nofile 655350 * soft stack 20480 * hard stack 20480 * soft nproc 655360 * hard nproc 655360 * soft core unlimited * hard core unlimited EOF echo "fs.aio-max-nr=1048576" >> /etc/sysctl.conf sysctl -p |
如果只是测试，您可以只设置 fs.aio-max-nr=1048576。
创建用户
1 2 3 4 5 6 7 | useradd -U admin -d /home/admin -s /bin/bash echo "admin:lhr" | chpasswd chown -R admin:admin /home/admin echo "admin ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers |
设置无密码SSH登陆
可以使用rac上的sshUserSetup.sh快速配置，只在lhrobproxy上运行：
1 | sh sshUserSetup.sh -user admin -hosts "lhrob1 lhrob2 lhrob3 lhrobproxy" -advanced exverify -confirm |
本人提供Oracle(OCP、OCM)、MySQL(OCP)、PostgreSQL(PGCA、PGCE、PGCM)等数据库的培训和考证业务，私聊QQ646634621或微信dbaup66，谢谢！
