OceanBase教程FG006-OceanBase集群扩容缩容实战
本文详细介绍OceanBase数据库集群的扩容和缩容操作,帮助读者掌握OceanBase集群的动态扩展和收缩能力。风哥教程参考OceanBase官方文档OceanBase8集群管理、OceanBase8运维指南等内容。
OceanBase集群支持在线扩容和缩容,无需停机即可完成节点的添加和移除,这对于业务的持续运行非常重要。通过本文的学习,读者将掌握OceanBase集群扩容和缩容的完整流程,包括环境准备、操作步骤、验证测试等内容。
集群扩容和缩容是OceanBase运维的重要操作,了解其操作流程和注意事项对于保证系统的稳定性和性能至关重要。
目录大纲
Part01-基础概念与理论知识
1.1 扩容缩容概述
OceanBase集群扩容是指向现有集群中添加新的节点,以增加集群的处理能力和存储容量。集群缩容是指从现有集群中移除节点,以减少资源消耗。
扩容和缩容的主要目的:
- 扩容:
- 应对业务增长,增加处理能力
- 提高集群的可用性和可靠性
- 分散负载,提高性能
- 缩容:
- 减少资源浪费
- 降低运维成本
- 优化集群结构
1.2 扩容缩容原理
OceanBase集群扩容和缩容的原理:
- 扩容原理:
- 添加新节点到集群
- 将数据分区重新分布到新节点
- 调整资源分配
- 更新集群元数据
- 缩容原理:
- 将待移除节点上的数据分区迁移到其他节点
- 调整资源分配
- 更新集群元数据
- 从集群中移除节点
OceanBase的扩容和缩容操作是在线进行的,不会影响业务的正常运行。
Part02-生产环境规划与建议
2.1 扩容规划
扩容规划包括:
- 需求分析:
- 业务增长趋势
- 当前集群负载情况
- 性能瓶颈分析
,风哥提示:。
- 硬件规划:
- 新节点的硬件配置
- 存储容量规划
- 网络带宽规划
- 网络规划:
- 新节点的IP地址
- 网络连接配置
- 防火墙规则
- 时间规划:
- 扩容操作时间窗口
- 数据迁移时间预估
- 验证测试时间
2.2 缩容规划
缩容规划包括:
- 需求分析:
- ,学习交流加群风哥微信: itpux-com。
- 业务负载下降情况
- 集群资源利用率
- 成本优化需求
- 节点选择:
- 选择负载较低的节点
- 确保移除节点后集群仍能正常运行
- 考虑节点的硬件状况
- 数据迁移规划:
- 数据迁移时间预估
- 迁移过程中的性能影响
- 数据一致性保证
- 时间规划:
- 缩容操作时间窗口
- 数据迁移时间
- 验证测试时间
Part03-生产环境项目实施方案
3.1 扩容操作步骤
OceanBase集群扩容操作步骤:
- 准备新节点环境
- 配置新节点网络
- 安装OceanBase软件
- 将新节点添加到集群
- 等待数据迁移完成
- 验证新节点状态
- 调整资源配置
,学习交流加群风哥QQ113257174。
3.2 缩容操作步骤
OceanBase集群缩容操作步骤:
- 选择待移除的节点
- 将节点上的数据迁移到其他节点
- 从集群中移除节点
- 停止节点上的OceanBase服务
- 验证集群状态
- 调整资源配置
Part04-生产案例与实战讲解
4.1 集群扩容实战
以三节点集群扩容到四节点为例,演示扩容操作。
mkdir -p /ob/app /ob/fgdata /ob/log /ob/config
cat > /etc/sysctl.d/oceanbase.conf << EOF
fs.file-max = 6815744
fs.nr_open = 6815744
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl = 15
vm.swappiness = 0
vm.overcommit_memory = 2
vm.overcommit_ratio = 90
vm.nr_hugepages = 16384
,更多视频教程www.fgedu.net.cn。
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
EOF
sysctl -p /etc/sysctl.d/oceanbase.conf
fs.nr_open = 6815744
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl = 15
vm.swappiness = 0
vm.overcommit_memory = 2
vm.overcommit_ratio = 90
vm.nr_hugepages = 16384
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
yum install -y libaio-devel ncurses-devel zlib-devel
Complete!
cat > /ob/config/cluster.yaml << EOF
oceanbase-ce:
servers:
– 192.168.1.100
,更多学习教程公众号风哥教程itpux_com。
– 192.168.1.101
– 192.168.1.102
– 192.168.1.103
global:
home_path: /ob/app
data_dir: /ob/fgdata
log_dir: /ob/log
devname: eth0
mysql_port: 2881
rpc_port: 2882
zone:
– zone1
– zone2
– zone3
– zone4
cluster_id: 1
memory_limit: 32G
system_memory: 16G
datafile_size: 100G
log_disk_size: 100G
cpu_count: 16
enable_syslog_wf: false
enable_syslog_recycle: true
max_syslog_file_count: 10
root_password: fgedu123
proxyro_password: fgedu123
obproxy:
servers:
– 192.168.1.100
– 192.168.1.101
,from DB视频:www.itpux.com。
– 192.168.1.102
– 192.168.1.103
global:
home_path: /ob/app/obproxy
listen_port: 2883
prometheus_listen_port: 2884
observer_list: 192.168.1.100:2881;192.168.1.101:2881;192.168.1.102:2881;192.168.1.103:2881
cluster_name: obcluster
password: fgedu123
EOF
obd cluster scale-out obcluster -c /ob/config/cluster.yaml
Open ssh connection ok
Cluster status check ok
Install oceanbase-ce-4.2.0.0 ok
Install obproxy-3.2.3 ok
Load cluster parameters ok
Check before start ok
Start observer ok
Start obproxy ok
Connect to observer ok
Wait for observer init ok
Cluster status: running
observer[192.168.1.100:2881] status: running
observer[192.168.1.101:2881] status: running
observer[192.168.1.102:2881] status: running
observer[192.168.1.103:2881] status: running
obproxy[192.168.1.100:2883] status: running
obproxy[192.168.1.101:2883] status: running
obproxy[192.168.1.102:2883] status: running
obproxy[192.168.1.103:2883] status: running
Scale-out obcluster successfully
obd cluster status obcluster
Open ssh connection ok
Cluster status check ok
Connect to observer ok
Wait for observer init ok
Cluster status: running
observer[192.168.1.100:2881] status: running
observer[192.168.1.101:2881] status: running
observer[192.168.1.102:2881] status: running
observer[192.168.1.103:2881] status: running
obproxy[192.168.1.100:2883] status: running
obproxy[192.168.1.101:2883] status: running
obproxy[192.168.1.102:2883] status: running
obproxy[192.168.1.103:2883] status: running
obclient -h192.168.1.100 -P2883 -uroot -pfgedu123 -Doceanbase
Your OceanBase connection id is 32768
Server version: OceanBase 4.2.0.0 (r10000032022101016)
Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.
Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.
obclient [oceanbase]>
SELECT * FROM oceanbase.DBA_OB_SERVERS;
| SVR_IP | SVR_PORT | ZONE | STATUS | START_SERVICE_TIME | STOP_TIME | SQL_PORT | RPC_PORT | WITH_ROOTSERVICE | CPU_CNT | USED_MEMORY | TOTAL_MEMORY | SVR_VERSION | BUILD_VERSION |
+—————-+———-+———-+————+—————————-+——–+—————-+————-+——————+——–+————————+—————————-+—————————-+——————+
| 192.168.1.100 | 2882 | zone1 | ACTIVE | 2023-01-01 00:00:00.000000 | NULL | 2881 | 2882 | TRUE | 16 | 10737418240 (10.00GB) | 34359738368 (32.00GB) | 4.2.0.0 (r10000032022101016) | 4.2.0.0_2022101016 |
| 192.168.1.101 | 2882 | zone2 | ACTIVE | 2023-01-01 00:00:00.000000 | NULL | 2881 | 2882 | FALSE | 16 | 10737418240 (10.00GB) | 34359738368 (32.00GB) | 4.2.0.0 (r10000032022101016) | 4.2.0.0_2022101016 |
| 192.168.1.102 | 2882 | zone3 | ACTIVE | 2023-01-01 00:00:00.000000 | NULL | 2881 | 2882 | FALSE | 16 | 10737418240 (10.00GB) | 34359738368 (32.00GB) | 4.2.0.0 (r10000032022101016) | 4.2.0.0_2022101016 |
| 192.168.1.103 | 2882 | zone4 | ACTIVE | 2023-01-01 00:30:00.000000 | NULL | 2881 | 2882 | FALSE | 16 | 10737418240 (10.00GB) | 34359738368 (32.00GB) | 4.2.0.0 (r10000032022101016) | 4.2.0.0_2022101016 |
+—————-+———-+———-+————+—————————-+——–+—————-+————-+——————+——–+————————+—————————-+—————————-+——————+
4 rows in set (0.01 sec)
EXIT;
4.2 集群缩容实战
以四节点集群缩容到三节点为例,演示缩容操作。
cat > /ob/config/cluster.yaml << EOF
oceanbase-ce:
servers:
– 192.168.1.100
– 192.168.1.101
– 192.168.1.102
global:
home_path: /ob/app
data_dir: /ob/fgdata
log_dir: /ob/log
devname: eth0
mysql_port: 2881
rpc_port: 2882
zone:
– zone1
– zone2
– zone3
cluster_id: 1
memory_limit: 32G
system_memory: 16G
datafile_size: 100G
log_disk_size: 100G
cpu_count: 16
enable_syslog_wf: false
enable_syslog_recycle: true
max_syslog_file_count: 10
root_password: fgedu123
proxyro_password: fgedu123
obproxy:
servers:
– 192.168.1.100
– 192.168.1.101
– 192.168.1.102
global:
home_path: /ob/app/obproxy
listen_port: 2883
prometheus_listen_port: 2884
observer_list: 192.168.1.100:2881;192.168.1.101:2881;192.168.1.102:2881
cluster_name: obcluster
password: fgedu123
EOF
obd cluster scale-in obcluster -c /ob/config/cluster.yaml
Open ssh connection ok
Cluster status check ok
Start to scale in observer[192.168.1.103:2881]
Wait for observer[192.168.1.103:2881] to be offline
Stop observer[192.168.1.103:2881] ok
Stop obproxy[192.168.1.103:2883] ok
Cluster status: running
observer[192.168.1.100:2881] status: running
observer[192.168.1.101:2881] status: running
observer[192.168.1.102:2881] status: running
obproxy[192.168.1.100:2883] status: running
obproxy[192.168.1.101:2883] status: running
obproxy[192.168.1.102:2883] status: running
Scale-in obcluster successfully
obd cluster status obcluster
Open ssh connection ok
Cluster status check ok
Connect to observer ok
Wait for observer init ok
Cluster status: running
observer[192.168.1.100:2881] status: running
observer[192.168.1.101:2881] status: running
observer[192.168.1.102:2881] status: running
obproxy[192.168.1.100:2883] status: running
obproxy[192.168.1.101:2883] status: running
obproxy[192.168.1.102:2883] status: running
obclient -h192.168.1.100 -P2883 -uroot -pfgedu123 -Doceanbase
Your OceanBase connection id is 32768
Server version: OceanBase 4.2.0.0 (r10000032022101016)
Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.
Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.
obclient [oceanbase]>
SELECT * FROM oceanbase.DBA_OB_SERVERS;
| SVR_IP | SVR_PORT | ZONE | STATUS | START_SERVICE_TIME | STOP_TIME | SQL_PORT | RPC_PORT | WITH_ROOTSERVICE | CPU_CNT | USED_MEMORY | TOTAL_MEMORY | SVR_VERSION | BUILD_VERSION |
+—————-+———-+———-+————+—————————-+——–+—————-+————-+——————+——–+————————+—————————-+—————————-+——————+
| 192.168.1.100 | 2882 | zone1 | ACTIVE | 2023-01-01 00:00:00.000000 | NULL | 2881 | 2882 | TRUE | 16 | 10737418240 (10.00GB) | 34359738368 (32.00GB) | 4.2.0.0 (r10000032022101016) | 4.2.0.0_2022101016 |
| 192.168.1.101 | 2882 | zone2 | ACTIVE | 2023-01-01 00:00:00.000000 | NULL | 2881 | 2882 | FALSE | 16 | 10737418240 (10.00GB) | 34359738368 (32.00GB) | 4.2.0.0 (r10000032022101016) | 4.2.0.0_2022101016 |
| 192.168.1.102 | 2882 | zone3 | ACTIVE | 2023-01-01 00:00:00.000000 | NULL | 2881 | 2882 | FALSE | 16 | 10737418240 (10.00GB) | 34359738368 (32.00GB) | 4.2.0.0 (r10000032022101016) | 4.2.0.0_2022101016 |
+—————-+———-+———-+————+—————————-+——–+—————-+————-+——————+——–+————————+—————————-+—————————-+——————+
3 rows in set (0.01 sec)
EXIT;
Part05-风哥经验总结与分享
5.1 扩容缩容最佳实践
- 在业务低峰期进行扩容和缩容操作
- 扩容前确保新节点的硬件配置与现有节点一致
- 缩容前确保待移除节点上的数据已完全迁移
- 扩容和缩容操作后,验证集群状态和数据一致性
- 定期监控集群性能,及时进行扩容或缩容
- 保持集群节点数量为奇数,确保高可用性
- 使用OBD工具进行扩容和缩容操作,简化操作流程
5.2 常见问题与解决方案
- 扩容失败:检查新节点的网络连接、权限配置、资源是否充足
- 缩容失败:检查待移除节点上的数据迁移状态,确保数据已完全迁移
- 数据迁移缓慢:检查网络带宽,调整数据迁移参数
- 集群状态异常:检查节点之间的网络连接,确保所有节点正常运行
- 性能下降:在扩容或缩容过程中,可能会出现暂时的性能下降,属于正常现象
故障排查方法:
- 查看OceanBase日志:/ob/log
- 检查集群状态:obd cluster status
- 查看数据迁移状态:通过OCP或系统视图查看
- 检查网络连接:ping、telnet
- 风哥教程参考OceanBase官方文档和社区论坛
本文由风哥教程整理发布,仅用于学习测试使用,转载注明出处:http://www.fgedu.net.cn/10327.html
