目录大纲
Part01-基础概念与理论知识
1.1 ZooKeeper架构概述
1.2 数据模型与节点
1.3 一致性协议原理
Part02-生产环境规划与建议
2.1 集群部署规划
2.2 节点设计规划
2.3 性能优化规划
Part03-生产环境项目实施方案
3.1 ZooKeeper安装配置
3.2 节点操作管理
3.3 集群监控维护
3.4 客户端开发
Part04-生产案例与实战讲解
4.1 配置中心案例
4.2 分布式锁案例
4.3 服务注册发现案例
Part05-风哥经验总结与分享
5.1 ZooKeeper最佳实践
5.2 运维经验总结
Part01-基础概念与理论知识
1.1 ZooKeeper架构概述
ZooKeeper是分布式协调服务。更多视频教程www.fgedu.net.cn 提供配置维护、分布式同步、组服务等功能。
1.2 数据模型与节点
ZooKeeper数据模型是树形结构,每个节点称为ZNode。学习交流加群风哥微信: itpux-com
– 持久节点:永久存在,直到被删除
– 临时节点:会话结束自动删除
– 持久顺序节点:带顺序号的持久节点
– 临时顺序节点:带顺序号的临时节点
1.3 一致性协议原理
ZooKeeper使用ZAB协议保证一致性。from bigdata视频:www.itpux.com
zkServer.sh version
# 查看ZooKeeper状态
zkServer.sh status
Apache ZooKeeper, version 3.8.1
# ZooKeeper状态
Mode: leader
Zxid: 0x100000abc
# 当前节点为Leader
Part02-生产环境规划与建议
2.1 集群部署规划
ZooKeeper集群部署需要考虑节点数量和资源配置。更多学习教程公众号风哥教程itpux_com
– 节点数量:奇数个,至少3个
– 内存配置:建议4GB以上
– 磁盘配置:使用独立磁盘存储快照
– 网络配置:低延迟网络环境
2.2 节点设计规划
节点设计影响应用性能。学习交流加群风哥QQ113257174
zkCli.sh -server fgedu01:2181
# 查看根节点
ls /
# 查看节点详情
ls -s /fgedu
Connecting to fgedu01:2181
Welcome to ZooKeeper!
JLine support is enabled
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
# 根节点
[cluster, controller, controller_epoch, fgedu, zookeeper]
# 节点详情
[config, lock, registry]
czxid = 0x100000001
ctime = Thu Jan 18 00:00:00 CST 2024
mzxid = 0x100000001
mtime = Thu Jan 18 00:00:00 CST 2024
pZxid = 0x100000010
cversion = 3
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 3
2.3 性能优化规划
性能优化需要从多个角度考虑。风哥提示:合理的快照和事务日志配置是性能优化的关键。
– 分离快照和事务日志
– 调整tickTime
– 优化JVM参数
– 监控节点状态
– 定期清理旧快照
Part03-生产环境项目实施方案
3.1 ZooKeeper安装配置
3.1.1 安装ZooKeeper
wget https://downloads.apache.org/zookeeper/zookeeper-3.8.1/apache-zookeeper-3.8.1-bin.tar.gz
tar -xzf apache-zookeeper-3.8.1-bin.tar.gz -C /bigdata/app/
ln -s /bigdata/app/apache-zookeeper-3.8.1-bin /bigdata/app/zookeeper
# 配置环境变量
export ZOOKEEPER_HOME=/bigdata/app/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin
# 配置zoo.cfg
cat > /bigdata/app/zookeeper/conf/zoo.cfg << 'EOF'
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/bigdata/zookeeper/data
dataLogDir=/bigdata/zookeeper/logs
clientPort=2181
server.1=fgedu01:2888:3888
server.2=fgedu02:2888:3888
server.3=fgedu03:2888:3888
autopurge.snapRetainCount=3
autopurge.purgeInterval=1
EOF
# 配置myid
echo “1” > /bigdata/zookeeper/data/myid
# 验证安装
zkServer.sh version
# 完成
# 环境变量
# 配置完成
# 配置文件
# 配置完成
# myid配置
# 配置完成
# 验证安装
Apache ZooKeeper, version 3.8.1
# ZooKeeper安装成功
3.1.2 启动ZooKeeper
zkServer.sh start
# 查看状态
zkServer.sh status
# 查看进程
jps | grep QuorumPeerMain
# 验证连接
zkCli.sh -server fgedu01:2181 <<< "quit"
ZooKeeper JMX enabled by default
Using config: /bigdata/app/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper … STARTED
# 查看状态
Mode: leader
Zxid: 0x100000000
# 进程查看
12345 QuorumPeerMain
# 验证连接
Connecting to fgedu01:2181
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
# ZooKeeper启动成功
3.2 节点操作管理
3.2.1 创建节点
zkCli.sh -server fgedu01:2181
# 创建持久节点
create /fgedu/config “config_data”
# 创建临时节点
create -e /fgedu/temp “temp_data”
# 创建顺序节点
create -s /fgedu/seq “seq_data”
# 创建子节点
create /fgedu/config/db “mysql”
create /fgedu/config/cache “redis”
# 查看节点
ls /fgedu
# 连接成功
# 创建持久节点
Created /fgedu/config
# 创建临时节点
Created /fgedu/temp
# 创建顺序节点
Created /fgedu/seq0000000001
# 创建子节点
Created /fgedu/config/db
Created /fgedu/config/cache
# 查看节点
[config, temp, seq0000000001]
3.2.2 读取节点
get /fgedu/config
# 获取节点详情
get -s /fgedu/config
# 获取子节点列表
ls /fgedu/config
# 获取子节点详情
ls -s /fgedu/config
# 递归获取节点
getAllChildrenNumber /fgedu
config_data
# 获取节点详情
config_data
czxid = 0x100000001
ctime = Fri Jan 19 04:00:00 CST 2024
mzxid = 0x100000001
mtime = Fri Jan 19 04:00:00 CST 2024
pZxid = 0x100000005
cversion = 2
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 11
numChildren = 2
# 子节点列表
[db, cache]
# 子节点详情
[db, cache]
total = 2
# 递归获取
5
3.2.3 更新和删除节点
set /fgedu/config “new_config_data”
# 带版本更新
set /fgedu/config “version_config” 1
# 删除节点
delete /fgedu/temp
# 递归删除节点
deleteall /fgedu/test
# 验证删除
ls /fgedu
cZxid = 0x100000001
ctime = Fri Jan 19 04:00:00 CST 2024
mZxid = 0x100000010
mtime = Fri Jan 19 04:10:00 CST 2024
# 带版本更新
cZxid = 0x100000001
mZxid = 0x100000011
# 删除节点
# 删除成功
# 递归删除
# 删除成功
# 验证删除
[config, seq0000000001]
3.3 集群监控维护
3.3.1 集群状态监控
zkServer.sh status
# 查看所有节点状态
for host in fgedu01 fgedu02 fgedu03; do
echo “=== ${host} ===”
ssh ${host} “zkServer.sh status”
done
# 查看四字命令
echo “stat” | nc fgedu01 2181
echo “mntr” | nc fgedu01 2181
echo “conf” | nc fgedu01 2181
Mode: leader
Zxid: 0x100000100
# 所有节点状态
=== fgedu01 ===
Mode: leader
=== fgedu02 ===
Mode: follower
=== fgedu03 ===
Mode: follower
# 四字命令
Zookeeper version: 3.8.1
Clients:
/192.168.1.10:12345[0](queued=0,recved=1,sent=0)
Latency min/avg/max: 0/1/10
Received: 1000
Sent: 1000
Connections: 1
Outstanding: 0
Zxid: 0x100000100
Mode: leader
Node count: 100
3.3.2 集群维护操作
ls -la /bigdata/zookeeper/data/
ls -la /bigdata/zookeeper/logs/
# 清理旧快照
zkCleanup.sh /bigdata/zookeeper/data 3
# 备份数据
tar -czf /backup/zookeeper/zk_data_$(date +%Y%m%d).tar.gz /bigdata/zookeeper/data/
# 恢复数据
zkServer.sh stop
tar -xzf /backup/zookeeper/zk_data_20240119.tar.gz -C /
zkServer.sh start
total 100
-rw-r–r– 1 zookeeper zookeeper 1 Jan 19 04:00 myid
-rw-r–r– 1 zookeeper zookeeper 1024 Jan 19 04:00 snapshot.100000010
total 100
-rw-r–r– 1 zookeeper zookeeper 1024 Jan 19 04:00 log.100000001
# 清理旧快照
Removing old snapshots
# 清理完成
# 备份数据
# 备份完成
# 恢复数据
ZooKeeper JMX enabled by default
Using config: /bigdata/app/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper … STOPPED
# 恢复完成
Starting zookeeper … STARTED
3.4 客户端开发
3.4.1 Java客户端示例
import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;
public class FgeduZkClient {
private ZooKeeper zk;
public void connect(String hosts) throws Exception {
zk = new ZooKeeper(hosts, 3000, event -> {
System.out.println(“Event: ” + event);
});
}
public void create(String path, String data) throws Exception {
zk.create(path, data.getBytes(),
ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
}
public String getData(String path) throws Exception {
byte[] data = zk.getData(path, false, new Stat());
return new String(data);
}
public void close() throws Exception {
zk.close();
}
}
// 使用示例
FgeduZkClient client = new FgeduZkClient();
client.connect(“fgedu01:2181,fgedu02:2181,fgedu03:2181”);
client.create(“/fgedu/app”, “app_config”);
String data = client.getData(“/fgedu/app”);
System.out.println(“Data: ” + data);
client.close();
Event: WatchedEvent state:SyncConnected type:None path:null
Data: app_config
# 客户端操作成功
3.4.2 Python客户端示例
from kazoo.client import KazooClient
# 连接ZooKeeper
zk = KazooClient(hosts=’fgedu01:2181,fgedu02:2181,fgedu03:2181′)
zk.start()
# 创建节点
zk.ensure_path(‘/fgedu/python’)
zk.create(‘/fgedu/python/config’, b’python_config’)
# 读取节点
data, stat = zk.get(‘/fgedu/python/config’)
print(f”Data: {data.decode()}”)
# 更新节点
zk.set(‘/fgedu/python/config’, b’new_config’)
# 删除节点
zk.delete(‘/fgedu/python/config’)
# 关闭连接
zk.stop()
Data: python_config
# 客户端操作成功
Part04-生产案例与实战讲解
4.1 配置中心案例
ZooKeeper可以作为分布式配置中心。更多视频教程www.fgedu.net.cn
# zk_config_center.sh
# from:www.itpux.com.qq113257174.wx:itpux-com
# web: http://www.fgedu.net.cn
echo “=== ZooKeeper Config Center ===”
echo “Date: $(date)”
# 创建配置节点
zkCli.sh -server fgedu01:2181 << EOF
create /fgedu/config/database “mysql://fgedu01:3306/fgedudb”
create /fgedu/config/redis “redis://fgedu01:6379”
create /fgedu/config/kafka “fgedu01:9092,fgedu02:9092,fgedu03:9092”
quit
EOF
# 读取配置
echo “=== Read Config ===”
zkCli.sh -server fgedu01:2181 << EOF
get /fgedu/config/database
get /fgedu/config/redis
get /fgedu/config/kafka
quit
EOF
echo “=== Config Center Completed ===”
./zk_config_center.sh
=== ZooKeeper Config Center ===
Date: Fri Jan 19 05:00:00 CST 2024
# 创建配置节点
Created /fgedu/config/database
Created /fgedu/config/redis
Created /fgedu/config/kafka
# 读取配置
=== Read Config ===
mysql://fgedu01:3306/fgedudb
redis://fgedu01:6379
fgedu01:9092,fgedu02:9092,fgedu03:9092
=== Config Center Completed ===
4.2 分布式锁案例
分布式锁是ZooKeeper的经典应用。学习交流加群风哥微信: itpux-com
zkCli.sh -server fgedu01:2181 << 'EOF' # 创建锁节点
create /fgedu/lock “”
# 获取锁(创建临时顺序节点)
create -e -s /fgedu/lock/request_ “”
# 查看锁节点
ls -s /fgedu/lock
# 判断是否获取锁
# 如果是最小序号节点,则获取锁成功
# 释放锁
delete /fgedu/lock/request_0000000001
quit
EOF
# 创建锁节点
Created /fgedu/lock
# 创建临时顺序节点
Created /fgedu/lock/request_0000000001
# 查看锁节点
[request_0000000001]
total = 1
# 释放锁
# 删除成功
4.3 服务注册发现案例
4.3.1 服务注册
zkCli.sh -server fgedu01:2181 << 'EOF' # 创建服务根节点
create /fgedu/services “”
# 注册服务
create -e /fgedu/services/user-service “192.168.1.10:8080”
create -e /fgedu/services/order-service “192.168.1.11:8080”
create -e /fgedu/services/product-service “192.168.1.12:8080”
# 查看服务列表
ls /fgedu/services
quit
EOF
# 创建服务根节点
Created /fgedu/services
# 注册服务
Created /fgedu/services/user-service
Created /fgedu/services/order-service
Created /fgedu/services/product-service
# 服务列表
[user-service, order-service, product-service]
4.3.2 服务发现
zkCli.sh -server fgedu01:2181 << 'EOF' # 获取服务地址
get /fgedu/services/user-service
# 监听服务变化
ls -w /fgedu/services
# 获取所有服务
ls /fgedu/services
quit
EOF
# 获取服务地址
192.168.1.10:8080
# 监听服务变化
WATCHER::
WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/fgedu/services
# 所有服务
[user-service, order-service, product-service]
Part05-风哥经验总结与分享
5.1 ZooKeeper最佳实践
在实际生产环境中,ZooKeeper使用需要注意以下几点:from bigdata视频:www.itpux.com
1. 集群节点数为奇数
2. 分离快照和事务日志
3. 监控集群状态
4. 定期备份数据
5. 合理设置会话超时
5.2 运维经验总结
5.2.1 运维建议
– 监控Leader选举
– 关注事务延迟
– 定期清理旧快照
– 做好容量规划
– 制定故障恢复方案
5.2.2 ZooKeeper运维脚本
# zk_maintenance.sh
# from:www.itpux.com.qq113257174.wx:itpux-com
# web: http://www.fgedu.net.cn
echo “=== ZooKeeper Maintenance ===”
echo “Date: $(date)”
# 1. 检查集群状态
echo “=== Check Cluster Status ===”
for host in fgedu01 fgedu02 fgedu03; do
echo “— ${host} —“
ssh ${host} “zkServer.sh status”
done
# 2. 检查节点数量
echo “=== Check Node Count ===”
echo “stat” | nc fgedu01 2181 | grep “Node count”
# 3. 检查磁盘使用
echo “=== Check Disk Usage ===”
df -h /bigdata/zookeeper/
# 4. 清理旧快照
echo “=== Cleanup Old Snapshots ===”
zkCleanup.sh /bigdata/zookeeper/data 3
echo “=== Maintenance Completed ===”
./zk_maintenance.sh
=== ZooKeeper Maintenance ===
Date: Fri Jan 19 06:00:00 CST 2024
=== Check Cluster Status ===
— fgedu01 —
Mode: leader
— fgedu02 —
Mode: follower
— fgedu03 —
Mode: follower
=== Check Node Count ===
Node count: 100
=== Check Disk Usage ===
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 50G 10G 40G 20% /bigdata
=== Cleanup Old Snapshots ===
Removing old snapshots
=== Maintenance Completed ===
本文由风哥教程整理发布,仅用于学习测试使用,转载注明出处:http://www.fgedu.net.cn/10327.html
