1. 首页 > 软件下载 > 正文

Canal下载-MySQL增量数据同步工具下载地址-Canal安装部署方法

1. Canal简介与版本说明

Canal是阿里巴巴开源的MySQL数据库增量日志解析工具,提供增量数据订阅和消费功能。Canal通过模拟MySQL Slave的交互协议,伪装自己为MySQL Slave,向MySQL Master发送dump协议,MySQL Master收到dump请求后推送binary log给Slave(即Canal),Canal解析binary log对象(原始为byte流)。更多学习教程www.fgedu.net.cn

Canal最新版本:

Canal 1.1.8 (2024年最新稳定版)
Canal 1.1.7 (2023年稳定版)
Canal 1.1.6 (2022年稳定版)
Canal 1.1.5 (2021年稳定版)
Canal 1.1.4 (2020年稳定版)

Canal主要组件:

canal-server:Canal服务端,负责解析MySQL binlog
canal-adapter:Canal适配器,负责数据同步到目标端
canal-admin:Canal管理后台,提供Web管理界面

2. Canal下载方式

Canal提供多种下载方式,包括官方GitHub Release、阿里云Maven仓库等。学习交流加群风哥微信: itpux-com

方式一:GitHub Release下载

# 创建下载目录
$ mkdir -p /fgeudb/software/canal
$ cd /fgeudb/software/canal

# 下载Canal Server 1.1.8
$ wget https://github.com/alibaba/canal/releases/download/canal-1.1.8/canal.deployer-1.1.8.tar.gz

# 下载Canal Adapter 1.1.8
$ wget https://github.com/alibaba/canal/releases/download/canal-1.1.8/canal.adapter-1.1.8.tar.gz

# 下载Canal Admin 1.1.8
$ wget https://github.com/alibaba/canal/releases/download/canal-1.1.8/canal.admin-1.1.8.tar.gz

# 查看下载文件
$ ls -lh
total 85M
-rw-r–r– 1 root root 28M Apr 4 10:00 canal.deployer-1.1.8.tar.gz
-rw-r–r– 1 root root 35M Apr 4 10:00 canal.adapter-1.1.8.tar.gz
-rw-r–r– 1 root root 22M Apr 4 10:00 canal.admin-1.1.8.tar.gz

方式二:国内镜像下载

# 使用阿里云镜像加速下载
$ wget https://mirrors.aliyun.com/apache/canal/1.1.8/canal.deployer-1.1.8.tar.gz

# 或使用华为云镜像
$ wget https://mirrors.huawei.com/apache/canal/1.1.8/canal.deployer-1.1.8.tar.gz

方式三:Maven依赖引入

# 在pom.xml中添加依赖
<dependency>
<groupId>com.alibaba.otter</groupId>
<artifactId>canal.client</artifactId>
<version>1.1.8</version>
</dependency>

# 如需使用Canal协议
<dependency>
<groupId>com.alibaba.otter</groupId>
<artifactId>canal.protocol</artifactId>
<version>1.1.8</version>
</dependency>

方式四:Docker镜像下载

# 拉取Canal Server镜像
$ docker pull canal/canal-server:v1.1.8

# 拉取Canal Admin镜像
$ docker pull canal/canal-admin:v1.1.8

# 查看镜像列表
$ docker images | grep canal
canal/canal-server v1.1.8 abc123def456 7 days ago 650MB
canal/canal-admin v1.1.8 def456ghi789 7 days ago 480MB

3. MySQL配置要求

Canal依赖MySQL的binlog功能,需要正确配置MySQL服务器。学习交流加群风哥QQ113257174

MySQL配置文件修改

# 编辑MySQL配置文件
$ vi /etc/my.cnf

# 在[mysqld]段添加以下配置
[mysqld]
# 开启binlog
log-bin=mysql-bin

# 设置binlog格式为ROW(必须)
binlog-format=ROW

# 设置server-id(必须唯一)
server-id=1

# binlog存储位置
log-bin=/data/mysql/mysql-bin

# binlog保留天数
expire_logs_days=7

# 每次事务提交都写入binlog
sync_binlog=1

# 开启GTID(可选,推荐)
gtid_mode=ON
enforce_gtid_consistency=ON

创建Canal用户

# 登录MySQL
$ mysql -uroot -p

# 创建Canal用户
mysql> CREATE USER ‘canal’@’%’ IDENTIFIED BY ‘canal123’;

# 授予必要权限
mysql> GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO ‘canal’@’%’;

# 刷新权限
mysql> FLUSH PRIVILEGES;

# 验证权限
mysql> SHOW GRANTS FOR ‘canal’@’%’;

输出示例如下:
+—————————————————-+
| Grants for canal@% |
+—————————————————-+
| GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO ‘canal’@’%’ |
+—————————————————-+

验证MySQL配置

# 检查binlog是否开启
mysql> SHOW VARIABLES LIKE ‘log_bin%’;

输出示例如下:
+———————————+—————————–+
| Variable_name | Value |
+———————————+—————————–+
| log_bin | ON |
| log_bin_basename | /data/mysql/mysql-bin |
| log_bin_index | /data/mysql/mysql-bin.index |
+———————————+—————————–+

# 检查binlog格式
mysql> SHOW VARIABLES LIKE ‘binlog_format’;

输出示例如下:
+—————+——-+
| Variable_name | Value |
+—————+——-+
| binlog_format | ROW |
+—————+——-+

# 检查server-id
mysql> SHOW VARIABLES LIKE ‘server_id’;

输出示例如下:
+—————+——-+
| Variable_name | Value |
+—————+——-+
| server_id | 1 |
+—————+——-+

生产环境建议:binlog格式必须设置为ROW,Canal才能正确解析数据变更。server-id在主从复制环境中必须唯一,建议设置为IP地址最后一段。sync_binlog设置为1确保数据安全,但会略微影响性能。

4. Canal安装部署

Canal安装部署包括服务端部署和管理端部署。风哥提示:生产环境建议使用Canal Admin进行集中管理。

步骤1:解压安装包

# 创建安装目录
$ mkdir -p /fgeudb/canal/{server,adapter,admin}

# 解压Canal Server
$ cd /fgeudb/canal/server
$ tar -zxvf /fgeudb/software/canal/canal.deployer-1.1.8.tar.gz

# 解压Canal Adapter
$ cd /fgeudb/canal/adapter
$ tar -zxvf /fgeudb/software/canal/canal.adapter-1.1.8.tar.gz

# 解压Canal Admin
$ cd /fgeudb/canal/admin
$ tar -zxvf /fgeudb/software/canal/canal.admin-1.1.8.tar.gz

# 查看目录结构
$ ls -la /fgeudb/canal/server/
total 16
drwxr-xr-x 2 root root 4096 Apr 4 10:00 bin
drwxr-xr-x 2 root root 4096 Apr 4 10:00 conf
drwxr-xr-x 2 root root 4096 Apr 4 10:00 lib
drwxr-xr-x 2 root root 4096 Apr 4 10:00 logs

步骤2:配置Canal Server

# 编辑canal.properties
$ vi /fgeudb/canal/server/conf/canal.properties

# 修改以下配置
canal.id = 1
canal.ip = 192.168.1.51
canal.port = 11111
canal.zkServers = 192.168.1.51:2181
canal.zookeeper.flush.period = 1000
canal.withoutNetty = false
canal.serverMode = tcp
canal.destinations = example
canal.auto.scan = true
canal.auto.scan.interval = 5
canal.instance.global.spring.xml = classpath:spring/default-instance.xml

步骤3:配置Instance实例

# 复制实例配置模板
$ cd /fgeudb/canal/server/conf
$ cp -r example fgedu_instance

# 编辑实例配置
$ vi /fgeudb/canal/server/conf/fgedu_instance/instance.properties

# 修改以下配置
canal.instance.master.address = 192.168.1.51:3306
canal.instance.master.journal.name =
canal.instance.master.position =
canal.instance.master.timestamp =
canal.instance.master.gtid =

canal.instance.dbUsername = canal
canal.instance.dbPassword = canal123
canal.instance.connectionCharset = UTF-8
canal.instance.defaultDatabaseName = fgedu_db
canal.instance.filter.regex = fgedu_db\\..*
canal.instance.filter.black.regex = .*\\..*_bak

# 配置MQ输出(可选)
canal.mq.topic = canal_fgedu
canal.mq.partition = 0

步骤4:启动Canal Server

# 启动Canal Server
$ cd /fgeudb/canal/server
$ sh bin/startup.sh

# 查看启动日志
$ tail -f logs/canal/canal.log

输出示例如下:
2026-04-04 10:00:00.000 [main] INFO com.alibaba.otter.canal.deployer.CanalLauncher – ## start the canal server.
2026-04-04 10:00:00.100 [main] INFO com.alibaba.otter.canal.deployer.CanalController – ## start the canal server[192.168.1.51:11111]
2026-04-04 10:00:00.200 [main] INFO com.alibaba.otter.canal.deployer.CanalLauncher – ## the canal server is running now ……

# 检查进程
$ ps -ef | grep canal

输出示例如下:
root 12345 1 2 10:00 pts/0 00:00:05 /usr/bin/java -server -Xms2g -Xmx2g -XX:+UseG1GC -classpath /fgeudb/canal/server/lib/* com.alibaba.otter.canal.deployer.CanalLauncher

# 检查端口
$ netstat -tlnp | grep 11111

输出示例如下:
tcp 0 0 192.168.1.51:11111 0.0.0.0:* LISTEN 12345/java

5. Canal配置详解

Canal配置包括全局配置和实例配置两部分,需要根据生产环境需求进行调整。更多学习教程公众号风哥教程itpux_com

canal.properties核心配置

# Canal服务端基本配置
canal.id = 1 # Canal实例ID,HA环境需唯一
canal.ip = 192.168.1.51 # 绑定IP地址
canal.port = 11111 # Canal服务端口
canal.metrics.pull.port = 11112 # metrics监控端口

# ZooKeeper配置(HA模式必需)
canal.zkServers = 192.168.1.51:2181,192.168.1.52:2181,192.168.1.53:2181
canal.zookeeper.flush.period = 1000 # ZK刷新周期

# 运行模式
canal.serverMode = tcp # tcp/kafka/rocketMQ/pulsar

# 目标实例列表
canal.destinations = fgedu_instance,order_instance

# 自动扫描配置
canal.auto.scan = true # 自动扫描instance目录
canal.auto.scan.interval = 5 # 扫描间隔(秒)

instance.properties核心配置

# MySQL源端配置
canal.instance.master.address = 192.168.1.51:3306
canal.instance.master.journal.name = mysql-bin.000001
canal.instance.master.position = 4
canal.instance.master.timestamp =

# GTID模式配置
canal.instance.gtidon = true
canal.instance.master.gtid = 3a4e7b00-1234-5678-90ab-cdef12345678:1-100

# 数据库连接配置
canal.instance.dbUsername = canal
canal.instance.dbPassword = canal123
canal.instance.connectionCharset = UTF-8
canal.instance.defaultDatabaseName = fgedu_db

# 数据过滤规则
canal.instance.filter.regex = fgedu_db\\.order_.*,fgedu_db\\.user_.*
canal.instance.filter.black.regex = .*\\.log_.*,.*\\.bak_.*

# 心跳配置
canal.instance.detectSql = SELECT 1
canal.instance.heartbeatInterval = 3

# 批处理配置
canal.instance.memory.buffer.size = 16384
canal.instance.memory.buffer.memunit = 1024

生产环境建议:filter.regex使用正则表达式精确匹配需要同步的表,避免同步不需要的数据。memory.buffer.size根据服务器内存调整,建议设置为16384或更大。heartbeatInterval设置为3秒确保连接稳定性。

6. Canal数据同步实战

Canal支持多种数据同步场景,包括TCP直连、Kafka消息队列等。from:www.itpux.com

场景一:Java客户端消费数据

# 创建Maven项目,添加依赖
<dependency>
<groupId>com.alibaba.otter</groupId>
<artifactId>canal.client</artifactId>
<version>1.1.8</version>
</dependency>

# Java客户端代码示例
import com.alibaba.otter.canal.client.CanalConnector;
import com.alibaba.otter.canal.client.CanalConnectors;
import com.alibaba.otter.canal.protocol.Message;
import com.alibaba.otter.canal.protocol.CanalEntry.*;

public class CanalClient {
public static void main(String[] args) {
CanalConnector connector = CanalConnectors.newSingleConnector(
new InetSocketAddress(“192.168.1.51”, 11111),
“fgedu_instance”, “”, “”);

try {
connector.connect();
connector.subscribe(“fgedu_db\\..*”);
connector.rollback();

while (true) {
Message message = connector.getWithoutAck(100);
long batchId = message.getId();
int size = message.getEntries().size();

if (batchId == -1 || size == 0) {
Thread.sleep(1000);
} else {
printEntry(message.getEntries());
}
connector.ack(batchId);
}
} finally {
connector.disconnect();
}
}
}

场景二:同步到Kafka

# 修改canal.properties配置MQ模式
$ vi /fgeudb/canal/server/conf/canal.properties

canal.serverMode = kafka
canal.mq.servers = 192.168.1.51:9092,192.168.1.52:9092,192.168.1.53:9092
canal.mq.retries = 3
canal.mq.batchSize = 16384
canal.mq.maxRequestSize = 1048576
canal.mq.lingerMs = 100
canal.mq.bufferMemory = 33554432
canal.mq.acks = all

# 修改instance.properties配置Topic
$ vi /fgeudb/canal/server/conf/fgedu_instance/instance.properties

canal.mq.topic = canal_fgedu
canal.mq.partition = 0
canal.mq.partitionsNum = 3
canal.mq.partitionHash = .*\\..*:$pk$

# 重启Canal Server
$ sh /fgeudb/canal/server/bin/stop.sh
$ sh /fgeudb/canal/server/bin/startup.sh

场景三:使用Canal Adapter同步到目标库

# 配置Adapter
$ vi /fgeudb/canal/adapter/conf/application.yml

server:
port: 8081

canal.conf:
mode: tcp
canalServerHost: 192.168.1.51:11111
batchSize: 500
syncBatchSize: 1000

srcDataSources:
defaultDS:
url: jdbc:mysql://192.168.1.51:3306/fgedu_db?useSSL=false
username: canal
password: canal123

canalAdapters:
– instance: fgedu_instance
groups:
– groupId: g1
outerAdapters:
– name: logger
– name: rdb
key: mysql1
properties:
jdbc.driverClassName: com.mysql.cj.jdbc.Driver
jdbc.url: jdbc:mysql://192.168.1.52:3306/fgedu_db_target
jdbc.username: root
jdbc.password: root123

# 配置表映射
$ vi /fgeudb/canal/adapter/conf/rdb/fgedu_order.yml

dataSourceKey: defaultDS
destination: fgedu_instance
groupId: g1
outerAdapterKey: mysql1
concurrent: true

dbMapping:
database: fgedu_db
table: order_info
targetTable: order_info
targetPk:
id: id
mapAll: true
etlCondition: “where create_time >= ‘2026-01-01′”

# 启动Adapter
$ cd /fgeudb/canal/adapter
$ sh bin/startup.sh

# 查看日志
$ tail -f logs/adapter/adapter.log

输出示例如下:
2026-04-04 10:00:00.000 [main] INFO c.a.o.canal.adapter.launcher.CanalAdapterApplication – ## start the canal adapter.
2026-04-04 10:00:00.100 [main] INFO c.a.o.canal.adapter.launcher.CanalAdapterApplication – ## the canal adapter is running now ……

7. Canal高可用配置

生产环境需要配置Canal高可用,确保服务稳定性。

步骤1:ZooKeeper集群部署

# 在三台服务器上部署ZooKeeper
# 服务器1: 192.168.1.51
# 服务器2: 192.168.1.52
# 服务器3: 192.168.1.53

# 配置zoo.cfg
$ vi /fgeudb/zookeeper/conf/zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/fgeudb/zookeeper/data
clientPort=2181
server.1=192.168.1.51:2888:3888
server.2=192.168.1.52:2888:3888
server.3=192.168.1.53:2888:3888

# 创建myid文件
$ echo “1” > /fgeudb/zookeeper/data/myid

# 启动ZooKeeper
$ /fgeudb/zookeeper/bin/zkServer.sh start

# 检查状态
$ /fgeudb/zookeeper/bin/zkServer.sh status

输出示例如下:
Mode: leader

步骤2:Canal Server HA配置

# 在两台服务器上配置Canal Server
# 服务器1: 192.168.1.51 (canal.id=1)
# 服务器2: 192.168.1.52 (canal.id=2)

# 编辑canal.properties
$ vi /fgeudb/canal/server/conf/canal.properties

canal.id = 1 # 第二台设置为2
canal.ip = 192.168.1.51 # 绑定各自IP
canal.port = 11111
canal.zkServers = 192.168.1.51:2181,192.168.1.52:2181,192.168.1.53:2181
canal.instance.global.spring.xml = classpath:spring/default-instance.xml

# 启动两台Canal Server
$ sh /fgeudb/canal/server/bin/startup.sh

# 查看HA状态
$ tail -f /fgeudb/canal/server/logs/canal/canal.log

输出示例如下:
2026-04-04 10:00:00.000 [main] INFO c.a.o.canal.deployer.CanalController – ## start the canal server[192.168.1.51:11111]
2026-04-04 10:00:00.100 [main] INFO c.a.o.canal.deployer.CanalLauncher – ## the canal server is running now ……
2026-04-04 10:00:00.200 [main] INFO c.a.o.canal.common.zookeeper.ZkClientx – zookeeper state changed (SyncConnected)

步骤3:监控Canal运行状态

# 使用Canal Admin管理
# 访问 http://192.168.1.51:8089
# 默认账号: admin
# 默认密码: 123456

# 通过ZooKeeper查看状态
$ /fgeudb/zookeeper/bin/zkCli.sh

[zk: localhost:2181(CONNECTED) 0] ls /otter/canal/destinations
[fgedu_instance]

[zk: localhost:2181(CONNECTED) 1] ls /otter/canal/destinations/fgedu_instance
[cluster, running]

[zk: localhost:2181(CONNECTED) 2] get /otter/canal/destinations/fgedu_instance/running

输出示例如下:
{“active”:true,”address”:”192.168.1.51:11111″}

生产环境建议:Canal HA模式下,同一时刻只有一个Server处于active状态,其他Server处于standby状态。当active Server宕机时,standby Server自动接管。建议配置监控告警,及时发现和处理故障。定期检查binlog位置,确保数据同步不丢失。

本文由风哥教程整理发布,仅用于学习测试使用,转载注明出处:http://www.fgedu.net.cn/10327.html

联系我们

在线咨询:点击这里给我发消息

微信号:itpux-com

工作日:9:30-18:30,节假日休息