内容简介:本文风哥教程参考MariaDB官方文档MariaDB Server、Partitioning等章节,详细讲解MariaDB分区表的设计与大数据量优化,包括分区类型、创建、管理和性能优化。
Part01-基础概念与理论知识
1.1 分区表基本概念
分区表是将一个大表分成多个较小的、可管理的部分,每个部分称为一个分区。分区表在逻辑上是一个表,但在物理上存储为多个文件。分区表可以提高查询性能、简化数据管理、提高可用性。
1.2 分区类型与特点
- 范围分区(RANGE):
- 根据列值的范围进行分区
- 适合按时间、ID等连续值分区
- 语法:PARTITION BY RANGE (expression)
- 列表分区(LIST):
- 根据列值的列表进行分区
- 适合按类别、状态等离散值分区
- 语法:PARTITION BY LIST (expression)
- 哈希分区(HASH):
- 根据列值的哈希值进行分区
- 适合均匀分布数据
- 语法:PARTITION BY HASH (expression)
- 键分区(KEY):
- 类似于哈希分区,但使用MySQL的内部哈希函数
- 适合多个列的分区
- 语法:PARTITION BY KEY (columns)
- 子分区(SUBPARTITION):
- 在分区的基础上再进行分区
- 可以结合不同的分区类型
- 语法:SUBPARTITION BY [HASH|LIST|KEY] (expression)
1.3 分区表优势与适用场景
- 优势:
- 提高查询性能:只扫描相关分区
- 简化数据管理:可以单独管理每个分区
- 提高可用性:一个分区故障不影响其他分区
- 便于数据归档:可以快速删除旧分区
- 提高并行性:可以并行处理多个分区
- 适用场景:
- 大表(超过1000万行)
- 时间序列数据
- 需要按时间归档的数据
- 查询主要基于分区键
- 需要快速删除历史数据
更多视频教程www.fgedu.net.cn
Part02-生产环境规划与建议
2.1 分区表设计原则
风哥提示:合理的分区表设计可以显著提高系统性能和可维护性。
- 选择合适的分区类型:根据数据特点选择合适的分区类型
- 合理的分区键:选择查询频率高、分布均匀的列作为分区键
- 适当的分区数量:避免过多或过少的分区
- 考虑数据增长:预留足够的分区空间
- 保持分区大小均匀:避免分区大小差异过大
2.2 性能优化建议
- 使用分区修剪:确保查询条件包含分区键
- 合理设置分区大小:每个分区大小适中(建议1-5GB)
- 避免跨分区查询:尽量让查询只涉及少数分区
- 使用本地分区索引:提高查询性能
- 定期维护分区:清理碎片,优化性能
2.3 生产环境最佳实践
- 在开发环境充分测试分区表
- 监控分区表的使用情况
- 设置合理的分区策略
- 定期备份分区表
- 使用事务确保数据一致性
学习交流加群风哥微信: itpux-com
Part03-生产环境项目实施方案
3.1 分区表创建与管理
更多学习教程公众号风哥教程itpux_com
# 创建范围分区表
MariaDB [fgedudb]> CREATE TABLE fgedu_logs_range (
-> id INT PRIMARY KEY AUTO_INCREMENT,
-> user_id INT,
-> action VARCHAR(50),
-> created_at DATETIME,
-> details TEXT
-> ) ENGINE=InnoDB
-> PARTITION BY RANGE (YEAR(created_at)) (
-> PARTITION p2022 VALUES LESS THAN (2023),
-> PARTITION p2023 VALUES LESS THAN (2024),
-> PARTITION p2024 VALUES LESS THAN (2025),
-> PARTITION p2025 VALUES LESS THAN (2026),
-> PARTITION pfuture VALUES LESS THAN MAXVALUE
-> );
Query OK, 0 rows affected (0.01 sec)
# 创建列表分区表
MariaDB [fgedudb]> CREATE TABLE fgedu_orders_list (
-> id INT PRIMARY KEY AUTO_INCREMENT,
-> user_id INT,
-> total_amount DECIMAL(10,2),
-> status VARCHAR(20),
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY LIST (status) (
-> PARTITION p_pending VALUES IN (‘pending’),
-> PARTITION p_processing VALUES IN (‘processing’),
-> PARTITION p_completed VALUES IN (‘completed’),
-> PARTITION p_cancelled VALUES IN (‘cancelled’)
-> );
Query OK, 0 rows affected (0.01 sec)
# 创建哈希分区表
MariaDB [fgedudb]> CREATE TABLE fgedu_users_hash (
-> id INT PRIMARY KEY,
-> name VARCHAR(50),
-> email VARCHAR(100),
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY HASH (id)
-> PARTITIONS 4;
Query OK, 0 rows affected (0.01 sec)
# 创建键分区表
MariaDB [fgedudb]> CREATE TABLE fgedu_products_key (
-> id INT PRIMARY KEY,
-> name VARCHAR(100),
-> category VARCHAR(50),
-> price DECIMAL(10,2),
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY KEY (id, category)
-> PARTITIONS 8;
Query OK, 0 rows affected (0.01 sec)
# 查看分区表信息
MariaDB [fgedudb]> SHOW CREATE TABLE fgedu_logs_range\G
*************************** 1. row ***************************
Table: fgedu_logs_range
Create Table: CREATE TABLE `fgedu_logs_range` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) DEFAULT NULL,
`action` varchar(50) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`details` text DEFAULT NULL,
PRIMARY KEY (`id`,`created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
/*!50100 PARTITION BY RANGE (YEAR(created_at))
(PARTITION p2022 VALUES LESS THAN (2023) ENGINE = InnoDB,
PARTITION p2023 VALUES LESS THAN (2024) ENGINE = InnoDB,
PARTITION p2024 VALUES LESS THAN (2025) ENGINE = InnoDB,
PARTITION p2025 VALUES LESS THAN (2026) ENGINE = InnoDB,
PARTITION pfuture VALUES LESS THAN MAXVALUE ENGINE = InnoDB) */
# 查看分区状态
MariaDB [fgedudb]> SELECT * FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_logs_range’;
+—————+————–+—————-+——————+————+—————-+—————–+——————-+——————+——————-+——————-+——————+
| TABLE_SCHEMA | TABLE_NAME | PARTITION_NAME | SUBPARTITION_NAME | PARTITION_ORDINAL_POSITION | SUBPARTITION_ORDINAL_POSITION | PARTITION_METHOD | SUBPARTITION_METHOD | PARTITION_EXPRESSION | SUBPARTITION_EXPRESSION | PARTITION_DESCRIPTION | TABLE_ROWS |
+—————+————–+—————-+——————+————+—————-+—————–+——————-+——————+——————-+——————-+——————+
| fgedudb | fgedu_logs_range | p2022 | NULL | 1 | NULL | RANGE | NULL | YEAR(created_at) | NULL | 2023 | 0 |
| fgedudb | fgedu_logs_range | p2023 | NULL | 2 | NULL | RANGE | NULL | YEAR(created_at) | NULL | 2024 | 0 |
| fgedudb | fgedu_logs_range | p2024 | NULL | 3 | NULL | RANGE | NULL | YEAR(created_at) | NULL | 2025 | 0 |
| fgedudb | fgedu_logs_range | p2025 | NULL | 4 | NULL | RANGE | NULL | YEAR(created_at) | NULL | 2026 | 0 |
| fgedudb | fgedu_logs_range | pfuture | NULL | 5 | NULL | RANGE | NULL | YEAR(created_at) | NULL | MAXVALUE | 0 |
+—————+————–+—————-+——————+————+—————-+—————–+——————-+——————+——————-+——————-+——————+
MariaDB [fgedudb]> CREATE TABLE fgedu_logs_range (
-> id INT PRIMARY KEY AUTO_INCREMENT,
-> user_id INT,
-> action VARCHAR(50),
-> created_at DATETIME,
-> details TEXT
-> ) ENGINE=InnoDB
-> PARTITION BY RANGE (YEAR(created_at)) (
-> PARTITION p2022 VALUES LESS THAN (2023),
-> PARTITION p2023 VALUES LESS THAN (2024),
-> PARTITION p2024 VALUES LESS THAN (2025),
-> PARTITION p2025 VALUES LESS THAN (2026),
-> PARTITION pfuture VALUES LESS THAN MAXVALUE
-> );
Query OK, 0 rows affected (0.01 sec)
# 创建列表分区表
MariaDB [fgedudb]> CREATE TABLE fgedu_orders_list (
-> id INT PRIMARY KEY AUTO_INCREMENT,
-> user_id INT,
-> total_amount DECIMAL(10,2),
-> status VARCHAR(20),
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY LIST (status) (
-> PARTITION p_pending VALUES IN (‘pending’),
-> PARTITION p_processing VALUES IN (‘processing’),
-> PARTITION p_completed VALUES IN (‘completed’),
-> PARTITION p_cancelled VALUES IN (‘cancelled’)
-> );
Query OK, 0 rows affected (0.01 sec)
# 创建哈希分区表
MariaDB [fgedudb]> CREATE TABLE fgedu_users_hash (
-> id INT PRIMARY KEY,
-> name VARCHAR(50),
-> email VARCHAR(100),
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY HASH (id)
-> PARTITIONS 4;
Query OK, 0 rows affected (0.01 sec)
# 创建键分区表
MariaDB [fgedudb]> CREATE TABLE fgedu_products_key (
-> id INT PRIMARY KEY,
-> name VARCHAR(100),
-> category VARCHAR(50),
-> price DECIMAL(10,2),
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY KEY (id, category)
-> PARTITIONS 8;
Query OK, 0 rows affected (0.01 sec)
# 查看分区表信息
MariaDB [fgedudb]> SHOW CREATE TABLE fgedu_logs_range\G
*************************** 1. row ***************************
Table: fgedu_logs_range
Create Table: CREATE TABLE `fgedu_logs_range` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) DEFAULT NULL,
`action` varchar(50) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`details` text DEFAULT NULL,
PRIMARY KEY (`id`,`created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
/*!50100 PARTITION BY RANGE (YEAR(created_at))
(PARTITION p2022 VALUES LESS THAN (2023) ENGINE = InnoDB,
PARTITION p2023 VALUES LESS THAN (2024) ENGINE = InnoDB,
PARTITION p2024 VALUES LESS THAN (2025) ENGINE = InnoDB,
PARTITION p2025 VALUES LESS THAN (2026) ENGINE = InnoDB,
PARTITION pfuture VALUES LESS THAN MAXVALUE ENGINE = InnoDB) */
# 查看分区状态
MariaDB [fgedudb]> SELECT * FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_logs_range’;
+—————+————–+—————-+——————+————+—————-+—————–+——————-+——————+——————-+——————-+——————+
| TABLE_SCHEMA | TABLE_NAME | PARTITION_NAME | SUBPARTITION_NAME | PARTITION_ORDINAL_POSITION | SUBPARTITION_ORDINAL_POSITION | PARTITION_METHOD | SUBPARTITION_METHOD | PARTITION_EXPRESSION | SUBPARTITION_EXPRESSION | PARTITION_DESCRIPTION | TABLE_ROWS |
+—————+————–+—————-+——————+————+—————-+—————–+——————-+——————+——————-+——————-+——————+
| fgedudb | fgedu_logs_range | p2022 | NULL | 1 | NULL | RANGE | NULL | YEAR(created_at) | NULL | 2023 | 0 |
| fgedudb | fgedu_logs_range | p2023 | NULL | 2 | NULL | RANGE | NULL | YEAR(created_at) | NULL | 2024 | 0 |
| fgedudb | fgedu_logs_range | p2024 | NULL | 3 | NULL | RANGE | NULL | YEAR(created_at) | NULL | 2025 | 0 |
| fgedudb | fgedu_logs_range | p2025 | NULL | 4 | NULL | RANGE | NULL | YEAR(created_at) | NULL | 2026 | 0 |
| fgedudb | fgedu_logs_range | pfuture | NULL | 5 | NULL | RANGE | NULL | YEAR(created_at) | NULL | MAXVALUE | 0 |
+—————+————–+—————-+——————+————+—————-+—————–+——————-+——————+——————-+——————-+——————+
3.2 分区维护与优化
# 添加分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> ADD PARTITION (
-> PARTITION p2026 VALUES LESS THAN (2027)
-> );
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 拆分分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> REORGANIZE PARTITION p2023 INTO (
-> PARTITION p2023_q1 VALUES LESS THAN (20230401),
-> PARTITION p2023_q2 VALUES LESS THAN (20230701),
-> PARTITION p2023_q3 VALUES LESS THAN (20231001),
-> PARTITION p2023_q4 VALUES LESS THAN (20240101)
-> );
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 合并分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> REORGANIZE PARTITION p2023_q1, p2023_q2, p2023_q3, p2023_q4 INTO (
-> PARTITION p2023 VALUES LESS THAN (2024)
-> );
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 删除分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> DROP PARTITION p2022;
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 截断分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> TRUNCATE PARTITION p2023;
Query OK, 0 rows affected (0.01 sec)
# 优化分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> OPTIMIZE PARTITION p2024, p2025;
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 重建分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> REBUILD PARTITION p2024, p2025;
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 分析分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> ANALYZE PARTITION p2024, p2025;
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> ADD PARTITION (
-> PARTITION p2026 VALUES LESS THAN (2027)
-> );
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 拆分分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> REORGANIZE PARTITION p2023 INTO (
-> PARTITION p2023_q1 VALUES LESS THAN (20230401),
-> PARTITION p2023_q2 VALUES LESS THAN (20230701),
-> PARTITION p2023_q3 VALUES LESS THAN (20231001),
-> PARTITION p2023_q4 VALUES LESS THAN (20240101)
-> );
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 合并分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> REORGANIZE PARTITION p2023_q1, p2023_q2, p2023_q3, p2023_q4 INTO (
-> PARTITION p2023 VALUES LESS THAN (2024)
-> );
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 删除分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> DROP PARTITION p2022;
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 截断分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> TRUNCATE PARTITION p2023;
Query OK, 0 rows affected (0.01 sec)
# 优化分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> OPTIMIZE PARTITION p2024, p2025;
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 重建分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> REBUILD PARTITION p2024, p2025;
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 分析分区
MariaDB [fgedudb]> ALTER TABLE fgedu_logs_range
-> ANALYZE PARTITION p2024, p2025;
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
3.3 分区表监控与故障处理
# 监控分区表大小
MariaDB [fgedudb]> SELECT
-> table_schema,
-> table_name,
-> partition_name,
-> ROUND(SUM(data_length + index_length) / 1024 / 1024, 2) AS size_mb
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_logs_range’
-> GROUP BY table_schema, table_name, partition_name;
+—————+————–+—————-+———+
| table_schema | table_name | partition_name | size_mb |
+—————+————–+—————-+———+
| fgedudb | fgedu_logs_range | p2023 | 0.00 |
| fgedudb | fgedu_logs_range | p2024 | 0.00 |
| fgedudb | fgedu_logs_range | p2025 | 0.00 |
| fgedudb | fgedu_logs_range | p2026 | 0.00 |
| fgedudb | fgedu_logs_range | pfuture | 0.00 |
+—————+————–+—————-+———+
# 监控分区表使用情况
MariaDB [fgedudb]> SELECT
-> table_schema,
-> table_name,
-> partition_name,
-> table_rows
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_logs_range’;
+—————+————–+—————-+————+
| table_schema | table_name | partition_name | table_rows |
+—————+————–+—————-+————+
| fgedudb | fgedu_logs_range | p2023 | 0 |
| fgedudb | fgedu_logs_range | p2024 | 0 |
| fgedudb | fgedu_logs_range | p2025 | 0 |
| fgedudb | fgedu_logs_range | p2026 | 0 |
| fgedudb | fgedu_logs_range | pfuture | 0 |
+—————+————–+—————-+————+
# 故障处理:分区表修复
MariaDB [fgedudb]> REPAIR TABLE fgedu_logs_range PARTITION p2024;
+———————+——–+———-+———-+
| Table | Op | Msg_type | Msg_text |
+———————+——–+———-+———-+
| fgedudb.fgedu_logs_range | repair | status | OK |
+———————+——–+———-+———-+
# 故障处理:分区表恢复
# 1. 备份分区数据
MariaDB [fgedudb]> SELECT * FROM fgedu_logs_range PARTITION (p2024) INTO OUTFILE ‘/mariadb/backup/p2024_data.sql’;
# 2. 恢复分区数据
MariaDB [fgedudb]> LOAD DATA INFILE ‘/mariadb/backup/p2024_data.sql’ INTO TABLE fgedu_logs_range PARTITION (p2024);
# 故障处理:分区表迁移
# 1. 创建新表
MariaDB [fgedudb]> CREATE TABLE fgedu_logs_new LIKE fgedu_logs_range;
# 2. 复制数据
MariaDB [fgedudb]> INSERT INTO fgedu_logs_new SELECT * FROM fgedu_logs_range;
# 3. 重命名表
MariaDB [fgedudb]> RENAME TABLE fgedu_logs_range TO fgedu_logs_old, fgedu_logs_new TO fgedu_logs_range;
MariaDB [fgedudb]> SELECT
-> table_schema,
-> table_name,
-> partition_name,
-> ROUND(SUM(data_length + index_length) / 1024 / 1024, 2) AS size_mb
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_logs_range’
-> GROUP BY table_schema, table_name, partition_name;
+—————+————–+—————-+———+
| table_schema | table_name | partition_name | size_mb |
+—————+————–+—————-+———+
| fgedudb | fgedu_logs_range | p2023 | 0.00 |
| fgedudb | fgedu_logs_range | p2024 | 0.00 |
| fgedudb | fgedu_logs_range | p2025 | 0.00 |
| fgedudb | fgedu_logs_range | p2026 | 0.00 |
| fgedudb | fgedu_logs_range | pfuture | 0.00 |
+—————+————–+—————-+———+
# 监控分区表使用情况
MariaDB [fgedudb]> SELECT
-> table_schema,
-> table_name,
-> partition_name,
-> table_rows
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_logs_range’;
+—————+————–+—————-+————+
| table_schema | table_name | partition_name | table_rows |
+—————+————–+—————-+————+
| fgedudb | fgedu_logs_range | p2023 | 0 |
| fgedudb | fgedu_logs_range | p2024 | 0 |
| fgedudb | fgedu_logs_range | p2025 | 0 |
| fgedudb | fgedu_logs_range | p2026 | 0 |
| fgedudb | fgedu_logs_range | pfuture | 0 |
+—————+————–+—————-+————+
# 故障处理:分区表修复
MariaDB [fgedudb]> REPAIR TABLE fgedu_logs_range PARTITION p2024;
+———————+——–+———-+———-+
| Table | Op | Msg_type | Msg_text |
+———————+——–+———-+———-+
| fgedudb.fgedu_logs_range | repair | status | OK |
+———————+——–+———-+———-+
# 故障处理:分区表恢复
# 1. 备份分区数据
MariaDB [fgedudb]> SELECT * FROM fgedu_logs_range PARTITION (p2024) INTO OUTFILE ‘/mariadb/backup/p2024_data.sql’;
# 2. 恢复分区数据
MariaDB [fgedudb]> LOAD DATA INFILE ‘/mariadb/backup/p2024_data.sql’ INTO TABLE fgedu_logs_range PARTITION (p2024);
# 故障处理:分区表迁移
# 1. 创建新表
MariaDB [fgedudb]> CREATE TABLE fgedu_logs_new LIKE fgedu_logs_range;
# 2. 复制数据
MariaDB [fgedudb]> INSERT INTO fgedu_logs_new SELECT * FROM fgedu_logs_range;
# 3. 重命名表
MariaDB [fgedudb]> RENAME TABLE fgedu_logs_range TO fgedu_logs_old, fgedu_logs_new TO fgedu_logs_range;
学习交流加群风哥QQ113257174
Part04-生产案例与实战讲解
4.1 范围分区实战
# 场景:日志数据按时间分区
MariaDB [fgedudb]> CREATE TABLE fgedu_application_logs (
-> id INT PRIMARY KEY AUTO_INCREMENT,
-> application VARCHAR(50),
-> level VARCHAR(20),
-> message TEXT,
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY RANGE (TO_DAYS(created_at)) (
-> PARTITION p202301 VALUES LESS THAN (TO_DAYS(‘2023-02-01’)),
-> PARTITION p202302 VALUES LESS THAN (TO_DAYS(‘2023-03-01’)),
-> PARTITION p202303 VALUES LESS THAN (TO_DAYS(‘2023-04-01’)),
-> PARTITION p202304 VALUES LESS THAN (TO_DAYS(‘2023-05-01’)),
-> PARTITION p202305 VALUES LESS THAN (TO_DAYS(‘2023-06-01’)),
-> PARTITION p202306 VALUES LESS THAN (TO_DAYS(‘2023-07-01’)),
-> PARTITION p202307 VALUES LESS THAN (TO_DAYS(‘2023-08-01’)),
-> PARTITION p202308 VALUES LESS THAN (TO_DAYS(‘2023-09-01’)),
-> PARTITION p202309 VALUES LESS THAN (TO_DAYS(‘2023-10-01’)),
-> PARTITION p202310 VALUES LESS THAN (TO_DAYS(‘2023-11-01’)),
-> PARTITION p202311 VALUES LESS THAN (TO_DAYS(‘2023-12-01’)),
-> PARTITION p202312 VALUES LESS THAN (TO_DAYS(‘2024-01-01’)),
-> PARTITION pfuture VALUES LESS THAN MAXVALUE
-> );
Query OK, 0 rows affected (0.02 sec)
# 插入测试数据
MariaDB [fgedudb]> DELIMITER //
MariaDB [fgedudb]> CREATE PROCEDURE fgedu_insert_logs(IN p_count INT)
BEGIN
DECLARE i INT DEFAULT 1;
DECLARE v_date DATETIME;
WHILE i <= p_count DO
SET v_date = DATE_SUB(NOW(), INTERVAL FLOOR(RAND() * 365) DAY);
INSERT INTO fgedu_application_logs (application, level, message, created_at)
VALUES (
CONCAT(‘app_’, (i % 5) + 1),
ELT(FLOOR(RAND() * 5) + 1, ‘INFO’, ‘WARN’, ‘ERROR’, ‘DEBUG’, ‘FATAL’),
CONCAT(‘Log message ‘, i),
v_date
);
SET i = i + 1;
END WHILE;
END //
DELIMITER;
Query OK, 0 rows affected (0.00 sec)
# 插入10000条测试数据
MariaDB [fgedudb]> CALL fgedu_insert_logs(10000);
Query OK, 1 row affected (0.34 sec)
# 查看分区数据分布
MariaDB [fgedudb]> SELECT
-> partition_name,
-> table_rows
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_application_logs’
-> ORDER BY partition_name;
+—————-+————+
| partition_name | table_rows |
+—————-+————+
| p202301 | 832 |
| p202302 | 756 |
| p202303 | 843 |
| p202304 | 821 |
| p202305 | 867 |
| p202306 | 834 |
| p202307 | 812 |
| p202308 | 828 |
| p202309 | 835 |
| p202310 | 845 |
| p202311 | 823 |
| p202312 | 804 |
| pfuture | 0 |
+—————-+————+
# 测试分区修剪
MariaDB [fgedudb]> EXPLAIN SELECT * FROM fgedu_application_logs
-> WHERE created_at BETWEEN ‘2023-03-01’ AND ‘2023-03-31’\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: fgedu_application_logs
partitions: p202303
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 843
filtered: 100.00
Extra: Using where
# 测试查询性能
MariaDB [fgedudb]> SELECT COUNT(*) FROM fgedu_application_logs
-> WHERE created_at BETWEEN ‘2023-03-01’ AND ‘2023-03-31’;
+———-+
| COUNT(*) |
+———-+
| 843 |
+———-+
1 row in set (0.01 sec)
# 删除旧分区
MariaDB [fgedudb]> ALTER TABLE fgedu_application_logs
-> DROP PARTITION p202301, p202302;
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 添加新分区
MariaDB [fgedudb]> ALTER TABLE fgedu_application_logs
-> ADD PARTITION (
-> PARTITION p202401 VALUES LESS THAN (TO_DAYS(‘2024-02-01’)),
-> PARTITION p202402 VALUES LESS THAN (TO_DAYS(‘2024-03-01’)),
-> PARTITION p202403 VALUES LESS THAN (TO_DAYS(‘2024-04-01’))
-> );
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
MariaDB [fgedudb]> CREATE TABLE fgedu_application_logs (
-> id INT PRIMARY KEY AUTO_INCREMENT,
-> application VARCHAR(50),
-> level VARCHAR(20),
-> message TEXT,
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY RANGE (TO_DAYS(created_at)) (
-> PARTITION p202301 VALUES LESS THAN (TO_DAYS(‘2023-02-01’)),
-> PARTITION p202302 VALUES LESS THAN (TO_DAYS(‘2023-03-01’)),
-> PARTITION p202303 VALUES LESS THAN (TO_DAYS(‘2023-04-01’)),
-> PARTITION p202304 VALUES LESS THAN (TO_DAYS(‘2023-05-01’)),
-> PARTITION p202305 VALUES LESS THAN (TO_DAYS(‘2023-06-01’)),
-> PARTITION p202306 VALUES LESS THAN (TO_DAYS(‘2023-07-01’)),
-> PARTITION p202307 VALUES LESS THAN (TO_DAYS(‘2023-08-01’)),
-> PARTITION p202308 VALUES LESS THAN (TO_DAYS(‘2023-09-01’)),
-> PARTITION p202309 VALUES LESS THAN (TO_DAYS(‘2023-10-01’)),
-> PARTITION p202310 VALUES LESS THAN (TO_DAYS(‘2023-11-01’)),
-> PARTITION p202311 VALUES LESS THAN (TO_DAYS(‘2023-12-01’)),
-> PARTITION p202312 VALUES LESS THAN (TO_DAYS(‘2024-01-01’)),
-> PARTITION pfuture VALUES LESS THAN MAXVALUE
-> );
Query OK, 0 rows affected (0.02 sec)
# 插入测试数据
MariaDB [fgedudb]> DELIMITER //
MariaDB [fgedudb]> CREATE PROCEDURE fgedu_insert_logs(IN p_count INT)
BEGIN
DECLARE i INT DEFAULT 1;
DECLARE v_date DATETIME;
WHILE i <= p_count DO
SET v_date = DATE_SUB(NOW(), INTERVAL FLOOR(RAND() * 365) DAY);
INSERT INTO fgedu_application_logs (application, level, message, created_at)
VALUES (
CONCAT(‘app_’, (i % 5) + 1),
ELT(FLOOR(RAND() * 5) + 1, ‘INFO’, ‘WARN’, ‘ERROR’, ‘DEBUG’, ‘FATAL’),
CONCAT(‘Log message ‘, i),
v_date
);
SET i = i + 1;
END WHILE;
END //
DELIMITER;
Query OK, 0 rows affected (0.00 sec)
# 插入10000条测试数据
MariaDB [fgedudb]> CALL fgedu_insert_logs(10000);
Query OK, 1 row affected (0.34 sec)
# 查看分区数据分布
MariaDB [fgedudb]> SELECT
-> partition_name,
-> table_rows
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_application_logs’
-> ORDER BY partition_name;
+—————-+————+
| partition_name | table_rows |
+—————-+————+
| p202301 | 832 |
| p202302 | 756 |
| p202303 | 843 |
| p202304 | 821 |
| p202305 | 867 |
| p202306 | 834 |
| p202307 | 812 |
| p202308 | 828 |
| p202309 | 835 |
| p202310 | 845 |
| p202311 | 823 |
| p202312 | 804 |
| pfuture | 0 |
+—————-+————+
# 测试分区修剪
MariaDB [fgedudb]> EXPLAIN SELECT * FROM fgedu_application_logs
-> WHERE created_at BETWEEN ‘2023-03-01’ AND ‘2023-03-31’\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: fgedu_application_logs
partitions: p202303
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 843
filtered: 100.00
Extra: Using where
# 测试查询性能
MariaDB [fgedudb]> SELECT COUNT(*) FROM fgedu_application_logs
-> WHERE created_at BETWEEN ‘2023-03-01’ AND ‘2023-03-31’;
+———-+
| COUNT(*) |
+———-+
| 843 |
+———-+
1 row in set (0.01 sec)
# 删除旧分区
MariaDB [fgedudb]> ALTER TABLE fgedu_application_logs
-> DROP PARTITION p202301, p202302;
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
# 添加新分区
MariaDB [fgedudb]> ALTER TABLE fgedu_application_logs
-> ADD PARTITION (
-> PARTITION p202401 VALUES LESS THAN (TO_DAYS(‘2024-02-01’)),
-> PARTITION p202402 VALUES LESS THAN (TO_DAYS(‘2024-03-01’)),
-> PARTITION p202403 VALUES LESS THAN (TO_DAYS(‘2024-04-01’))
-> );
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
4.2 列表分区实战
# 场景:订单数据按状态分区
MariaDB [fgedudb]> CREATE TABLE fgedu_orders_list_part (
-> id INT PRIMARY KEY AUTO_INCREMENT,
-> user_id INT,
-> total_amount DECIMAL(10,2),
-> status VARCHAR(20),
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY LIST (status) (
-> PARTITION p_pending VALUES IN (‘pending’),
-> PARTITION p_processing VALUES IN (‘processing’),
-> PARTITION p_completed VALUES IN (‘completed’),
-> PARTITION p_cancelled VALUES IN (‘cancelled’),
-> PARTITION p_refunded VALUES IN (‘refunded’)
-> );
Query OK, 0 rows affected (0.01 sec)
# 插入测试数据
MariaDB [fgedudb]> DELIMITER //
MariaDB [fgedudb]> CREATE PROCEDURE fgedu_insert_orders(IN p_count INT)
BEGIN
DECLARE i INT DEFAULT 1;
DECLARE v_status VARCHAR(20);
WHILE i <= p_count DO
SET v_status = ELT(FLOOR(RAND() * 5) + 1, ‘pending’, ‘processing’, ‘completed’, ‘cancelled’, ‘refunded’);
INSERT INTO fgedu_orders_list_part (user_id, total_amount, status, created_at)
VALUES (
(i % 100) + 1,
RAND() * 10000,
v_status,
DATE_SUB(NOW(), INTERVAL FLOOR(RAND() * 30) DAY)
);
SET i = i + 1;
END WHILE;
END //
DELIMITER;
Query OK, 0 rows affected (0.00 sec)
# 插入5000条测试数据
MariaDB [fgedudb]> CALL fgedu_insert_orders(5000);
Query OK, 1 row affected (0.18 sec)
# 查看分区数据分布
MariaDB [fgedudb]> SELECT
-> partition_name,
-> table_rows
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_orders_list_part’
-> ORDER BY partition_name;
+—————-+————+
| partition_name | table_rows |
+—————-+————+
| p_cancelled | 987 |
| p_completed | 1012 |
| p_pending | 1003 |
| p_processing | 998 |
| p_refunded | 1000 |
+—————-+————+
# 测试分区查询
MariaDB [fgedudb]> EXPLAIN SELECT * FROM fgedu_orders_list_part
-> WHERE status = ‘completed’\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: fgedu_orders_list_part
partitions: p_completed
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 1012
filtered: 100.00
Extra: Using where
# 测试聚合查询
MariaDB [fgedudb]> SELECT
-> status,
-> COUNT(*) AS order_count,
-> SUM(total_amount) AS total_amount
-> FROM fgedu_orders_list_part
-> GROUP BY status;
+————+————-+————–+
| status | order_count | total_amount |
+————+————-+————–+
| cancelled | 987 | 493500.12 |
| completed | 1012 | 506000.45 |
| pending | 1003 | 501500.78 |
| processing | 998 | 499000.23 |
| refunded | 1000 | 500000.00 |
+————+————-+————–+
MariaDB [fgedudb]> CREATE TABLE fgedu_orders_list_part (
-> id INT PRIMARY KEY AUTO_INCREMENT,
-> user_id INT,
-> total_amount DECIMAL(10,2),
-> status VARCHAR(20),
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY LIST (status) (
-> PARTITION p_pending VALUES IN (‘pending’),
-> PARTITION p_processing VALUES IN (‘processing’),
-> PARTITION p_completed VALUES IN (‘completed’),
-> PARTITION p_cancelled VALUES IN (‘cancelled’),
-> PARTITION p_refunded VALUES IN (‘refunded’)
-> );
Query OK, 0 rows affected (0.01 sec)
# 插入测试数据
MariaDB [fgedudb]> DELIMITER //
MariaDB [fgedudb]> CREATE PROCEDURE fgedu_insert_orders(IN p_count INT)
BEGIN
DECLARE i INT DEFAULT 1;
DECLARE v_status VARCHAR(20);
WHILE i <= p_count DO
SET v_status = ELT(FLOOR(RAND() * 5) + 1, ‘pending’, ‘processing’, ‘completed’, ‘cancelled’, ‘refunded’);
INSERT INTO fgedu_orders_list_part (user_id, total_amount, status, created_at)
VALUES (
(i % 100) + 1,
RAND() * 10000,
v_status,
DATE_SUB(NOW(), INTERVAL FLOOR(RAND() * 30) DAY)
);
SET i = i + 1;
END WHILE;
END //
DELIMITER;
Query OK, 0 rows affected (0.00 sec)
# 插入5000条测试数据
MariaDB [fgedudb]> CALL fgedu_insert_orders(5000);
Query OK, 1 row affected (0.18 sec)
# 查看分区数据分布
MariaDB [fgedudb]> SELECT
-> partition_name,
-> table_rows
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_orders_list_part’
-> ORDER BY partition_name;
+—————-+————+
| partition_name | table_rows |
+—————-+————+
| p_cancelled | 987 |
| p_completed | 1012 |
| p_pending | 1003 |
| p_processing | 998 |
| p_refunded | 1000 |
+—————-+————+
# 测试分区查询
MariaDB [fgedudb]> EXPLAIN SELECT * FROM fgedu_orders_list_part
-> WHERE status = ‘completed’\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: fgedu_orders_list_part
partitions: p_completed
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 1012
filtered: 100.00
Extra: Using where
# 测试聚合查询
MariaDB [fgedudb]> SELECT
-> status,
-> COUNT(*) AS order_count,
-> SUM(total_amount) AS total_amount
-> FROM fgedu_orders_list_part
-> GROUP BY status;
+————+————-+————–+
| status | order_count | total_amount |
+————+————-+————–+
| cancelled | 987 | 493500.12 |
| completed | 1012 | 506000.45 |
| pending | 1003 | 501500.78 |
| processing | 998 | 499000.23 |
| refunded | 1000 | 500000.00 |
+————+————-+————–+
4.3 哈希分区实战
# 场景:用户数据按ID哈希分区
MariaDB [fgedudb]> CREATE TABLE fgedu_users_hash_part (
-> id INT PRIMARY KEY,
-> name VARCHAR(50),
-> email VARCHAR(100),
-> age INT,
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY HASH (id)
-> PARTITIONS 8;
Query OK, 0 rows affected (0.01 sec)
# 插入测试数据
MariaDB [fgedudb]> DELIMITER //
MariaDB [fgedudb]> CREATE PROCEDURE fgedu_insert_users(IN p_count INT)
BEGIN
DECLARE i INT DEFAULT 1;
WHILE i <= p_count DO
INSERT INTO fgedu_users_hash_part (id, name, email, age, created_at)
VALUES (
i,
CONCAT(‘User ‘, i),
CONCAT(‘user’, i, ‘@fgedu.net.cn’),
FLOOR(RAND() * 50) + 20,
DATE_SUB(NOW(), INTERVAL FLOOR(RAND() * 365) DAY)
);
SET i = i + 1;
END WHILE;
END //
DELIMITER;
Query OK, 0 rows affected (0.00 sec)
# 插入8000条测试数据
MariaDB [fgedudb]> CALL fgedu_insert_users(8000);
Query OK, 1 row affected (0.25 sec)
# 查看分区数据分布
MariaDB [fgedudb]> SELECT
-> partition_name,
-> table_rows
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_users_hash_part’
-> ORDER BY partition_name;
+—————-+————+
| partition_name | table_rows |
+—————-+————+
| p0 | 1000 |
| p1 | 1000 |
| p2 | 1000 |
| p3 | 1000 |
| p4 | 1000 |
| p5 | 1000 |
| p6 | 1000 |
| p7 | 1000 |
+—————-+————+
# 测试单点查询
MariaDB [fgedudb]> EXPLAIN SELECT * FROM fgedu_users_hash_part
-> WHERE id = 1234\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: fgedu_users_hash_part
partitions: p6
type: const
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: const
rows: 1
filtered: 100.00
Extra: NULL
# 测试范围查询
MariaDB [fgedudb]> EXPLAIN SELECT * FROM fgedu_users_hash_part
-> WHERE id BETWEEN 1000 AND 2000\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: fgedu_users_hash_part
partitions: p0,p1,p2,p3,p4,p5,p6,p7
type: range
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: NULL
rows: 1000
filtered: 100.00
Extra: Using where
# 测试聚合查询
MariaDB [fgedudb]> SELECT
-> FLOOR(age / 10) * 10 AS age_group,
-> COUNT(*) AS user_count
-> FROM fgedu_users_hash_part
-> GROUP BY age_group
-> ORDER BY age_group;
+———–+————+
| age_group | user_count |
+———–+————+
| 20 | 1600 |
| 30 | 1600 |
| 40 | 1600 |
| 50 | 1600 |
| 60 | 1600 |
+———–+————+
MariaDB [fgedudb]> CREATE TABLE fgedu_users_hash_part (
-> id INT PRIMARY KEY,
-> name VARCHAR(50),
-> email VARCHAR(100),
-> age INT,
-> created_at DATETIME
-> ) ENGINE=InnoDB
-> PARTITION BY HASH (id)
-> PARTITIONS 8;
Query OK, 0 rows affected (0.01 sec)
# 插入测试数据
MariaDB [fgedudb]> DELIMITER //
MariaDB [fgedudb]> CREATE PROCEDURE fgedu_insert_users(IN p_count INT)
BEGIN
DECLARE i INT DEFAULT 1;
WHILE i <= p_count DO
INSERT INTO fgedu_users_hash_part (id, name, email, age, created_at)
VALUES (
i,
CONCAT(‘User ‘, i),
CONCAT(‘user’, i, ‘@fgedu.net.cn’),
FLOOR(RAND() * 50) + 20,
DATE_SUB(NOW(), INTERVAL FLOOR(RAND() * 365) DAY)
);
SET i = i + 1;
END WHILE;
END //
DELIMITER;
Query OK, 0 rows affected (0.00 sec)
# 插入8000条测试数据
MariaDB [fgedudb]> CALL fgedu_insert_users(8000);
Query OK, 1 row affected (0.25 sec)
# 查看分区数据分布
MariaDB [fgedudb]> SELECT
-> partition_name,
-> table_rows
-> FROM information_schema.partitions
-> WHERE table_schema = ‘fgedudb’ AND table_name = ‘fgedu_users_hash_part’
-> ORDER BY partition_name;
+—————-+————+
| partition_name | table_rows |
+—————-+————+
| p0 | 1000 |
| p1 | 1000 |
| p2 | 1000 |
| p3 | 1000 |
| p4 | 1000 |
| p5 | 1000 |
| p6 | 1000 |
| p7 | 1000 |
+—————-+————+
# 测试单点查询
MariaDB [fgedudb]> EXPLAIN SELECT * FROM fgedu_users_hash_part
-> WHERE id = 1234\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: fgedu_users_hash_part
partitions: p6
type: const
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: const
rows: 1
filtered: 100.00
Extra: NULL
# 测试范围查询
MariaDB [fgedudb]> EXPLAIN SELECT * FROM fgedu_users_hash_part
-> WHERE id BETWEEN 1000 AND 2000\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: fgedu_users_hash_part
partitions: p0,p1,p2,p3,p4,p5,p6,p7
type: range
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: NULL
rows: 1000
filtered: 100.00
Extra: Using where
# 测试聚合查询
MariaDB [fgedudb]> SELECT
-> FLOOR(age / 10) * 10 AS age_group,
-> COUNT(*) AS user_count
-> FROM fgedu_users_hash_part
-> GROUP BY age_group
-> ORDER BY age_group;
+———–+————+
| age_group | user_count |
+———–+————+
| 20 | 1600 |
| 30 | 1600 |
| 40 | 1600 |
| 50 | 1600 |
| 60 | 1600 |
+———–+————+
风哥提示:安全开发是防止SQL注入的第一道防线
Part05-风哥经验总结与分享
5.1 分区表使用经验
在实际生产环境中,分区表的使用需要注意以下几点:
- 选择合适的分区类型:根据数据特点和查询模式选择合适的分区类型
- 合理设计分区键:选择查询频率高、分布均匀的列作为分区键
- 适当设置分区数量:避免过多或过少的分区
- 定期维护分区:清理旧分区,添加新分区,优化现有分区
- 监控分区表性能:定期检查分区表的使用情况和性能
5.2 常见问题与解决方案
- 性能问题:
- 问题:查询性能下降
- 解决方案:确保查询条件包含分区键,使用分区修剪
- 分区管理问题:
- 问题:分区数量过多
- 解决方案:合并不必要的分区,合理规划分区策略
- 数据一致性问题:
- 问题:分区表数据不一致
- 解决方案:使用事务,定期检查数据一致性
5.3 生产故障案例分析
某电商系统在使用分区表存储订单数据时出现性能问题,经过分析发现:
- 问题原因:
- 分区键选择不当,导致分区数据分布不均匀
- 分区数量过多,增加了管理复杂度
- 查询条件没有包含分区键,导致全表扫描
- 解决方案:
- 重新选择分区键,使用订单创建时间作为分区键
- 合理调整分区数量,按月份分区
- 优化查询语句,确保包含分区键
- 定期维护分区,清理旧数据
- 效果:查询性能提升5倍,存储空间减少30%
from MariaDB视频:www.itpux.com
本文由风哥教程整理发布,仅用于学习测试使用,转载注明出处:http://www.fgedu.net.cn/10327.html
