MySQL partition table is classified by month

MySQL partition table is classified by month

It is recommended that the data volume of a single MySQL table should not exceed 20 million rows, otherwise it will have a significant impact on performance. I recently took over a project with a single table containing over 70 million rows of data. A simple query took more than 50 minutes to produce any results, which was really frustrating. In the end, we decided to use a partitioned table.

Create a table

After a general table (innodb) is created, there is only one idb file:

create table normal_table(id int primary key, no int)

View the database file:

normal_table.ibd  

Create a partition table partitioned by month, attention! In addition to the regular primary key, the month field (the field used for partitioning) must also be the primary key:

create table partition_table(id int AUTO_INCREMENT, create_date date, name varchar(10), 
primary key(id, create_date)) ENGINE=INNODB DEFAULT CHARSET=utf8 
partition by range(month(create_date))(
partition quarter1 values ​​less than(4),
partition quarter2 values ​​less than(7),
partition quarter3 values ​​less than(10),
partition quarter4 values ​​less than(13)
);

View the database file:

partition_table#p#quarter1.ibd  
partition_table#p#quarter2.ibd  
partition_table#p#quarter3.ibd  
partition_table#p#quarter4.ibd

insert

insert into partition_table(create_date, name) values("2021-01-25", "tom1");
insert into partition_table(create_date, name) values("2021-02-25", "tom2");
insert into partition_table(create_date, name) values("2021-03-25", "tom3");
insert into partition_table(create_date, name) values("2021-04-25", "tom4");
insert into partition_table(create_date, name) values("2021-05-25", "tom5");
insert into partition_table(create_date, name) values("2021-06-25", "tom6");
insert into partition_table(create_date, name) values("2021-07-25", "tom7");
insert into partition_table(create_date, name) values("2021-08-25", "tom8");
insert into partition_table(create_date, name) values("2021-09-25", "tom9");
insert into partition_table(create_date, name) values("2021-10-25", "tom10");
insert into partition_table(create_date, name) values("2021-11-25", "tom11");
insert into partition_table(create_date, name) values("2021-12-25", "tom12");

Query

select count(*) from partition_table;
> 12

 
Query the data of the second partition (second quarter):
select * from partition_table PARTITION(quarter2);

4 2021-04-25 tom4
5 2021-05-25 tom5
6 2021-06-25 tom6

delete

When a table is deleted, all partition files of the table will be deleted.

Supplement: Mysql automatically partitions by month table

The core two stored procedures:

  • auto_create_partition is used to create table partitions. After being called, table partitions are created for the table until the end of the next month.
  • auto_del_partition is used to delete table partitions to facilitate the recovery of historical data space.
DELIMITER $$
DROP PROCEDURE IF EXISTS auto_create_partition$$
CREATE PROCEDURE `auto_create_partition`(IN `table_name` varchar(64))
BEGIN
   SET @next_month:=CONCAT(date_format(date_add(now(),interval 2 month),'%Y%m'),'01');
   SET @SQL = CONCAT( 'ALTER TABLE `', table_name, '`',
     ' ADD PARTITION (PARTITION p', @next_month, " VALUES LESS THAN (TO_DAYS(",
       @next_month ,")) );" );
   PREPARE STMT FROM @SQL;
   EXECUTE STMT;
   DEALLOCATE PREPARE STMT;
END$$

DROP PROCEDURE IF EXISTS auto_del_partition$$
CREATE PROCEDURE `auto_del_partition`(IN `table_name` varchar(64), IN `reserved_month` int)
BEGIN
 DECLARE v_finished INTEGER DEFAULT 0;
 DECLARE v_part_name varchar(100) DEFAULT "";
 DECLARE part_cursor CURSOR FOR 
  select partition_name from information_schema.partitions where table_schema = schema()
   and table_name=@table_name and partition_description < TO_DAYS(CONCAT(date_format(date_sub(now(),interval reserved_month month),'%Y%m'),'01'));
 DECLARE continue handler FOR 
  NOT FOUND SET v_finished = TRUE;
 OPEN part_cursor;
read_loop: LOOP
 FETCH part_cursor INTO v_part_name;
 if v_finished = 1 then
  leave read_loop;
 end if;
 SET @SQL = CONCAT( 'ALTER TABLE `', table_name, '` DROP PARTITION ', v_part_name, ";" );
 PREPARE STMT FROM @SQL;
 EXECUTE STMT;
 DEALLOCATE PREPARE STMT;
 END LOOP;
 CLOSE part_cursor;
END$$

DELIMITER ;

Here is an example

-- Assume there is a table called records, set the partition condition to partition by month by end_time DROP TABLE IF EXISTS `records`;
CREATE TABLE `records` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `start_time` datetime NOT NULL,
  `end_time` datetime NOT NULL,
  `memo` varchar(128) CHARACTER SET utf8mb4 NOT NULL,
  PRIMARY KEY (`id`,`end_time`)
) 
PARTITION BY RANGE (TO_DAYS(end_time))(
 PARTITION p20200801 VALUES LESS THAN ( TO_DAYS('20200801'))
);

DROP EVENT IF EXISTS `records_auto_partition`;

-- Create an Event that executes once a month and saves up to 6 months of data DELIMITER $$
CREATE EVENT `records_auto_partition`
ON SCHEDULE EVERY 1 MONTH ON COMPLETION PRESERVE
ENABLE
DO
BEGIN
call auto_create_partition('records');
call auto_del_partition('records',6);
END$$
DELIMITER ;

A few notes:

  • For MySQL 5.1 and above, the index field of the table partition must be the primary key.
  • In a stored procedure, DECLARE must be followed by BEGIN, otherwise an incomprehensible error will be reported.
  • The cursor DECLARE must be after the definition statement, otherwise an error will be reported
  • If you installed MySQL yourself, the Event function may not be enabled, and an error message will be displayed when creating an Event. Modify my.cnf, add event_scheduler=1 under [mysqld], and restart.

This is the end of this article about how to implement monthly classification in MySQL partitioned tables. For more information about MySQL partitioned tables by month, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!

You may also be interested in:
  • Why must the partition key in a MySQL partition table be part of the primary key?
  • A performance bug about MySQL partition tables
  • Detailed explanation of MySQL partition table
  • Management and maintenance of Mysql partition tables
  • MySQL optimization partition table
  • Summary of MySQL partition table management commands

<<:  SystemC environment configuration method under Linux system

>>:  HTML form tag usage learning tutorial

Recommend

How to import js configuration file on Vue server

Table of contents background accomplish Supplemen...

How to install MySQL database on Ubuntu

Ubuntu is a free and open source desktop PC opera...

DIV background semi-transparent text non-translucent style

DIV background is semi-transparent, but the words ...

How to install and uninstall open-vswitch in Linux

1. Compile and install ovs from source code: Inst...

Nexus private server construction principle and tutorial analysis

one. Why build a Nexus private server? All develo...

The correct way to migrate MySQL data to Oracle

There is a table student in the mysql database, i...

Detailed explanation of the difference between CSS link and @import

How to add css in html? There are three ways to s...

Node.js solves the problem of Chinese garbled characters in client request data

Node.js solves the problem of Chinese garbled cha...

How to modify the forgotten password when installing MySQL on Mac

1. Install MySQL database on mac 1. Download MySQ...

The difference between method=post/get in Form

Form provides two ways of data transmission - get ...

MySQL 8.0.14 installation and configuration method graphic tutorial

This article records the installation and configu...

Details of the underlying data structure of MySQL indexes

Table of contents 1. Index Type 1. B+ Tree 2. Wha...

MySQL not null constraint case explanation

Table of contents Set a not null constraint when ...

Using CSS3 to achieve progress bar effect and dynamically add percentage

During the project, I started using the js reques...