Detailed explanation of non-primary key column overflow monitoring in MySQL tables

Detailed explanation of non-primary key column overflow monitoring in MySQL tables

Today, I fell into the trap again. I have encountered MySQL primary key overflow before, and I monitored it through prometheus. For details, see this MySQL primary key overflow review

The pitfalls encountered this time were more hidden. A column of type int signed in a log table is full. The quick solution is of course to switch to a new table as an emergency measure, and then move some historical data from the old table to the hot table.

It’s time to mend the fold after the failure. After resolving the issue, quickly write a script to review all other production tables.

The following is a detection script I am using temporarily. It is not perfect yet, so I will use it as it is.

Divide into 2 files (1 sql file, 1 shell script)

The content of check.sql is as follows:

SELECT 
cast( pow(2, case data_type
  when 'tinyint' then 7
  when 'smallint' then 15
  when 'mediumint' then 23
  when 'int' then 31
  when 'bigint' then 63
  end+(column_type like '% unsigned'))-1 as decimal(30,0)) as max_int,
' - ',
concat ('(', concat('select ','max(',COLUMN_NAME,')',' from ',TABLE_SCHEMA,'.',TABLE_NAME),')') 
from 
information_schema.COLUMNS 
where 
TABLE_SCHEMA NOT IN ('information_schema','sys','test','mysql','performance_schema') 
AND 
 DATA_TYPE IN ('int' ) ;

Execute directly in the database, the effect is similar to this:

The content of check.sh is as follows:

#!/bin/bash
# Monitor int type. When the available space is less than 5 million, remind to do DDL operation. # Set max_execution_time of session level to 2 seconds to prevent large columns without index from slowing down the database. However, this may miss some columns, so please pay attention. # Note: I have not checked bigint type here. If necessary, please modify DATA_TYPE in check.sql where condition to add bigint check source /etc/profile
set -u

mkdir $(date +%F) -pv

# step1 detection for host in {'192.168.1.100','192.168.1.110','192.168.1.120','192.168.1.130'}; do

mysql -udts -pdts -h${host} -BN < check.sql 2>/dev/null > sql.log
wait

echo "Description: | Maximum value allowed for the current column | SQL for inspection " >> $(date +%F)/$host.log

while read line; do
  ret=$(mysql -udts -pdts -h${host} -BNe "set session max_execution_time=2000;select $line" 2>/dev/null)
  echo ${ret}
  if [[ "${ret}" == "NULL" ]]; then
  continue
  fi
  if [ ${ret} -lt 5000000 ] ; then 
   echo "$line remaining space ${ret}, the table available water level is less than 5 million, it is recommended to modify the DDL to bigint type" >> $(date +%F)/$host.log
  
  fi
done < ./sql.log

done

# step2 Package the inspection contents and send them by email (you may need to modify this according to your production situation)
tar czf $(date +%F).tar.gz $(date +%F)
sendemail -s 192.168.1.200 -f [email protected] -t [email protected] -a $(date +%F).tar.gz -u "$(date +%F) int water level inspection log" -o message-content-type=html -o message-charset=utf8 -m "For details, please see the attachment"

# step3 clean up the date-named directories and tar.gz files generated daily. I won’t post the commands here.

Then add a cronjob at 10 am every day.

Finally, the content of the emails received every day is roughly similar to the following:

This is the end of this article on how to monitor the overflow of non-primary key columns in MySQL tables. For more information about MySQL non-primary key column overflows, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!

You may also be interested in:
  • In-depth discussion on auto-increment primary keys in MySQL
  • The difference and advantages and disadvantages of Mysql primary key UUID and auto-increment primary key
  • MySQL primary key naming strategy related
  • What to do if the auto-increment primary key in MySQL is used up
  • Solution to running out of MySQL's auto-increment ID (primary key)
  • In-depth analysis of why MySQL does not recommend using uuid or snowflake id as primary key
  • Use prometheus to count the remaining available percentage of MySQL auto-increment primary keys
  • Mysql SQL statement operation to add or modify primary key

<<:  Implementation of k8s deployment of docker container

>>:  Method of dynamically loading geojson based on Vue+Openlayer

Recommend

MySQL index failure principle

Table of contents 1. Reasons for index failure 2....

js to achieve simple magnifying glass effects

This article example shares the specific code of ...

10 Deadly Semantic Mistakes in Web Typography

<br />This is from the content of Web front-...

Example test MySQL enum type

When developing a project, you will often encount...

JavaScript Regular Expressions Explained

Table of contents 1. Regular expression creation ...

SQL serial number acquisition code example

This article mainly introduces the sql serial num...

MySQL master-slave replication configuration process

Main library configuration 1. Configure mysql vim...

Correct steps to install Nginx in Linux

Preface If you are like me, as a hard-working Jav...

In-depth study of how to use positioning in CSS (summary)

Introduction to Positioning in CSS position attri...

Detailed explanation of Mysql transaction isolation level read commit

View MySQL transaction isolation level mysql> ...

How to solve the problem of character set when logging in to Linux

Character set error always exists locale: Cannot ...