Skip to content

Commit dee592d

Browse files
author
Sujatha Sivakumar
committed
Bug#26414532: MYSQLRPLSYNC ERRORS OUT BECAUSE SLAVE IS USING
--SUPER-READ-ONLY OPTION Problem: ======== After upgrading MySQL server from 5.7.17 to 5.7.18 'mysqlrplsync' utility tool fails with following error. ERROR: Query failed. 1290 (HY000): The MySQL server is running with the --super-read-only option so it cannot execute this statement When this occurs, the Slave_SQL_Running on the slave server stops running and needs to be manually started again. The slave server does have the super-read-only option set to true. When running MySQL 5.7.17 this wasn't an issue. Analysis: ======== When replication information repository is set to 'TABLE' few replication specific administration commands listed below, will try to update the repository tables. 1) START SLAVE 2) STOP SLAVE 3) CHANGE MASTER 4) RESET SLAVE 5) RESET MASTER 6) SET GLOBAL GTID_PURGED 7) FLUSH BINARY LOGS 8) 'gtid_executed' TABLE compression thread. First three commands were fixed as part of Bug#22097534: SUPER_READ_ONLY ABORTS STOP SLAVE IF RELAY_LOG_INFO_REPOSITORY=TABLE, DBG CRASH The reset of the commands were fixed as part of Bug#22857926: ASSERTION `! IS_SET()' AT SQL_ERROR.CC:38 IN READ_ONLY MODE FOR MANY RPL CMDS. During the Bug#22857926 fix, the previous Bug#22097534 fix specific changes were reverted a new approach was taken. The new approach used a flag named 'ignore_global_read_lock' whose intention is to ignore the read only option and allow commits to replication specific system tables. If this flag is set then the additional check for 'read only' option is not done. But the above fix works fine only when 'AUTOCOMMIT=ON'. In the case of AUTOCOMMIT=OFF the 'ignore_global_read_lock' flag is not set. Hence commands fail to ignore read_only restriction. STOP SLAVE command behavior explained in detail: STOP SLAVE command is an IMPLICIT COMMIT command, i.e parser will append an explicit BEGIN and COMMIT for this command. BEGIN STOP SLAVE; COMMIT; STOP SLAVE command will try to stop both the applier thread and receiver thread and update their positions in 'slave_relay_log_info' and 'slave_master_info' tables. Case: AUTOCOMMIT=ON. 1) BEGIN 2) Update 'slave_relay_log_info' table and commit as a real transaction. System_table_access::close_table will invoke ha_commit_trans with 'ignore_global_read_lock=true'. Since autocommit is 'ON' each statement will do a complete commit and clean up the transaction context. 3) Update 'slave_master_info' also does the same. 4) COMMIT. COMMIT will invoke 'trans_commit_implicit'. It will check if there any active transactions which needs to be committed implicitly. Since step 2-3 are real commit transactions this final commit has nothing to do. It will exit without proceeding further. Case: AUTOCOMMIT=OFF. 1) BEGIN 2) Update 'slave_relay_log_info' table. This transaction is considered as a multi statement transaction mode, it will not commit on its own. It will wait for final COMMIT execution. 3) Update 'slave_master_info' also does the same. 4) COMMIT. COMMIT will invoke 'trans_commit_implicit'. It will check if there any active transactions which needs to be committed . Since there are active transaction sessions from step 2-3 'ha_commit_trans' will be called as shown below. 'trans_commit_implicit' will invoke ha_commit_trans (thd,all=true,ignore_global_read_lock=false) This will not pass the read_only check and the command will fail. This issue happens for commands that update info tables other than 'gtid_executed' table. In case of 'gtid_executed' table irrespective of autocommit variable all the transactions are considered as 'real' transactions and they will do complete commit. Fix: === Introduced a new variable, to flag the sql command under execution to ignore the read_only/super_read_only. If the variable is set then the command is allowed to execute even though read_only/super_read_only is enabled otherwise command is blocked.
1 parent 1c582ac commit dee592d

16 files changed

+437
-147
lines changed

‎mysql-test/extra/rpl_tests/rpl_ignore_super_read_only.test

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# ==== Purpose ====
2+
#
3+
# Verify that we permit replication operations with super_read_only=ON.
4+
#
5+
# ==== Implementation ====
6+
#
7+
# 1) SET GLOBAL super_read_only to 1 on slave
8+
# 2) Execute 'STOP SLAVE' command
9+
# 3) Execute 'CHANGE REPLICATION FILTER' command
10+
# 4) Execute 'CHANGE MASTER TO' command
11+
# 5) Execute 'RESET SLAVE FOR CHANNEL <channel_name>' command
12+
# 7) Execute 'RESET SLAVE ALL FOR CHANNEL <channel_name>' command
13+
# 8) Execute 'RESET SLAVE' command
14+
# 9) Execute 'START SLAVE' command
15+
# 10) Execute 'RESET MASTER' command (with GTID_MODE=ON will assert)
16+
# 11) Execute 'FLUSH BINARY LOGS' command (with GTID_MODE=ON will assert)
17+
# 12) Execute 'SET GLOBAL gtid_purged='aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1''
18+
# command
19+
# 14) Check that replication works fine
20+
# 15) Restore GLOBAL super_read_only to 0
21+
#
22+
# ==== References ====
23+
#
24+
# Bug#22097534: SUPER_READ_ONLY ABORTS STOP SLAVE IF
25+
# RELAY_LOG_INFO_REPOSITORY=TABLE, DBG CRASH
26+
# Bug#22857926: ASSERTION `! IS_SET()' AT SQL_ERROR.CC:38 IN READ_ONLY MODE FOR
27+
# MANY RPL CMDS.
28+
# Bug#26414532: MYSQLRPLSYNC ERRORS OUT BECAUSE SLAVE IS USING --SUPER-READ-ONLY
29+
# OPTION
30+
#
31+
32+
if (!$autocommit_opt)
33+
{
34+
--die !!!ERROR IN TEST: you must set $autocommit_opt
35+
}
36+
37+
--source include/rpl_connection_master.inc
38+
CREATE TABLE t1(a INT);
39+
INSERT INTO t1 VALUES(1);
40+
DROP TABLE t1;
41+
--source include/sync_slave_sql_with_master.inc
42+
43+
SET @saved_value_super= @@GLOBAL.super_read_only;
44+
SET @saved_value= @@GLOBAL.read_only;
45+
SET @saved_autocommit= @@GLOBAL.autocommit;
46+
SET GLOBAL super_read_only= 1;
47+
--eval SET AUTOCOMMIT= $autocommit_opt
48+
SHOW VARIABLES like '%autocommit%';
49+
50+
--echo ####################################################################
51+
--echo # Test Case1: STOP SLAVE command
52+
--echo ####################################################################
53+
--source include/stop_slave.inc
54+
55+
--echo ####################################################################
56+
--echo # Test Case2: CHANGE REPLICATION FILTER command
57+
--echo ####################################################################
58+
CHANGE REPLICATION FILTER REPLICATE_DO_DB=(test);
59+
CHANGE REPLICATION FILTER REPLICATE_DO_DB=();
60+
61+
--echo ####################################################################
62+
--echo # Test Case3: CHANGE MASTER command
63+
--echo ####################################################################
64+
CHANGE MASTER TO MASTER_CONNECT_RETRY=20;
65+
CHANGE MASTER TO MASTER_HOST='dummy' FOR CHANNEL 'aaa';
66+
67+
--echo ####################################################################
68+
--echo # Test Case4: RESET SLAVE FOR CHANNEL/RESET SLAVE ALL/RESET SLAVE
69+
--echo # commands
70+
--echo ####################################################################
71+
RESET SLAVE FOR CHANNEL 'aaa';
72+
RESET SLAVE ALL FOR CHANNEL 'aaa';
73+
RESET SLAVE;
74+
75+
--echo ####################################################################
76+
--echo # Test Case5: START SLAVE command
77+
--echo ####################################################################
78+
--source include/start_slave.inc
79+
--source include/rpl_connection_master.inc
80+
--source include/sync_slave_sql_with_master.inc
81+
82+
# Check that mysql.gtid_executed table is empty.
83+
--let $assert_text= mysql.gtid_executed table must have zero records
84+
--let $assert_cond= [SELECT count(*) FROM mysql.gtid_executed] = 0
85+
--source include/assert.inc
86+
87+
--echo ####################################################################
88+
--echo # Test Case6: FLUSH BINARY LOGS command
89+
--echo ####################################################################
90+
# FLUSH BINARY LOGS asserts when it is trying to update gtid_executed table
91+
# during binary log rotation.
92+
FLUSH BINARY LOGS;
93+
94+
# Check that an entry is updated in gtid_executed table without causing any
95+
# assert.
96+
--disable_query_log
97+
if ( `SELECT @@GLOBAL.GTID_MODE = "ON"` )
98+
{
99+
--let $table=mysql.gtid_executed
100+
--let $count=1
101+
--source include/wait_until_rows_count.inc
102+
}
103+
--enable_query_log
104+
105+
--echo ####################################################################
106+
--echo # Test Case7: RESET MASTER command
107+
--echo ####################################################################
108+
RESET MASTER;
109+
110+
--echo ####################################################################
111+
--echo # Test Case8: SET GLOBAL GTID_PURGED command
112+
--echo ####################################################################
113+
--let $master_uuid=`SELECT @@SERVER_UUID`
114+
--replace_result $master_uuid MASTER_UUID
115+
--eval SET GLOBAL gtid_purged= '$master_uuid:1'
116+
117+
118+
--echo "Clean up"
119+
# Reset slave to clean state
120+
SET AUTOCOMMIT= @saved_autocommit;
121+
SET GLOBAL super_read_only= @saved_value_super;
122+
SET GLOBAL read_only= @saved_value;
123+
124+
# Start fresh slave
125+
--source include/stop_slave.inc
126+
--replace_result $MASTER_MYPORT MASTER_PORT
127+
--replace_column 2 ####
128+
--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_PORT=$MASTER_MYPORT, MASTER_USER='root';
129+
--source include/start_slave.inc
130+
131+
--source include/rpl_connection_master.inc
132+
--source include/sync_slave_sql_with_master.inc
133+
134+
--source include/rpl_reset.inc

‎mysql-test/suite/rpl/r/rpl_gtid_compress_thd_ignore_super_read_only.result

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
include/master-slave.inc
2+
Warnings:
3+
Note #### Sending passwords in plain text without SSL/TLS is extremely insecure.
4+
Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
5+
[connection master]
6+
[connection slave]
7+
call mtr.add_suppression("You need to use --log-bin*");
8+
SET @saved_value_super= @@GLOBAL.super_read_only;
9+
SET @saved_value= @@GLOBAL.read_only;
10+
SET GLOBAL super_read_only= 1;
11+
[connection master]
12+
CREATE TABLE t1(a INT) ENGINE=INNODB;
13+
INSERT INTO t1 VALUES(1);
14+
INSERT INTO t1 VALUES(2);
15+
INSERT INTO t1 VALUES(3);
16+
INSERT INTO t1 VALUES(4);
17+
DROP TABLE t1;
18+
include/sync_slave_sql_with_master.inc
19+
#
20+
# Verify that gtid_executed table has MASTER_UUID:1:6
21+
#
22+
SELECT * FROM mysql.gtid_executed;
23+
source_uuid interval_start interval_end
24+
MASTER_UUID 1 6
25+
"Clean up"
26+
SET GLOBAL super_read_only= @saved_value_super;
27+
SET GLOBAL read_only= @saved_value;
28+
include/rpl_end.inc

‎mysql-test/suite/rpl/r/rpl_operation_ignore_super_read_only.result

Lines changed: 104 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,45 +3,136 @@ Warnings:
33
Note #### Sending passwords in plain text without SSL/TLS is extremely insecure.
44
Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
55
[connection master]
6+
[connection master]
67
CREATE TABLE t1(a INT);
78
INSERT INTO t1 VALUES(1);
9+
DROP TABLE t1;
810
include/sync_slave_sql_with_master.inc
911
SET @saved_value_super= @@GLOBAL.super_read_only;
1012
SET @saved_value= @@GLOBAL.read_only;
13+
SET @saved_autocommit= @@GLOBAL.autocommit;
1114
SET GLOBAL super_read_only= 1;
15+
SET AUTOCOMMIT= ON;
16+
SHOW VARIABLES like '%autocommit%';
17+
Variable_name Value
18+
autocommit ON
19+
####################################################################
20+
# Test Case1: STOP SLAVE command
21+
####################################################################
1222
include/stop_slave.inc
23+
####################################################################
24+
# Test Case2: CHANGE REPLICATION FILTER command
25+
####################################################################
1326
CHANGE REPLICATION FILTER REPLICATE_DO_DB=(test);
1427
CHANGE REPLICATION FILTER REPLICATE_DO_DB=();
28+
####################################################################
29+
# Test Case3: CHANGE MASTER command
30+
####################################################################
1531
CHANGE MASTER TO MASTER_CONNECT_RETRY=20;
32+
CHANGE MASTER TO MASTER_HOST='dummy' FOR CHANNEL 'aaa';
33+
####################################################################
34+
# Test Case4: RESET SLAVE FOR CHANNEL/RESET SLAVE ALL/RESET SLAVE
35+
# commands
36+
####################################################################
37+
RESET SLAVE FOR CHANNEL 'aaa';
38+
RESET SLAVE ALL FOR CHANNEL 'aaa';
39+
RESET SLAVE;
40+
####################################################################
41+
# Test Case5: START SLAVE command
42+
####################################################################
1643
include/start_slave.inc
1744
[connection master]
18-
INSERT INTO t1 VALUES(2);
1945
include/sync_slave_sql_with_master.inc
20-
include/assert.inc [Table t1 must have two records]
21-
include/stop_slave.inc
2246
include/assert.inc [mysql.gtid_executed table must have zero records]
47+
####################################################################
48+
# Test Case6: FLUSH BINARY LOGS command
49+
####################################################################
2350
FLUSH BINARY LOGS;
24-
CHANGE MASTER TO MASTER_HOST='dummy' FOR CHANNEL 'aaa';
25-
RESET SLAVE FOR CHANNEL 'aaa';
26-
RESET SLAVE ALL FOR CHANNEL 'aaa';
27-
RESET SLAVE;
28-
RESET MASTER;
29-
SET GLOBAL gtid_purged='aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1';
30-
include/assert.inc [Found a record with source_uuid 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' in gtid_executed table.]
51+
####################################################################
52+
# Test Case7: RESET MASTER command
53+
####################################################################
3154
RESET MASTER;
55+
####################################################################
56+
# Test Case8: SET GLOBAL GTID_PURGED command
57+
####################################################################
58+
SET GLOBAL gtid_purged= 'MASTER_UUID:1';
59+
"Clean up"
60+
SET AUTOCOMMIT= @saved_autocommit;
3261
SET GLOBAL super_read_only= @saved_value_super;
3362
SET GLOBAL read_only= @saved_value;
34-
SET GLOBAL gtid_purged='aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1';
63+
include/stop_slave.inc
64+
CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_PORT=MASTER_PORT, MASTER_USER='root';;
65+
Warnings:
66+
Note #### Sending passwords in plain text without SSL/TLS is extremely insecure.
67+
Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
68+
include/start_slave.inc
69+
[connection master]
70+
include/sync_slave_sql_with_master.inc
71+
include/rpl_reset.inc
72+
[connection master]
73+
CREATE TABLE t1(a INT);
74+
INSERT INTO t1 VALUES(1);
75+
DROP TABLE t1;
76+
include/sync_slave_sql_with_master.inc
77+
SET @saved_value_super= @@GLOBAL.super_read_only;
78+
SET @saved_value= @@GLOBAL.read_only;
79+
SET @saved_autocommit= @@GLOBAL.autocommit;
3580
SET GLOBAL super_read_only= 1;
81+
SET AUTOCOMMIT= OFF;
82+
SHOW VARIABLES like '%autocommit%';
83+
Variable_name Value
84+
autocommit OFF
85+
####################################################################
86+
# Test Case1: STOP SLAVE command
87+
####################################################################
88+
include/stop_slave.inc
89+
####################################################################
90+
# Test Case2: CHANGE REPLICATION FILTER command
91+
####################################################################
92+
CHANGE REPLICATION FILTER REPLICATE_DO_DB=(test);
93+
CHANGE REPLICATION FILTER REPLICATE_DO_DB=();
94+
####################################################################
95+
# Test Case3: CHANGE MASTER command
96+
####################################################################
97+
CHANGE MASTER TO MASTER_CONNECT_RETRY=20;
98+
CHANGE MASTER TO MASTER_HOST='dummy' FOR CHANNEL 'aaa';
99+
####################################################################
100+
# Test Case4: RESET SLAVE FOR CHANNEL/RESET SLAVE ALL/RESET SLAVE
101+
# commands
102+
####################################################################
103+
RESET SLAVE FOR CHANNEL 'aaa';
104+
RESET SLAVE ALL FOR CHANNEL 'aaa';
105+
RESET SLAVE;
106+
####################################################################
107+
# Test Case5: START SLAVE command
108+
####################################################################
109+
include/start_slave.inc
110+
[connection master]
111+
include/sync_slave_sql_with_master.inc
112+
include/assert.inc [mysql.gtid_executed table must have zero records]
113+
####################################################################
114+
# Test Case6: FLUSH BINARY LOGS command
115+
####################################################################
116+
FLUSH BINARY LOGS;
117+
####################################################################
118+
# Test Case7: RESET MASTER command
119+
####################################################################
36120
RESET MASTER;
121+
####################################################################
122+
# Test Case8: SET GLOBAL GTID_PURGED command
123+
####################################################################
124+
SET GLOBAL gtid_purged= 'MASTER_UUID:1';
125+
"Clean up"
126+
SET AUTOCOMMIT= @saved_autocommit;
37127
SET GLOBAL super_read_only= @saved_value_super;
38128
SET GLOBAL read_only= @saved_value;
39-
DROP TABLE t1;
129+
include/stop_slave.inc
40130
CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_PORT=MASTER_PORT, MASTER_USER='root';;
41131
Warnings:
42132
Note #### Sending passwords in plain text without SSL/TLS is extremely insecure.
43133
Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
44134
include/start_slave.inc
45135
[connection master]
46-
DROP TABLE t1;
136+
include/sync_slave_sql_with_master.inc
137+
include/rpl_reset.inc
47138
include/rpl_end.inc

‎mysql-test/suite/rpl/t/rpl_gtid_compress_thd_ignore_super_read_only-slave.opt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
--skip-log-bin --gtid_executed_compression_period=5

‎mysql-test/suite/rpl/t/rpl_gtid_compress_thd_ignore_super_read_only.test

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# ==== Purpose ====
2+
#
3+
# Verify that we permit replication operations with super_read_only=ON.
4+
#
5+
# ==== Implementation ====
6+
#
7+
# 1) Have gtid_mode on and binlog less slave.
8+
# 2) set super_read_only=on on slave server.
9+
# 3) Set gtid_executed_compression_period=5 so that compression thread wakes
10+
# up after 5 gtid transactions are executed.
11+
# 4) Execute 6 transaction on master so that it triggers compression thread
12+
# on slave to update mysql.gtid_executed table.
13+
# 5) Verify that Slave server is up and running.
14+
# 6) Check that replication works fine
15+
# 7) Restore GLOBAL super_read_only to 0
16+
#
17+
# ==== References ====
18+
#
19+
# Bug#22097534: SUPER_READ_ONLY ABORTS STOP SLAVE IF
20+
# RELAY_LOG_INFO_REPOSITORY=TABLE, DBG CRASH
21+
# Bug#22857926: ASSERTION `! IS_SET()' AT SQL_ERROR.CC:38 IN READ_ONLY MODE
22+
# FOR MANY RPL CMDS.
23+
# Bug#26414532: MYSQLRPLSYNC ERRORS OUT BECAUSE SLAVE IS USING
24+
# --SUPER-READ-ONLY OPTION
25+
#
26+
27+
# Test in this file only makes sense in standard replication,
28+
# so it is skipped in group replication.
29+
--source include/not_group_replication_plugin.inc
30+
# Gtids are mandatory for the test.
31+
--source include/have_gtid.inc
32+
# Test in this file is binlog format agnostic, thus no need
33+
# to rerun it for every format.
34+
--source include/have_binlog_format_row.inc
35+
--source include/master-slave.inc
36+
37+
--source include/rpl_connection_slave.inc
38+
call mtr.add_suppression("You need to use --log-bin*");
39+
SET @saved_value_super= @@GLOBAL.super_read_only;
40+
SET @saved_value= @@GLOBAL.read_only;
41+
SET GLOBAL super_read_only= 1;
42+
43+
--source include/rpl_connection_master.inc
44+
--let $master_uuid= `SELECT @@GLOBAL.SERVER_UUID`
45+
CREATE TABLE t1(a INT) ENGINE=INNODB;
46+
INSERT INTO t1 VALUES(1);
47+
INSERT INTO t1 VALUES(2);
48+
INSERT INTO t1 VALUES(3);
49+
INSERT INTO t1 VALUES(4);
50+
DROP TABLE t1;
51+
--source include/sync_slave_sql_with_master.inc
52+
53+
--echo #
54+
--echo # Verify that gtid_executed table has MASTER_UUID:1:6
55+
--echo #
56+
--replace_result $master_uuid MASTER_UUID
57+
SELECT * FROM mysql.gtid_executed;
58+
59+
--echo "Clean up"
60+
# Reset slave to clean state
61+
SET GLOBAL super_read_only= @saved_value_super;
62+
SET GLOBAL read_only= @saved_value;
63+
64+
--source include/rpl_end.inc

0 commit comments

Comments
 (0)