Bug#25135304: RBR: WRONG FIELD LENGTH IN ERROR MESSAGE

Sujatha Sivakumar · Sujatha Sivakumar · commit 47bd3f7cf3c8 · 2017-02-24T10:53:23.000+05:30
Description:
============
In row based replication, when replicating from a table with
a field with character set set to UTF8mb3 to the same table
with the same field set to character set UTF8mb4 I get a
confusing error message:

For VARCHAR: VARCHAR(1) 'utf8mb3' to VARCHAR(1) 'utf8mb4'

"Column 0 of table 'test.t1' cannot be converted from type
'varchar(3)' to type 'varchar(1)'"

Similar issue with CHAR type as well.

Issue with respect to BLOB types:

For BLOB: LONGBLOB to TINYBLOB - Error message displays
incorrect blob type.

"Column 0 of table 'test.t1' cannot be converted from type
'tinyblob' to type 'tinyblob'"

Analysis:
=========
In Row based replication charset information is not sent as
part of metadata from master to slave.

For VARCHAR field its character length is converted into
equivalent octets/bytes and stored internally. At the time
of displaying the data to user it is converted back to
original character length.

For example:
VARCHAR(2)- utf8mb3 is stored as:2*3 = VARCHAR(6)
At the time of displaying it to user
VARCHAR(6)- charset utf8mb3:6/3= VARCHAR(2).

At present the internally converted octect length is sent
from master to slave with out providing the charset
information. On slave side if the type conversion fails
'show_sql_type' function is used to get the type specific
information from metadata. Since there is no charset
information is available the filed type is displayed as
VARCHAR(6).

This results in confused error message.

For CHAR fields
CHAR(1)- utf8mb3 - CHAR(3)
CHAR(1)- utf8mb4 - CHAR(4)

'show_sql_type' function which retrieves type information
from metadata uses (bytes/local charset length) to get
actual character length. If slave's chaset is 'utf8mb4' then

CHAR(3/4)--&gt;CHAR(0)
CHAR(4/4)--&gt;CHAR(1).

This results in confused error message.

Analysis for BLOB type issue:

BLOB's length is represented in two forms.
1. Actual length
i.e
  (length &lt; 256) type= MYSQL_TYPE_TINY_BLOB;
  (length &lt; 65536) type= MYSQL_TYPE_BLOB; ...

2. packlength - The number of bytes used to represent the
length of the blob
  1- tinyblob
  2- blob ...

In row based replication only the packlength is written in
the binary log. On the slave side this packlength is
interpreted as actual length of the blob.  Hence the length
is always &lt; 256 and the type is displayed as tiny blob.

Fix:
===
For CHAR and VARCHAR fields display their length in bytes
for both source and target fields. For target field display
the charset information if it is relevant.

For blob type changed the code to use the packlength and
display appropriate blob type in error message.
diff --git a/mysql-test/suite/rpl/r/rpl_type_conv_utf8_utf8mb4.result b/mysql-test/suite/rpl/r/rpl_type_conv_utf8_utf8mb4.result
@@ -0,0 +1,71 @@
+include/master-slave.inc
+Warnings:
+Note	####	Sending passwords in plain text without SSL/TLS is extremely insecure.
+Note	####	Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
+[connection master]
+####################################################################
+# Test Case1: Improved error message with charset information
+####################################################################
+[connection master]
+SET SQL_LOG_BIN=0;
+CREATE TABLE t1 (c1 VARCHAR(1) CHARACTER SET 'utf8mb3');
+SET SQL_LOG_BIN=1;
+[connection slave]
+CREATE TABLE t1 (c1 VARCHAR(1) CHARACTER SET 'utf8mb4');
+[connection master]
+INSERT INTO t1 VALUES ('a');
+[connection slave]
+include/wait_for_slave_sql_error.inc [errno=1677]
+Matching lines are:
+DATE_TIME [ERROR] Slave SQL for channel '': Column 0 of table 'test.t1' cannot be converted from type 'varchar(3(bytes))' to type 'varchar(4(bytes) utf8mb4)', Error_code: 1677
+Occurrences of '\'varchar\(3\(bytes\)\)\' to type \'varchar\(4\(bytes\) utf8mb4\)\'' in the input file: 1
+[connection master]
+DROP TABLE t1;
+[connection slave]
+DROP TABLE t1;
+include/rpl_reset.inc
+####################################################################
+# Test Case2: Improved error message with charset information for CHAR
+# type
+####################################################################
+[connection master]
+SET SQL_LOG_BIN=0;
+CREATE TABLE t1 (c1 CHAR(1) CHARACTER SET 'utf8mb3');
+SET SQL_LOG_BIN=1;
+[connection slave]
+CREATE TABLE t1 (c1 CHAR(1) CHARACTER SET 'utf8mb4');
+[connection master]
+INSERT INTO t1 VALUES ('a');
+[connection slave]
+include/wait_for_slave_sql_error.inc [errno=1677]
+Matching lines are:
+DATE_TIME [ERROR] Slave SQL for channel '': Column 0 of table 'test.t1' cannot be converted from type 'char(3(bytes))' to type 'char(4(bytes) utf8mb4)', Error_code: 1677
+Occurrences of '\'char\(3\(bytes\)\)\' to type \'char\(4\(bytes\) utf8mb4\)\'' in the input file: 1
+[connection master]
+DROP TABLE t1;
+[connection slave]
+DROP TABLE t1;
+include/rpl_reset.inc
+####################################################################
+# Test Case3: For BLOB type fileds, when type conversion failed on
+# slave, the errormessage had incorrect type names.
+####################################################################
+[connection master]
+SET SQL_LOG_BIN=0;
+CREATE TABLE t1 (c1 LONGBLOB);
+SET SQL_LOG_BIN=1;
+[connection slave]
+CREATE TABLE t1 (c1 TINYBLOB);
+[connection master]
+INSERT INTO t1 VALUES ('a');
+[connection slave]
+include/wait_for_slave_sql_error.inc [errno=1677]
+Matching lines are:
+DATE_TIME [ERROR] Slave SQL for channel '': Column 0 of table 'test.t1' cannot be converted from type 'longblob' to type 'tinyblob', Error_code: 1677
+Occurrences of '\'longblob\' to type \'tinyblob\'' in the input file: 1
+[connection master]
+DROP TABLE t1;
+[connection slave]
+DROP TABLE t1;
+include/rpl_reset.inc
+include/rpl_end.inc
diff --git a/mysql-test/suite/rpl/t/rpl_type_conv_utf8_utf8mb4-slave.opt b/mysql-test/suite/rpl/t/rpl_type_conv_utf8_utf8mb4-slave.opt
@@ -0,0 +1 @@
+--no-console --log-error=$MYSQLTEST_VARDIR/tmp/slave.err
diff --git a/mysql-test/suite/rpl/t/rpl_type_conv_utf8_utf8mb4.test b/mysql-test/suite/rpl/t/rpl_type_conv_utf8_utf8mb4.test
@@ -0,0 +1,137 @@
+# ==== Purpose ====
+#
+# Test verifies that when slave side type conversion fails in row based
+# replication, more informative error message is displayed. It also verifies
+# that in the case of blob fields appropriate type name is displayed in error
+# message.
+#
+# ==== Implementation ====
+#
+# Steps:
+# Test case1:
+#          1. Create a table on master with VARCHAR filed and charset
+#          'utf8mb3'.
+#          2. Create a table on slave with VARCHAR field and charset
+#          'utf8mb4'.
+#          3. Insert a tuple on master.
+#          4. Verify that slave provides more informative error message with
+#          respect to difference in charsets.
+# Test case2: Repeat same steps as above for CHAR field
+# Test case3:
+#          1. Create a table on master with LONGBLOB field.
+#          2. Create a table on slave with TINYBLOB field.
+#          3. Insert a tuple on master.
+#          4. Verify that error message displayed on slave clearly states type
+#          conversion failure from 'longblob' to 'tinyblob'.
+#          5. Also verify that error message doesn't show additional details
+#          of charset when not required.
+#
+# ==== References ====
+#
+# Bug#25135304: RBR: WRONG FIELD LENGTH IN ERROR MESSAGE
+#
+
+--source include/have_binlog_format_row.inc
+# Inorder to grep a specific error pattern in error log a fresh error log
+# needs to be generated.
+--source include/force_restart.inc
+--source include/master-slave.inc
+
+--echo ####################################################################
+--echo # Test Case1: Improved error message with charset information
+--echo ####################################################################
+--source include/rpl_connection_master.inc
+SET SQL_LOG_BIN=0;
+CREATE TABLE t1 (c1 VARCHAR(1) CHARACTER SET 'utf8mb3');
+SET SQL_LOG_BIN=1;
+
+--source include/rpl_connection_slave.inc
+CREATE TABLE t1 (c1 VARCHAR(1) CHARACTER SET 'utf8mb4');
+
+--source include/rpl_connection_master.inc
+INSERT INTO t1 VALUES ('a');
+
+--source include/rpl_connection_slave.inc
+--let $slave_sql_errno= convert_error(ER_SLAVE_CONVERSION_FAILED);
+--source include/wait_for_slave_sql_error.inc
+
+# Error msg before: Column 0 of table 'test.t1' cannot be converted from type 'varchar(3)' to type 'varchar(1)'
+# Error msg after : Column 0 of table 'test.t1' cannot be converted from type 'varchar(3(bytes))' to type 'varchar(4(bytes) utf8mb4)'
+--replace_regex /[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]:[0-9][0-9]\.[0-9][0-9][0-9][0-9][0-9][0-9][-+Z][0-9:]* *[0-9]* *\[/DATE_TIME [/ s/Worker .* end_log_pos [0-9]*; //
+--let $grep_file=$MYSQLTEST_VARDIR/tmp/slave.err
+--let $grep_pattern=\'varchar\(3\(bytes\)\)\' to type \'varchar\(4\(bytes\) utf8mb4\)\'
+--source include/grep_pattern.inc
+
+--source include/rpl_connection_master.inc
+DROP TABLE t1;
+--source include/rpl_connection_slave.inc
+DROP TABLE t1;
+--let $rpl_only_running_threads= 1
+--source include/rpl_reset.inc
+
+--echo ####################################################################
+--echo # Test Case2: Improved error message with charset information for CHAR
+--echo # type
+--echo ####################################################################
+--source include/rpl_connection_master.inc
+SET SQL_LOG_BIN=0;
+CREATE TABLE t1 (c1 CHAR(1) CHARACTER SET 'utf8mb3');
+SET SQL_LOG_BIN=1;
+
+--source include/rpl_connection_slave.inc
+CREATE TABLE t1 (c1 CHAR(1) CHARACTER SET 'utf8mb4');
+
+--source include/rpl_connection_master.inc
+INSERT INTO t1 VALUES ('a');
+
+--source include/rpl_connection_slave.inc
+--let $slave_sql_errno= convert_error(ER_SLAVE_CONVERSION_FAILED);
+--source include/wait_for_slave_sql_error.inc
+
+# Error msg before: Column 0 of table 'test.t1' cannot be converted from type  'char(0)' to type 'char(1)'
+# Error msg after : Column 0 of table 'test.t1' cannot be converted from type  'char(3(bytes))' to type 'char(4(bytes) utf8mb4)'
+--replace_regex /[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]:[0-9][0-9]\.[0-9][0-9][0-9][0-9][0-9][0-9][-+Z][0-9:]* *[0-9]* *\[/DATE_TIME [/ s/Worker .* end_log_pos [0-9]*; //
+--let $grep_file=$MYSQLTEST_VARDIR/tmp/slave.err
+--let $grep_pattern=\'char\(3\(bytes\)\)\' to type \'char\(4\(bytes\) utf8mb4\)\'
+--source include/grep_pattern.inc
+
+--source include/rpl_connection_master.inc
+DROP TABLE t1;
+--source include/rpl_connection_slave.inc
+DROP TABLE t1;
+--let $rpl_only_running_threads= 1
+--source include/rpl_reset.inc
+
+--echo ####################################################################
+--echo # Test Case3: For BLOB type fileds, when type conversion failed on
+--echo # slave, the errormessage had incorrect type names.
+--echo ####################################################################
+--source include/rpl_connection_master.inc
+SET SQL_LOG_BIN=0;
+CREATE TABLE t1 (c1 LONGBLOB);
+SET SQL_LOG_BIN=1;
+
+--source include/rpl_connection_slave.inc
+CREATE TABLE t1 (c1 TINYBLOB);
+
+--source include/rpl_connection_master.inc
+INSERT INTO t1 VALUES ('a');
+
+--source include/rpl_connection_slave.inc
+--let $slave_sql_errno= convert_error(ER_SLAVE_CONVERSION_FAILED);
+--source include/wait_for_slave_sql_error.inc
+
+# Error msg before: Column 0 of table 'test.t1' cannot be converted from type 'tinyblob' to type 'tinyblob'
+# Error msg after : Column 0 of table 'test.t1' cannot be converted from type 'longblob' to type 'tinyblob'
+--replace_regex /[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]:[0-9][0-9]\.[0-9][0-9][0-9][0-9][0-9][0-9][-+Z][0-9:]* *[0-9]* *\[/DATE_TIME [/ s/Worker .* end_log_pos [0-9]*; //
+--let $grep_file=$MYSQLTEST_VARDIR/tmp/slave.err
+--let $grep_pattern=\'longblob\' to type \'tinyblob\'
+--source include/grep_pattern.inc
+
+--source include/rpl_connection_master.inc
+DROP TABLE t1;
+--source include/rpl_connection_slave.inc
+DROP TABLE t1;
+--let $rpl_only_running_threads= 1
+--source include/rpl_reset.inc
+--source include/rpl_end.inc
diff --git a/sql/rpl_utility.cc b/sql/rpl_utility.cc
@@ -1,4 +1,4 @@
-/* Copyright (c) 2006, 2016, Oracle and/or its affiliates. All rights reserved.
+/* Copyright (c) 2006, 2017, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -162,7 +162,7 @@ static void show_sql_type(enum_field_types type, uint16 metadata, String *str,
       const CHARSET_INFO *cs= str->charset();
       size_t length=
         cs->cset->snprintf(cs, (char*) str->ptr(), str->alloced_length(),
-                           "varchar(%u)", metadata);
+                           "varchar(%u(bytes))", metadata);
       str->length(length);
     }
     break;
@@ -212,22 +212,22 @@ static void show_sql_type(enum_field_types type, uint16 metadata, String *str,
       it is necessary to check the pack length to figure out what kind
       of blob it really is.
      */
-    switch (get_blob_type_from_length(metadata))
+    switch (metadata)
     {
-    case MYSQL_TYPE_TINY_BLOB:
+    case 1:
       str->set_ascii(STRING_WITH_LEN("tinyblob"));
       break;
 
-    case MYSQL_TYPE_MEDIUM_BLOB:
-      str->set_ascii(STRING_WITH_LEN("mediumblob"));
+    case 2:
+      str->set_ascii(STRING_WITH_LEN("blob"));
       break;
 
-    case MYSQL_TYPE_LONG_BLOB:
-      str->set_ascii(STRING_WITH_LEN("longblob"));
+    case 3:
+      str->set_ascii(STRING_WITH_LEN("mediumblob"));
       break;
 
-    case MYSQL_TYPE_BLOB:
-      str->set_ascii(STRING_WITH_LEN("blob"));
+    case 4:
+      str->set_ascii(STRING_WITH_LEN("longblob"));
       break;
 
     default:
@@ -245,7 +245,7 @@ static void show_sql_type(enum_field_types type, uint16 metadata, String *str,
       uint bytes= (((metadata >> 4) & 0x300) ^ 0x300) + (metadata & 0x00ff);
       size_t length=
         cs->cset->snprintf(cs, (char*) str->ptr(), str->alloced_length(),
-                           "char(%d)", bytes / field_cs->mbmaxlen);
+                           "char(%d(bytes))", bytes);
       str->length(length);
     }
     break;
@@ -660,11 +660,11 @@ table_def::compatible_with(THD *thd, Relay_log_info *rli,
       const char *tbl_name= table->s->table_name.str;
       char source_buf[MAX_FIELD_WIDTH];
       char target_buf[MAX_FIELD_WIDTH];
+      String field_sql_type;
       enum loglevel report_level= INFORMATION_LEVEL;
       String source_type(source_buf, sizeof(source_buf), &my_charset_latin1);
       String target_type(target_buf, sizeof(target_buf), &my_charset_latin1);
       show_sql_type(type(col), field_metadata(col), &source_type, field->charset());
-      field->sql_type(target_type);
       if (!ignored_error_code(ER_SLAVE_CONVERSION_FAILED))
       {
         report_level= ERROR_LEVEL;
@@ -674,6 +674,24 @@ table_def::compatible_with(THD *thd, Relay_log_info *rli,
       else if (log_warnings > 1)
         report_level= WARNING_LEVEL;
 
+      if (field->has_charset() &&
+          (field->type() == MYSQL_TYPE_VARCHAR ||
+           field->type() == MYSQL_TYPE_STRING))
+      {
+        field_sql_type.append((field->type() == MYSQL_TYPE_VARCHAR) ?
+                              "varchar" : "char");
+        const CHARSET_INFO *cs= field->charset();
+        size_t length= cs->cset->snprintf(cs, (char*) target_type.ptr(),
+                                          target_type.alloced_length(),
+                                          "%s(%u(bytes) %s)",
+                                          field_sql_type.c_ptr_safe(),
+                                          field->field_length,
+                                          field->charset()->csname);
+        target_type.length(length);
+      }
+      else
+        field->sql_type(target_type);
+
       if (report_level != INFORMATION_LEVEL)
         rli->report(report_level, ER_SLAVE_CONVERSION_FAILED,
                     ER(ER_SLAVE_CONVERSION_FAILED),

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1 @@`
	`1`	`+--no-console --log-error=$MYSQLTEST_VARDIR/tmp/slave.err`