3

I have a huge table in a MariaDb (10.4.10-MariaDB-1:10.4.10+maria~bionic) and I am adding a new column using

alter table Appointment add column responsible_organization varchar(256);

The existing table is this:

CREATE TABLE `Appointment` (
    `id` VARCHAR(256) NOT NULL,
    `version` VARCHAR(24) NOT NULL,
    `repetition_ref` VARCHAR(256) NULL DEFAULT NULL,
    `type` VARCHAR(256) NULL DEFAULT NULL,
    `comment` VARCHAR(2048) NULL DEFAULT NULL,
    `description` VARCHAR(2048) NULL DEFAULT NULL,
    `end` DATETIME NULL DEFAULT NULL,
    `start` DATETIME NULL DEFAULT NULL,
    `status` VARCHAR(256) NULL DEFAULT NULL,
    `statuschangedate` DATETIME NULL DEFAULT NULL,
    `deliverystatus` VARCHAR(256) NULL DEFAULT NULL,
    `reasoncancelled` VARCHAR(256) NULL DEFAULT NULL,
    `visit_type` VARCHAR(256) NULL DEFAULT NULL,
    `modified_db_time` TIMESTAMP(3) NOT NULL DEFAULT current_timestamp(3) ON UPDATE current_timestamp(3),
    `markedasdeleted` DATETIME NULL DEFAULT NULL,
    PRIMARY KEY (`id`, `version`),
    INDEX `FKc4f6e4y3ftaya162pwf7v4uj4` (`deliverystatus`),
    INDEX `FKeuhxsh83rlweegn404penommb` (`reasoncancelled`),
    INDEX `FK5xmurewn61wf4n3of5yx2nsmg` (`visit_type`),
    INDEX `modified_db_time` (`modified_db_time`),
    CONSTRAINT `FK5xmurewn61wf4n3of5yx2nsmg` FOREIGN KEY (`visittype`) REFERENCES `Coding` (`id`),
    CONSTRAINT `FKc4f6e4y3ftaya162pwf7v4uj4` FOREIGN KEY (`deliverystatus`) REFERENCES `CodeableConcept` (`id`),
    CONSTRAINT `FKeuhxsh83rlweegn404penommb` FOREIGN KEY (`reasoncancelled`) REFERENCES `CodingDt` (`id`)
)
;

As far as I read the MariaDb documentation it should choose the most efficient algorithm if I don't specify any. I would expect it to use INPLACE at the minimum. But when I run it i can see in the process list that it is running with the state "Copy to tmp table". So this is the COPY algorithm right?

I then tried to force it to use INSTANT as sugested by @o-jones. That gave me this output:

MariaDB [mydb]> alter table Appointment add column responsibleorganisation varchar(256), ALGORITHM=INSTANT;
ERROR 1846 (0A000): ALGORITHM=INSTANT is not supported. Reason: Cannot change column type. Try ALGORITHM=COPY

Weird, since I am adding a column.

I am wondering if it has something to do with the table being created on an older version of MariaDb and not having been rebuilt recently. I have found references to this being an issue for tables with old style temporal columns.

The variables old_alter_table and alter_algorithm both have the value DEFAULT

My table has 110+ million rows, so I would have liked to find a way to optimize this.

Any ideas?

3
  • 1
    Does specifying ALGORITHM=INSTANT throw an error? How about ALGORITHM=INPLACE? According to the docs, either of these may take a lot of time, but allow the table to be used while the ALTER operation is in progress. That's called "online DDL.` And, even the faster algorithms may need tmp table support -- that simply means a copy, possibly in RAM, of some data.
    – O. Jones
    Commented Jan 12, 2022 at 11:34
  • Just got my DBA to kill the process and try with INSTANT. MariaDB [mydb]> alter table Appointment add column responsibleorganisation varchar(256), ALGORITHM=INSTANT; ERROR 1846 (0A000): ALGORITHM=INSTANT is not supported. Reason: Cannot change column type. Try ALGORITHM=COPY Weird the error does not seem to match the alter statement. I am not changing a column I am adding one.
    – Søren
    Commented Jan 12, 2022 at 13:27
  • Sometimes ALTER TABLE thinks it has to change some column's type even when that's not what you specify. Looks like this is one of those times. Sigh.
    – O. Jones
    Commented Jan 12, 2022 at 13:53

2 Answers 2

4

MariaDB Server 10.4.10 was released over 2 years ago, in November 2019. Is this repeatable with a more recent version? There have been some fixes to ALTER TABLE since then.

If the problem is repeatable with the latest version in the 10.4 series, I would suggest that you file a bug report at https://jira.mariadb.org with a minimal reproducible test case (CREATE TABLE and ALTER TABLE statements).

With a more recent version, the answer could have been that MDEV-20590 introduced a way to disable instant operations that involve changing the data file format. SET GLOBAL innodb_instant_alter_column_allowed=never; would make ADD COLUMN always rebuild the table, like it did before MariaDB Server 10.3.

1
  • Works on db-fiddle on 10.3.32 and 10.4.22 (after FK typo visittype -> visittype).
    – danblack
    Commented Feb 3, 2022 at 3:46
2

I have seen this in cases where a database was created in an earlier version of MariaDB. Sometimes the storage format for some columns has changed, but the upgrade did not rewrite all of the data. Instead it waits until certain schema changes are made and then it modifies that underlying data format. In my cases it was because some of the columns were date/datetime and the temporal storage format had changed in 10.3, but many of my tables were still in the old format. Any schema change I made in 10.4 wanted to update the format for all of those columns, which was a pain for tables with many millions of rows.

If you rebuild the table first using "ALTER TABLE tab_name ENGINE = InnoDB;" then the column add would be able to be added "instantly". Although the alter will take as long as the non-instant column add would. However, there are circumstances where doing the processes separately might be advantageous.

In my case I had to do a manual table rebuild because I could not allow the live table to be locked for as long as the rebuild would take. So I wrote a stored procedure to take the table and create a copy of it gradually, and keep the copy up-to-date. The copy process was throttled to avoid putting to much strain on the db and then once the copy was ready I was able to swap the old and new tables quickly with renames so that the down time was measured in seconds instead of hours. Once the new table was in place I was able to do instant ddl operations on it like normal.

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.