55

I'm trying to figure out how to optimize a very slow query in MySQL (I didn't design this):

SELECT COUNT(*) FROM change_event me WHERE change_event_id > '1212281603783391';
+----------+
| COUNT(*) |
+----------+
|  3224022 |
+----------+
1 row in set (1 min 0.16 sec)

Comparing that to a full count:

select count(*) from change_event;
+----------+
| count(*) |
+----------+
|  6069102 |
+----------+
1 row in set (4.21 sec)

The explain statement doesn't help me here:

 explain SELECT COUNT(*) FROM change_event me WHERE change_event_id > '1212281603783391'\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: me
         type: range
possible_keys: PRIMARY
          key: PRIMARY
      key_len: 8
          ref: NULL
         rows: 4120213
        Extra: Using where; Using index
1 row in set (0.00 sec)

OK, it still thinks it needs roughly 4 million entries to count, but I could count lines in a file faster than that! I don't understand why MySQL is taking this long.

Here's the table definition:

CREATE TABLE `change_event` (
  `change_event_id` bigint(20) NOT NULL default '0',
  `timestamp` datetime NOT NULL,
  `change_type` enum('create','update','delete','noop') default NULL,
  `changed_object_type` enum('Brand','Broadcast','Episode','OnDemand') NOT NULL,
  `changed_object_id` varchar(255) default NULL,
  `changed_object_modified` datetime NOT NULL default '1000-01-01 00:00:00',
  `modified` datetime NOT NULL default '1000-01-01 00:00:00',
  `created` datetime NOT NULL default '1000-01-01 00:00:00',
  `pid` char(15) default NULL,
  `episode_pid` char(15) default NULL,
  `import_id` int(11) NOT NULL,
  `status` enum('success','failure') NOT NULL,
  `xml_diff` text,
  `node_digest` char(32) default NULL,
  PRIMARY KEY  (`change_event_id`),
  KEY `idx_change_events_changed_object_id` (`changed_object_id`),
  KEY `idx_change_events_episode_pid` (`episode_pid`),
  KEY `fk_import_id` (`import_id`),
  KEY `idx_change_event_timestamp_ce_id` (`timestamp`,`change_event_id`),
  KEY `idx_change_event_status` (`status`),
  CONSTRAINT `fk_change_event_import` FOREIGN KEY (`import_id`) REFERENCES `import` (`import_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8

Version:

$ mysql --version
mysql  Ver 14.12 Distrib 5.0.37, for pc-solaris2.8 (i386) using readline 5.0

Is there something obvious I'm missing? (Yes, I've already tried "SELECT COUNT(change_event_id)", but there's no performance difference).

2
  • How about if you try something like... SELECT COUNT(*) FROM change_event me WHERE change_event_id > 0; Does it effect the performance? Feb 4, 2009 at 15:37
  • ovid - if you're able, please add the output of 'SHOW INDEX FROM change_event'
    – Alnitak
    Feb 4, 2009 at 16:03

8 Answers 8

59

InnoDB uses clustered primary keys, so the primary key is stored along with the row in the data pages, not in separate index pages. In order to do a range scan you still have to scan through all of the potentially wide rows in data pages; note that this table contains a TEXT column.

Two things I would try:

  1. run optimize table. This will ensure that the data pages are physically stored in sorted order. This could conceivably speed up a range scan on a clustered primary key.
  2. create an additional non-primary index on just the change_event_id column. This will store a copy of that column in index pages which be much faster to scan. After creating it, check the explain plan to make sure it's using the new index.

(you also probably want to make the change_event_id column bigint unsigned if it's incrementing from zero)

6
  • 8
    The "optimize table" didn't help much, but the redundant index solved the problem. Thanks!
    – Ovid
    Feb 4, 2009 at 16:49
  • 24
    This is the first time I've ever seen anyone suggest creating a redundant index on a PRIMARY KEY column as a performance hack in MySQL. I'm pretty interested in the details of why this works and the kinds of queries for which it is useful. Do you have any links to further reading on the topic?
    – Mark Amery
    Jun 16, 2014 at 10:43
  • OPTIMIZE TABLE is rarely of use, especially on InnoDB tables. Any improvement may be because you freshly loaded the entire table into cache.
    – Rick James
    Feb 23, 2017 at 2:52
  • 1
    @MarkAmery MySQL innodb format stores all row data in the primary index; if you don't have a primary key, one is synthesized for use in the storage index. This means that rather than it being an index over bigints, it's an index over the whole data tuple, so it has to stride through, so it's not fast to scan. Mar 29, 2018 at 13:16
  • 2
    @MarkAmery for more details, see dev.mysql.com/doc/refman/5.7/en/innodb-index-types.html - primary key index is clustered index for row storage - it's subtle implication, blink and you miss it. Mar 29, 2018 at 13:18
15

Here are a few things I suggest:

  • Change the column from a "bigint" to an "int unsigned". Do you really ever expect to have more than 4.2 billion records in this table? If not, then you're wasting space (and time) the the extra-wide field. MySQL indexes are more efficient on smaller data types.

  • Run the "OPTIMIZE TABLE" command, and see whether your query is any faster afterward.

  • You might also consider partitioning your table according to the ID field, especially if older records (with lower ID values) become less relevant over time. A partitioned table can often execute aggregate queries faster than one huge, unpartitioned table.


EDIT:

Looking more closely at this table, it looks like a logging-style table, where rows are inserted but never modified.

If that's true, then you might not need all the transactional safety provided by the InnoDB storage engine, and you might be able to get away with switching to MyISAM, which is considerably more efficient on aggregate queries.

12
  • 1
    Given that we have numbers like "1212281603783397", I think that already overflows "int unsigned" (it's a high-res timestamp). "OPTIMIZE TABLE" had no performance impact :( Isn't MyISAM much slower with "where" clauses since it needs to do a table scan? Also, we'd lose our FK constraint.
    – Ovid
    Feb 4, 2009 at 16:36
  • Why use a timestamp for your primary key, if you already have a timestamp field? Also, what happens if two events happen at the same instant? If I were you, I'd use a simple auto-increment field for the pkey.
    – benjismith
    Feb 4, 2009 at 16:43
  • The WHERE clause doesn't necessarily cause a full table scan. For a simple query (equals, less-than, greater-than, etc) on an indexed column, the query optimizer uses the index to find relevant pages, and then only scans those pages. A FTS would be required if you were doing date-math or substrings.
    – benjismith
    Feb 4, 2009 at 16:47
  • 1
    an auto-increment key might actually be suboptimal for a logging table in innodb as it requires a brief full table lock in order to acquire the next increment.
    – ʞɔıu
    Feb 4, 2009 at 16:47
  • Good point. I was actually thinking in terms of MyISAM when I made that suggestion, since I see no reason for this table to use InnoDB, since it isn't really transactional.
    – benjismith
    Feb 4, 2009 at 16:50
5

I've run into behavior like this before with IP geolocation databases. Past some number of records, MySQL's ability to get any advantage from indexes for range-based queries apparently evaporates. With the geolocation DBs, we handled it by segmenting the data into chunks that were reasonable enough to allow the indexes to be used.

4
  • What a nasty solution. Nonetheless, I brought it up earlier and barring some strange configuration fix or other solution, we might be forced to go this route :(
    – Ovid
    Feb 4, 2009 at 15:52
  • This is a great solution that respects a basic principle of computer solutions: programming in-the-large is qualitatively different from programming in-the-small. In the case of databases, the access plans and the use of indexes changes dramatically as size increases past certain thresholds. Feb 4, 2009 at 16:13
  • I came across a similar problem with geolocation database, and after various optimization attempts like indexing, partitioning etc i just gave a shot to dividing large tables into smaller dataset, which finally proved to be acceptable in terms of performance.
    – shashi009
    Apr 19, 2016 at 4:49
  • For a geolocation database, you should definitely use MyISAM
    – Tofandel
    May 4, 2021 at 11:09
3

Check to see how fragmented your indexes are. At my company we have a nightly import process that trashes our indexes and over time it can have a profound impact on data access speeds. For example we had a SQL procedure that took 2 hours to run one day after de-fragmenting the indexes it took 3 minutes. we use SQL Server 2005 ill look for a script that can check this on MySQL.

Update: Check out this link: http://dev.mysql.com/doc/refman/5.0/en/innodb-file-defragmenting.html

2
1

Run "analyze table_name" on that table - it's possible that the indices are no longer optimal.

You can often tell this by running "show index from table_name". If the cardinality value is NULL then you need to force re-analysis.

3
  • "analyze table change_event" had no impact on performance. Thanks, though.
    – Ovid
    Feb 4, 2009 at 15:44
  • did it make the plain "select count()" any faster? I've just tried on a 110M record MyISAM table. "select count()" was instant. Selecting the count for ~half the table took 2m48 the first time, and 27s the second time.
    – Alnitak
    Feb 4, 2009 at 15:52
  • 2
    MyISAM has radically different performance characteristics from InnoDB. That's because MyISAM does table level locking and effectively only has one transaction at a time. InnoDB behaves much differently under the covers.
    – Ovid
    Feb 4, 2009 at 15:56
1

MySQL does say "Using where" first, since it does need to read all records/values from the index data to actually count them. With InnoDb it also tries to "grab" that 4 mil record range to count it.

You may need to experiment with different transaction isolation levels: http://dev.mysql.com/doc/refman/5.1/en/set-transaction.html#isolevel_read-uncommitted

and see which one is better.

With MyISAM it would be just fast, but with intensive write model will result in lock issues.

0

To make the search more efficient, although I recommend adding index. I leave the command for you to try the metrics again

CREATE INDEX ixid_1 ON change_event (change_event_id);

and repeat query

SELECT COUNT(*) FROM change_event me WHERE change_event_id > '1212281603783391';

-JACR

-1

I would create a "counters" table and add "create row"/"delete row" triggers to the table you are counting. The triggers should increase/decrease count values on "counters" table on every insert/delete, so you won't need to compute them every time you need them.

You can also accomplish this on the application side by caching the counters but this will involve clearing the "counter cache" on every insertion/deletion.

For some reference take a look at this http://pure.rednoize.com/2007/04/03/mysql-performance-use-counter-tables/

1
  • 1
    Except that we need counts on ranges, so a managing a count via triggers doesn't work (unless I've misunderstood you).
    – Ovid
    Feb 4, 2009 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.