2

I am trying to figure out the settings in mySQL on indexes. When does mySQL ignore an index?

Here are the results of an experiment. I have a table with an index on the AGE column as follows.

CREATE TABLE `USERS` (
  `ID` int(11) NOT NULL,
  `FIRSTNAME` varchar(45) NOT NULL,
  `LASTNAME` varchar(45) DEFAULT NULL,
  `USERNAME` varchar(45) DEFAULT NULL,
  `ROLE` int(11) DEFAULT NULL,
  `PASSWORD` varchar(45) DEFAULT NULL,
  `AGE` int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

ALTER TABLE `USERS`
  ADD PRIMARY KEY (`ID`),
  ADD KEY `AGE` (`AGE`);

Results of EXPLAIN queries. The first three statements use the index. The second set of statements ignore the index and do a full table scan.

The range of AGE is random between 20 and 100 years. There are 1000 rows in the table.

/* utilizes the index on AGE */
/* case 1 */
SELECT ID, AGE  FROM USERS WHERE AGE > 20; 

/* case 2 */
SELECT AGE  FROM USERS WHERE AGE > 44;

/* case 3 */
SELECT * FROM USERS WHERE AGE > 84;


/* does not use index on AGE */

/* case 4 */
SELECT AGE, FIRSTNAME FROM USERS WHERE AGE > 83;

/* case 5 */
SELECT * FROM USERS WHERE AGE > 83;

/* case 6 */
SELECT AGE FROM USERS WHERE AGE > 18;

Some observations that I have seen. Can anyone confirm my conclusions are correct?

1) SELECT * will use the index when 15% or fewer of the rows are selected. 2) SELECT AGE will use the index when 1 or more rows are selected.

2
  • Use formatting tools to make your post more readable. Use code blocking for code and log and error texts and bold and italics to highlight things
    – Morse
    Commented Jul 13, 2018 at 1:33
  • I presume these are all tested against the same table. Case 3 and case 5 are particularly puzzling. Possibly retest with caching disabled (i.e. `SELECT SQL_NO_CACHE <rest of query>).
    – Joshua R.
    Commented Jul 13, 2018 at 2:32

1 Answer 1

3

The 15% is usually around 20%, and it depends on statistics gleaned from data in the table. I saw someone nail down about 29% as the cutoff in his query. You are actually hitting 20%:

(100-85+1)/(100-20+1) = 19.8%
(100-84+1)/(100-20+1) = 21.0%

So, that explains cases 3, 4, 5. The rationale is that for a high percentage, a table scan is more efficient than bouncing between the index BTree and the data+PK BTree.

The index is "covering" for these two. That is, all the necessary columns are found in a single index. Hence, it should use the index, not do a table scan:

SELECT AGE ...
SELECT ID, AGE ...

Note: In InnoDB, secondary indexes include the PRIMARY KEY implicitly. That is, INDEX(age) is virtually the same as INDEX(age, id).

That explains case 1 and 2, but fails to explain case 6. Case 6 should have used the index to return the entire list of AGEs. (not 0 rows, as your comment states??)

Your tests are the tip of the iceberg, but you have gotten a lot farther than most beginners at trying to fathom the depths of MySQL's relatively simple Optimizer. (I have several years' head start on you.)

There are a few more Rules of Thumb here .

Please continue experimenting and publishing your results.

2
  • Great response, but not clear to me how 3 (used index) and 5 (didn't) are explained by your first statement. Could you elaborate?
    – Joshua R.
    Commented Jul 13, 2018 at 2:45
  • 1
    @JoshuaR. - Case 3 had fewer index rows to touch, so the back and forth was deemed not-too-costly. Case 5 had more.
    – Rick James
    Commented Jul 13, 2018 at 3:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.