7

We recently migrated our Magento application from a private host to AWS web services. We noticed that some of the internal functionality of Magento was taking an excessively long period of time to execute after the migration, so started to investigate.

One of the queries in question is a simple customer select query, with around 9-10 regular joins to attribute tables to get the attributes.

We have run some tests on the query and found that the difference between the old host and AWS is that on the old host, the MySQL optimizer appears to use the correct index, whereas in AWS it resorts to using filesort, ignoring the index.

Using FORCE INDEX(index_name) makes the query execute correctly in AWS, however we don't want to go down this road and would rather fix the issue in the database configuration than make manual hacks throughout our Magento application. To be clear, this is not an issue with our indexes, they are set up correctly.

For background:

  • I have copied all the MySQL parameters from the old host's my.cnf file to a parameter group in RDS, but nothing made any difference
  • The tables are all InnoDB
  • I've run analyze, repair and optimize queries etc
  • The query takes around 45 seconds to complete on RDS
  • The query took around 2 seconds to complete on the old host, or when I use FORCE INDEX() to force RDS to behave in the same way as the old host

The old MySQL server was running version 5.1.61, and the AWS RDS instance we are running is on 5.6.19. A consulting group suggested to us that we downgrade our RDS instance to 5.1.61, however again we don't want to do this as it is not a sustainable solution.

The query in question is below (shortened by removing fields from the select for the sake of space):

SELECT
    `e`.*
    -- various field names here, removed
FROM `customer_entity` AS `e`
LEFT JOIN `customer_entity_int` AS `at_default_billing` ON (`at_default_billing`.`entity_id` = `e`.`entity_id`) AND (`at_default_billing`.`attribute_id` = '13')
LEFT JOIN `customer_address_entity_varchar` AS `at_billing_postcode` ON (`at_billing_postcode`.`entity_id` = `at_default_billing`.`value`) AND (`at_billing_postcode`.`attribute_id` = '30')
LEFT JOIN `customer_address_entity_varchar` AS `at_billing_city` ON (`at_billing_city`.`entity_id` = `at_default_billing`.`value`) AND (`at_billing_city`.`attribute_id` = '26')
LEFT JOIN `customer_address_entity_varchar` AS `at_billing_telephone` ON (`at_billing_telephone`.`entity_id` = `at_default_billing`.`value`) AND (`at_billing_telephone`.`attribute_id` = '31')
LEFT JOIN `customer_address_entity_varchar` AS `at_billing_regione` ON (`at_billing_regione`.`entity_id` = `at_default_billing`.`value`) AND (`at_billing_regione`.`attribute_id` = '28')
LEFT JOIN `customer_address_entity_varchar` AS `at_billing_country_id` ON (`at_billing_country_id`.`entity_id` = `at_default_billing`.`value`) AND (`at_billing_country_id`.`attribute_id` = '27')
LEFT JOIN `core_store` AS `at_store_name` ON (at_store_name.`store_id`=e.store_id)
LEFT JOIN `customer_entity_varchar` AS `firstname` ON e.entity_id = firstname.entity_id AND firstname.attribute_id = 5
LEFT JOIN `customer_entity_varchar` AS `lastname` ON e.entity_id = lastname.entity_id AND lastname.attribute_id = 7 

WHERE (`e`.`entity_type_id` = '1') 

ORDER BY `e`.`entity_id`
DESC LIMIT 20;

A summary of the differences between servers when EXPLAINing this query is below:

RDS - MySQL v5.6.19:

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: e
         type: ref
possible_keys: IDX_CUSTOMER_ENTITY_ENTITY_TYPE_ID
          key: IDX_CUSTOMER_ENTITY_ENTITY_TYPE_ID
      key_len: 2
          ref: const
         rows: 653990
        Extra: Using temporary; Using filesort

Old host - MySQL v5.1.61, or when FORCE INDEX is used on RDS:

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: e
         type: ref
possible_keys: IDX_CUSTOMER_ENTITY_ENTITY_TYPE_ID
          key: IDX_CUSTOMER_ENTITY_ENTITY_TYPE_ID
      key_len: 2
          ref: const
         rows: 644775
        Extra: Using where

I'm aware that it's highly likely that the difference in those database versions has changed the way the query optimizer works, and I'm not aware of what the differences in those versions are, but I'm looking for a solution that will help us to address those differences.


Edit: here's a comparison list from RDS for the parameters I've copied over from the old host vs the default parameters for this MySQL version. None of these parameters have affected the result above whether they are there or whether I used standard parameters in RDS:

enter image description here

14
  • Hard to answer on Stack Overflow, since it's likely going to be some small subtle part of your configuration that fixes it. Posting the table definitions somewhere might help. Nov 2, 2014 at 17:18
  • I can post the table definitions if you'd like, but as I say they're standard Magento tables
    – scrowler
    Nov 4, 2014 at 10:30
  • What happens (in terms of index usage) if you remove the order by in your query on RDS and do the explain plan again? Nov 4, 2014 at 15:39
  • Hi Brian, removing the ORDER BY stops the query from using indexes or filesort entirely, as it doesn't need to order anything.
    – scrowler
    Nov 4, 2014 at 19:27
  • @scrowler I doubt it has to do something with the memory allocation to MYSQL. How much you allotted to Mysql in your old and new server?
    – Kalpesh
    Nov 4, 2014 at 21:02

3 Answers 3

1
+500

Looking at the query you provided, it looks like you might be getting snagged by Bug #74030. A patch was contributed for a later version (5.6.20 and 5.7.4), but hasn't been applied yet, looking at the release notes. It might be good to sign-in and vote for that bug.

For now, as much as it pains me, your consultant group may be right (accidentally, perhaps)..until the patch is applied.

1

My bet is on MySQL conf:

  • metadata stats: from 5.6 innodb_stats_on_metadata default value changed to off. It can really change the query plan.
  • now that you already ran a bunch of queries on your new installation, you could still use mysqltuner script, it's always good trying.
  • please note that 5.6 changed default innodb_file_per_table to 1 and Query Cache is disabled. You can find other default changes here: https://blogs.oracle.com/supportingmysql/entry/server_defaults_changes_in_mysql
0

This looks like a case mentioned in the documentation regarding the optimizer not being able to figure out the proper index because the key used to select the data is different than the one used to sort the data.

I can't speak to any changes from version to version, but here is the blurb that speaks to this case:

In some cases, MySQL cannot use indexes to resolve the ORDER BY, although it still uses indexes to find the rows that match the WHERE clause. These cases include the following:

The key used to fetch the rows is not the same as the one used in the ORDER BY:

SELECT * FROM t1 WHERE key2=constant ORDER BY key1;

Here is the documentation in question, it offers up some suggestions to correct things. I'm not sure what you have tried already though.

7
  • Thanks, but I've mentioned that it's not an index issue :-)
    – scrowler
    Nov 4, 2014 at 18:04
  • My answer attempts to tell you why it's not using the index, not that you have index issues. Your options might include adding another index (you already said you don't need to do this) to cover this situation or might tell you how to rewrite your query so the optimizer will use the index that's already created. See what happens when you remove the order by. Does it use the index you expect? If so, you should be able to figure out how to rewrite your query so the optimizer can use what indexes you have. Nov 4, 2014 at 18:32
  • Hi Brian, thanks for your info. I have already been through the documentation and ensured that the correct indexes are in place. I've ensured that they are correct by recreating every possible combination of column indexes, including up to three columns in some cases, and have found that the existing indexes still take preference in the list of indexes that show in the explain query, although none of them get used. In this particular case, I've created an index on entity_typd_id and entity_id, and vice-versa - although the tables already contain these grouped indexes.
    – scrowler
    Nov 4, 2014 at 19:18
  • Just thought I'd add a bit more info - rewriting this query would fix the problem of course, however the symptoms of this query are being shown throughout the entire application, so rewriting is really out of the question.. And further, this query and the others affected are essentially straight-out-of-the-box Magento queries
    – scrowler
    Nov 4, 2014 at 19:19
  • I didn't realize more than one query was a problem. So this is really an example of one query that is affected by the version differences then? Nov 4, 2014 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.