4

I am currently uploading datafeeds into my database.In my database were dumped because of more than 1crores records are there.I need to improve or increase my MYSQL query performance in my site.Here my executed query below....

select SUM(SPRICE) AS Tot, MIN(SMIN) AS Min from 
(SELECT COUNT(LS.SALEPRICE) AS SPRICE, MIN(LS.SALEPRICE) AS SMIN 
 FROM `linkshare` LS 
 WHERE LS.`PRODUCTNAME` LIKE '%DVS Men\'s Comanche Skate Shoe%' 
 UNION 
 SELECT COUNT(CJ.PRICE) AS SPRICE, MIN(CJ.PRICE) AS SMIN 
 FROM `cjfeeds` CJ 
 WHERE CJ.NAME LIKE '%DVS Men\'s Comanche Skate Shoe%' ) AS xyz

In this above query, its working perfectly in local database and my database contain less than 50 thousand records...How to improve my query in live server ?please guide me.....

Explain Query

Also My Query took 39.4626 sec. How can i reduce this query running time?

3
  • You need to normalize your database. Mar 30, 2013 at 11:24
  • Pick something other than Fisher Price My First SQL Server? And if you want a serious comment: your LIKE with wildcard matching and the start and end of your strings is likely doing the most harm to your query. Mar 30, 2013 at 11:28
  • 1
    Note that you should be using UNION ALL rather than UNION (which defaults to UNION DISTINCT) in your query.
    – user359040
    Mar 30, 2013 at 11:29

2 Answers 2

3

ok, going to edit my answer to first deal more specifically with your query, the earlier advice would work but your query is fairly insane so let's discuss why.

Everything you need is actually in the EXPLAIN output here, your UNION is causing 3.4 million tuple accesses and the derived table query (after the concatenation) is ~0.9million.

  • Add an index on PRODUCTNAME in both tables

  • UNION? wtf? I assume what's going on here is you have two fairly similar/identical tables and you're doing a doing a UNION of this fairly dodgy filter query to basically concat one on to the other. This is warning sign number one, this query would be faster if you can simplify this and have one table with a type enum, e.g. type (LS|CJ) or a foreign key and a types table depending on your requirements.

  • Assuming you don't want to do that permanently for some reason, (and you should), you can create a temporary table for this computation from the two selects. Once you have all the info in one table, because you're doing a simple select your count, sum will be fast.

MySQL has an EXPLAIN command which you can prefix to any query, e.g.

EXPLAIN select SUM(SPRICE) AS Tot, MIN(SMIN) AS Min from (SELECT COUNT(LS.SALEPRICE) AS SPRICE, MIN(LS.SALEPRICE) AS SMIN FROM `linkshare` LS WHERE LS.`PRODUCTNAME` LIKE '%DVS Men\'s Comanche Skate Shoe%' UNION SELECT COUNT(CJ.PRICE) AS SPRICE, MIN(CJ.PRICE) AS SMIN FROM `cjfeeds` CJ WHERE CJ.NAME LIKE '%DVS Men\'s Comanche Skate Shoe%' ) AS xyz;

The output can be somewhat cryptic for beginners, check out a tutorial on it for more info. In general:

  • Avoid 'LIKE %blah%' style queries where possible, as Mark Bannister suggested these will not use any indexes you create.
  • Create an index on any fields used in selects (in tables with more than a thousand rows).
  • Keep your quickly-growing tables as lean as possible
  • Use fixed width columns where possible, e.g. char/varchar instead of TEXT/BLOB
  • If you're running a compound slow query on a large data set, consider caching it/tuning your my.cnf table cache size.

    In summary, always try to do exact string matches as these can be indexed. Your issue stems from a poorly normalized table structure. Normalized just means (in a high-level non-techy way) that you've organized your data in a way that makes it less duplicated, and therefore more consistent. The bonus is that it makes it easier to do efficient queries on it. If you think you need wildcard queries you probably need to categorize your products e.g. into categories like 'shoes', to do this, add a product_categories table with a schema like |category_id, category_name|. Then in your products table (if a product can be in only one category) add a foreign key e.g. category_id, add an index to the category_id field, then query the products by category_id

e.g. select * FROM products where category_id=5

If you think you need to fuzzy match on your data it really sounds as though it's a bit disorganized. If it's unavoidable, see if your devops people can set up a read slave so your slow queries don't hurt any important systems.

5
  • It is useless to tweak an engine if you assembled it wrong way. Mar 30, 2013 at 11:32
  • 1
    Note that like 'blah%' may be able to make use of an index, but like '%blah%' is unlikely to ever use an index.
    – user359040
    Mar 30, 2013 at 11:37
  • i created index in price field. but how can i use this index in my query. Mar 30, 2013 at 11:46
  • Hi @Mark, i updated the explain query screen shot. Kindly give me some suggestions. Mar 30, 2013 at 13:53
  • As @MarkBannister mentioned, always avoid '%blah%' queries as it will not use the index as the index will match some prefix of the left side of the string (if you've created one on product name). If you haven't, then add one and use a query with an exact comparison which is amenable to indexing e.g. product_name = 'some name'.
    – Mark
    Jul 13, 2015 at 9:17
0

Use EXPLAIN to find out what is going on under the bonnet

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.