1

I have noticed, that I am having problem with writing SQL queries, because of the problem with Mysqld, in peak hours. It causes my website to load 3-5 times slower than usually. So I'have tried siege -d5 -c150 http://mydomain.com/ and looked into top and my mysqld takes over 700% of CPU! I've also noticed in mysql status: Copying to tmp table and queries adding to some queue or something like this.

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
25877 mysql     20   0 1076m 227m 8268 S 749.0  2.8 224:02.21 mysqld

This is my query

SELECT COUNT(downloaded.id) AS downloaded_count
    , downloaded.file_name
    ,uploaded.* 
FROM `downloaded` JOIN uploaded 
ON downloaded.file_name = uploaded.file_name 
WHERE downloaded.completed = '1' 
AND uploaded.active = '1' 
AND uploaded.nsfw = '0' 
AND downloaded.datetime > DATE_SUB(NOW(), INTERVAL 7 DAY) 
GROUP BY downloaded.file_name 
ORDER BY downloaded_count DESC LIMIT 30;

Showing rows 0 - 29 ( 30 total, Query took 0.1639 sec) //is this that much? shouldn't it be 0.01s instead?

UPDATED: (removed ORDER BY)

Showing rows 0 - 29 ( 30 total, Query took 0.0064 sec) Why ORDER BY makes it 20x slower?

EXPLAIN

+----+-------------+------------+------+---------------+-----------+---------+--------------------------+------+----------------------------------------------+
| id | select_type | table      | type | possible_keys | key       | key_len | ref                      | rows | Extra                                        |
+----+-------------+------------+------+---------------+-----------+---------+--------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | uploaded   | ALL  | file_name_up  | NULL      | NULL    | NULL                     | 3139 | Using where; Using temporary; Using filesort |
|  1 | SIMPLE      | downloaded | ref  | file_name     | file_name | 767     | piqik.uploaded.file_name |    8 | Using where                                  |
+----+-------------+------------+------+---------------+-----------+---------+--------------------------+------+----------------------------------------------+

table: uploaded (Total 720.5 KiB)

CREATE TABLE IF NOT EXISTS `uploaded` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `sid` int(1) NOT NULL,
  `file_name` varchar(255) NOT NULL,
  `file_size` varchar(255) NOT NULL,
  `file_ext` varchar(255) NOT NULL,
  `file_name_keyword` varchar(255) NOT NULL,
  `access_key` varchar(40) NOT NULL,
  `upload_datetime` datetime NOT NULL,
  `last_download` datetime NOT NULL,
  `file_password` varchar(255) NOT NULL DEFAULT '',
  `nsfw` int(1) NOT NULL,
  `votes` int(11) NOT NULL,
  `downloads` int(11) NOT NULL,
  `video_thumbnail` int(1) NOT NULL DEFAULT '0',
  `video_duration` varchar(255) NOT NULL DEFAULT '',
  `video_resolution` varchar(11) NOT NULL,
  `video_additional` varchar(255) NOT NULL DEFAULT '',
  `active` int(1) NOT NULL DEFAULT '1',
  PRIMARY KEY (`id`),
  FULLTEXT KEY `file_name_keyword` (`file_name_keyword`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8 AUTO_INCREMENT=3328 ;

table: downloaded (Total 5,152.0 KiB)

CREATE TABLE IF NOT EXISTS `downloaded` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `file_name` varchar(255) NOT NULL,
  `completed` int(1) NOT NULL,
  `client_ip_addr` varchar(40) NOT NULL,
  `client_access_key` varchar(40) NOT NULL,
  `datetime` datetime NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB  DEFAULT CHARSET=utf8 AUTO_INCREMENT=31475 ;

(not sure why I've chosen InnoDB here)

Please note, that I am (still) not using indexes (which as I read is very important!) because of lack of knowledge and I am not sure how to add them correctly.

So the question is, how to improve this query to prevent webserver from slow loading of website? I have only "few" records and can not believe I am having so major problems, people here deal with millions of records and their projects work. How do webhosting companies prevent this problem? (I am hosting only my webpages with over 150 concurrent clients)

Additional info:

Mysql: 5.5.33

Nginx 1.2.1, php5-fpm

Debian 7.1 Wheezy

2x L5420 @ 2.50GHz

8GB RAM

6
  • Please explain, why downvoting and voting for close. Jan 23, 2014 at 8:07
  • I'd say that the answer to the question "why is my SELECT so slow?" is "you have no indexes", anyway this is probably a question better suited for dba.stackexchange.com (more knowledgeable audience over there about databases).
    – Alex
    Jan 23, 2014 at 8:12
  • When asking for mysql query optimization, if you look at other questions that ask for the same - you'll notice that either in comments or in answers people ask for output of EXPLAIN SELECT <your query>, what engine is being used and what the configuration variables are. Without those, and without knowing what kind of hard disk you're using - it's impossible to tell you what you might have done wrong and how to fix it. And that's probably why there's the downvote and close vote.
    – N.B.
    Jan 23, 2014 at 9:12
  • I've added explain, hope now its all :) Jan 23, 2014 at 12:13
  • 1
    Your query returns 3139 rows. After that, it goes through all 3139 rows to order them, after which it discards 3109 rows in order to return 30 rows back (that's how LIMIT works). However, this operation should be cached so if you were to re-run this query a few times, you'll get cached result out. If you want to further improve performance, you should have both engines as InnoDB and larger buffer_pool_size variable so that HDD isn't involved when dealing with this query.
    – N.B.
    Jan 23, 2014 at 12:38

1 Answer 1

2

A few observations:

  • You may not have actively chosen InnoDB as a storage engine: it will be the default engine for your version of MySQL. It's probably the right choice for your context, though, as it offers row-level locking instead of table-level locking (amongst other things) which you likely want.
  • Don't quote your integers in your comparisons (eg uploaded.active = '1'). You'll end up with slower string comparison, instead of integer comparison.
  • The comparison downloaded.datetime > DATE_SUB(NOW(), INTERVAL 7 DAY) with a derived value is going to be slower than comparison with a normal column value.

Regarding the last point, you could replace this with a user defined variable declared before the query:

SET @one_week_ago := DATE_SUB(NOW(), INTERVAL 7 DAY);

and then within the query compare to that pre-computed value:

...
downloaded.datetime > @one_week_ago
...

More importantly, though, you'll definitely want to have an index on any key that you're joining on.

In this case, you can add them by:

CREATE INDEX idx_file_name ON uploaded(file_name);
CREATE INDEX idx_file_name ON downloaded(file_name);

If you don't have indices, you're going to end up with multiple full table scans, which is slow.

There is a cost to adding an index: it takes up space, and it also means writes to the table are slower because the index has to be updated to include them. If this is a query that is running as part of the operation of your website, though, you definitely need the indices.

2
  • Thanks for your post. What do you suggest to use instead of that last step with DATE_SUB(); ? Jan 23, 2014 at 12:16
  • I've edited my answer and added a suggestion. Jan 23, 2014 at 23:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.