I've got a problem with a slow MySQL query (MySQL 5+). Let's think of three tables:
customers:
- id_customer : int (PRIMARY)
- name : varchar(255)
customers_addresses:
- id_customers_addresses : int (PRIMARY)
- id_customer : int (INDEX)
- street : varchar(255)
- zipcode : varchar(255)
- city : varchar(255)
customers_contacts:
- id_customers_contacts : int (PRIMARY)
- id_customer : int (INDEX)
- type : varchar(255)
- value : varchar(255)
Now, my goal is to gather all addresses and contact information in one query and with one row per customer. My first try was by using LEFT JOIN
s as some customers do not have any address and/or contact information:
SELECT customers.id_customer,
customers.name,
X.contact AS contact,
Y.street,
Y.zipcode,
Y.city
FROM customers
LEFT JOIN
(
SELECT
GROUP_CONCAT( CONCAT( type, ': ', value ) SEPARATOR ', ' ) AS contact,
id_customer
FROM customers_contacts
GROUP BY id_customer
) AS X
ON X.id_customer = customers.id_customer
LEFT JOIN
(
SELECT
GROUP_CONCAT(street SEPARATOR '<br>' ) AS street,
GROUP_CONCAT(zipcode SEPARATOR '<br>' ) AS zipcode,
GROUP_CONCAT(city SEPARATOR '<br>' ) AS city,
id_customer
FROM customers_addresses
GROUP BY id_customer
) AS Y
ON Y.id_customer = customers.id_customer
WHERE Y.street LIKE '%Avenue%'
ORDER BY customers.name DESC
LIMIT 0, 20
This query took over 130 seconds to complete (for ~7000 entries in each table), which is far from good.
Prepending EXPLAIN EXTENDED
gives:
id select_type table type possible_keys key key_len ref rows filtered Extra
1 PRIMARY customers ref name name 3 const 4334 100.00 Using where; Using temporary; Using filesort
1 PRIMARY <derived2> ALL NULL NULL NULL NULL 7793 100.00
1 PRIMARY <derived3> ALL NULL NULL NULL NULL 8580 100.00 Using where
3 DERIVED customers_addresses index NULL id_customer 5 NULL 8651 100.00
2 DERIVED customers_contacts index NULL id_customer 4 NULL 9314 100.00
I read some stackoverflow posts and the MySQL Documentation. Both saying INNER JOIN
is a lot faster. I tried to replicate LEFT JOIN
behaviour with INNER JOIN
s by using UNION ALL
:
SELECT customers.id_customer,
customers.name,
X.contact AS contact,
Y.street,
Y.zipcode,
Y.city
FROM customers
INNER JOIN
(
SELECT
GROUP_CONCAT( CONCAT( type, ': ', value ) SEPARATOR ', ' ) AS contact,
id_customer
FROM customers_contacts
GROUP BY id_customer
UNION ALL
SELECT
'' AS contact,
id_customer
FROM customers
WHERE id_customer NOT IN (SELECT DISTINCT id_customer FROM customers_contacts)
) AS X
ON X.id_customer = customers.id_customer
INNER JOIN
(
SELECT
GROUP_CONCAT(street SEPARATOR '<br>' ) AS street,
GROUP_CONCAT(zipcode SEPARATOR '<br>' ) AS zipcode,
GROUP_CONCAT(city SEPARATOR '<br>' ) AS city,
id_customer
FROM customers_addresses
GROUP BY id_customer
UNION ALL
SELECT
'' AS street,
'' AS zipcode,
'' AS city,
id_customer
FROM customers
WHERE id_customer NOT IN (SELECT DISTINCT id_customer FROM customers_addresses)
) AS Y
ON Y.id_customer = customers.id_customer
WHERE Y.street LIKE '%Avenue%'
ORDER BY customers.name DESC
LIMIT 0, 20
This query improved the performance by 20 seconds. But 110 seconds is still not acceptable.
Prepending EXPLAIN EXTENDED
:
id select_type table type possible_keys key key_len ref rows filtered Extra
1 PRIMARY <derived2> ALL NULL NULL NULL NULL 8596 100.00 Using temporary; Using filesort
1 PRIMARY <derived5> ALL NULL NULL NULL NULL 8604 100.00 Using join buffer
1 PRIMARY customers eq_ref PRIMARY,name,name3 PRIMARY 4 Y.id_kunde 1 100.00 Using where
5 DERIVED customers_addresses index NULL id_kunde 5 NULL 8651 100.00
6 UNION customers index NULL name2 767 NULL 8677 100.00 Using where; Using index
7 DEPENDENT SUBQUERY customers_addresses index_subquery id_kunde id_kunde 5 func 2 100.00 Using index
NULL UNION RESULT <union5,6> ALL NULL NULL NULL NULL NULL NULL
2 DERIVED customers_contacts index NULL id_kunde 4 NULL 10411 100.00
3 UNION customers index NULL name2 767 NULL 8677 100.00 Using where; Using index
4 DEPENDENT SUBQUERY customers_contacts index_subquery id_kunde id_kunde 4 func 1 100.00 Using index
NULL UNION RESULT <union2,3> ALL NULL NULL NULL NULL NULL NULL
So here's my question: How to improve one of these queries and/or database tables to get a super fast response? I'm not only interested in the solution but also in strategies how to prevent such performance kills in future.
Best regards.
WHERE
part while tidying up the query for stack overflow. should be correct now (tested)LEFT JOIN
oncustomers_addresses
, when it has to have a streetLIKE '%Avenue%'
?