2

I got this query. It take ~0.0854 seconds to excutes. I find it a little slow. Below see my explain

SELECT
    stops.stop_number,
    stops.stop_name_1,
    stops.stop_name_2
FROM
    tranzit.stops_times

INNER JOIN 
    tranzit.stops
ON
    (
        stops_times.stop_id = stops.stop_id
    )

INNER JOIN 
    tranzit.trips
ON
    (
        stops_times.trip_id = trips.trip_id
    )   
WHERE
    trips.route_id = 109 AND
    trips.trip_direction = 1 AND
    trips.trip_period_start <= "2011-11-24" AND
    trips.trip_period_end >= "2011-11-24"

GROUP BY
    stops.stop_id
ORDER BY
    stops_times.time_sequence ASC
LIMIT 
    0, 200

Explain

id  select_type     table           type            possible_keys                                           key                         key_len     ref                             rows    Extra
1   SIMPLE          trips           index_merge     trip_id,trip_period_start,trip_period_end,trip_dir...   route_id,trip_direction     3,1         NULL                            271     Using intersect(route_id,trip_direction); Using wh...
1   SIMPLE          stops_times     ref             stop_id,trip_id                                         trip_id                     16          tranzit.trips.trip_id           24   
1   SIMPLE          stops           ref             stop_id                                                 stop_id                     3           tranzit.stops_times.stop_id     1       Using where

And I have indexe on trips :

Table   Non_unique  Key_name    Seq_in_index    Column_name     Collation   Cardinality     Sub_part    Packed  Null    Index_type  Comment
trips   1   agency_id   1   agency_id   A   2   NULL    NULL        BTREE    
trips   1   trip_id     1   trip_id     A   9361    NULL    NULL        BTREE    
trips   1   trip_period_start   1   trip_period_start   A   2   NULL    NULL        BTREE    
trips   1   trip_period_end     1   trip_period_end     A   2   NULL    NULL        BTREE    
trips   1   trip_direction  1   trip_direction  A   2   NULL    NULL        BTREE    
trips   1   route_id    1   route_id    A   106     NULL    NULL        BTREE    
trips   1   shape_id    1   shape_id    A   520     NULL    NULL        BTREE    
trips   1   trip_terminus   1   trip_terminus   A   301     NULL    NULL        BTREE    

Indexes on stops

stop_number BTREE   Non Non stop_number 4626    A       
agency_id   BTREE   Non Non agency_id   1   A       
stop_id BTREE   Non Non stop_id 4626    A       

Thanks for any help

7
  • I don't see many places for further improvements. Try profiling your query and see how the 0.0854secs are distributed over the different steps during query execution (p.e. sending data, lockings, ordering etc.)
    – Bjoern
    Nov 21, 2011 at 8:01
  • Any hint about how to do this ? Nov 21, 2011 at 8:08
  • What kind of hardware/os setup is this running on? How many rows in each table? Is it cacheable? 0.08 seconds isn't that bad, maybe you can just deal with it.
    – regality
    Nov 21, 2011 at 8:12
  • Q6600 @ 2.4, Deb64bits, only 2 GB ram but will increase to 8GB in working environnement...Table stops have ~4.5k, stops_times ~414k and trips have ~9.1k ... Rows will increase by 10 maybe in the working environnement... Yes it is cacheable but faster is better, i am looking to optimize, i know this ain't easy. InnoDB have been setup this way : 1024MB allowed for memory cache. Nov 21, 2011 at 8:14
  • @DavidBélanger Here's a link to the MySQL Documentation which shows how to profile a query: dev.mysql.com/doc/refman/5.0/en/show-profiles.html
    – Bjoern
    Nov 21, 2011 at 8:26

5 Answers 5

1

Given how many rows you have in the tables it is already running pretty quick. You could try a few different approaches such as added more where conditions or performing a simple select and then running a second query to get the needed join fields. But these aren't where you really need to focus.

The important question is how will this query behave in the wild. If you are running it 100 times every second you need to know if it is going to degrade and become a bottleneck. If it can run in 0.08 every time, then that still allows for a very responsive application.

The most important strategy however, if it is possible and came be made effective, is using memcache or a similar option to prevent running the query all the time.

1
  • Yes it may be running 100 times every second in the near future. 40-50 is more suggestive. I will give it a try using a second query. I may then cache the data for each query using specific folder like a folder named after stop_id or even a trip_id ... since trip_id is bin(16) with md5, it will be unique Nov 21, 2011 at 8:51
1

As people wrote before:

Split to 2 queries:

  1. Trip information, by group_concat to make it faster
 SELECT group_concat(trip_id) FROM trips WHERE 
     trips.route_id = 109 AND 
     trips.trip_direction = 1 AND 
     trips.trip_period_start = "2011-11-24" 
  1. Next Information
 SELECT 
    stops.stop_number, 
    stops.stop_name_1, 
    stops.stop_name_2 
 FROM 
    tranzit.stops_times,
    tranzit.stops 

 WHERE 
    stops_times.stop_id = stops.stop_id
 AND
    stops_times.trip_id in ( ...)

GROUP BY, ...

I think it will be faster, as you don't need other information from trips table outside the query.

2
  • group_concat slow the thing a lot... runnign from 2 to 25 seconds ... by the way trip_id is bin(16) who hold md5 hash Nov 21, 2011 at 18:39
  • See my answer below, your answer is part of it. Thanks! Nov 21, 2011 at 18:48
0

the most tricky part is on the range query trip_period_start, trip_period_end,
I think you can consider a composite key like:-

alter table trips
add index testing
(
  route_id, trip_direction, trip_period_start, trip_period_end
);

depend on how many unique value for trip_direction,
if always only a few unique values,

alter table trips
add index testing
(
  route_id, trip_period_start, trip_period_end, trip_direction
);
0

Already less than 1 tenth of a second and you want it faster? ok... I would build a composite index on ( route_id, trip_direction, trip_period_start ) as those are the three critical elements of your query. Also, in that order to have the smallest granularity to the front of the index (specific route). Then, within that, its direction, then, the dates. Next, I would swap the order of the query with the trips table up front since you are doing INNER joins. Additionally, have an index on your "stops_times" table on TRIP_ID. By starting with the first table with its qualifiers, then joining to the child-level tabls via relations, you still get the elements, but you are running against the smallest index set first on trips.

select STRAIGHT_JOIN
      stops.stop_number,
      stops.stop_name_1,
      stops.stop_name_2
   from
      tranzit.trips
         join tranzit.stops_times
            on trips.trip_id = stops_times.trip_id
            join tranzit.stops
               on stops_times.stop_id = stops.stop_id
   where 
          trips.route_id = 109 
      AND trips.trip_direction = 1 
      AND trips.trip_period_start <= "2011-11-24" 
      AND trips.trip_period_end >= "2011-11-24"
   group by
      stops.stop_id
   ORDER BY
      stops_times.time_sequence
   LIMIT 
      0, 200
1
  • Gave me between 2 and 25 seconds.. way too slow. Thanks for trying ! Nov 21, 2011 at 18:55
0

I found something that work like a charm. My results number are :

  • 0.0011
  • 0.0008
  • 0.0017 (highest)
  • 0.0006 (lowest)
  • 0.0013

These result aren't from the cache. I switch all the WHERE in t (trips.agency_id, trips.route_id, trips.trip_direction, trips.trip_period_start, trips.trip_period_end) and it is working very good ! I can't explain why but if someone can, i'd like to see why. Thanks a lot everyone !

PS : Even without trips.agency_id it is working great.

SELECT
    stops.stop_number, 
    stops.stop_name_1, 
    stops.stop_name_2
FROM 
    tranzit.stops_times,
    tranzit.stops,
    (
        SELECT 
            trips.trip_id
        FROM
            tranzit.trips
        WHERE
            trips.agency_id = 5 AND
            trips.route_id = 109 AND
            trips.trip_direction = 0 AND
            trips.trip_period_start <= "2011-12-01" AND
            trips.trip_period_end >= "2011-12-01"
        LIMIT 1
    ) as t

WHERE 
    stops_times.stop_id = stops.stop_id AND
    stops_times.trip_id in (t.trip_id)
GROUP BY
    stops_times.stop_id
ORDER BY
    stops_times.time_sequence ASC
LIMIT 
    0, 200


id  select_type     table           type        possible_keys       key         key_len     ref                             rows    Extra
1   PRIMARY         <derived2>      system      NULL                NULL        NULL        NULL                            1       Using temporary; Using filesort
1   PRIMARY         stops_times     ref         trip_id,stop_id     trip_id     16          const                           33      Using where
1   PRIMARY         stops           ref         stop_id             stop_id     3           tranzit.stops_times.stop_id     1       Using where
2   DERIVED         trips           ref         testing             testing     4                                           275     Using where

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.