2

I use Zend Framework in my project. I need to insert multiple records and I found that Zend_Db suprisingly slower thatn my_sql query (several times), that made me think I did something wrong. Here are two examples.

Zend_Db_Adapter:

        $startTime = microtime(true);
        $db = Zend_Db_Table::getDefaultAdapter();
        $db->beginTransaction();

        $dateAdded = date('Y-m-d H:i:s');
        $lastChanged = $dateAdded;                  

        foreach ($importDataNamespace->data as $subscriberNum => $subscriber)
        {
            foreach ($fieldsMap as $fieldNumber => $fieldTag) {
                if (isset($subscriber[$fieldNumber])) {
                    $subscriberData[$fieldTag] = $subscriber[$fieldNumber]; 
                } else {
                    $subscriberData[$fieldTag] = '';
                }
            }
            $query = 'INSERT INTO subscribers (list_id, account_id, email_address, first_name, last_name, date_added, last_changed) ' .
                     'VALUES (' . 52 . ', ' . 29 . ', ' .  $db->quote($subscriberData['EMAIL']) . ', ' . $db->quote($subscriberData['FNAME']) .
                     ', ' . $db->quote($subscriberData['LNAME']) . ', ' . $db->quote($dateAdded) . ', ' . $db->quote($lastChanged) . ')';
            $db->query($query);                                                                 
        }
        $db->commit();

        $this->view->time = microtime(true) - $startTime;

Example with mysql_query:

        $startTime = microtime(true);

        $user = 'root';
        $password = 'password';
        $db = 'database';
        $connect = @mysql_connect('localhost',$user,$password) or die("Failed to connect database");
        @mysql_select_db($db) or die("Failed to select database");          

        $dateAdded = date('Y-m-d H:i:s');
        $lastChanged = $dateAdded;      

        $result = mysql_query('SET autocommit = 0');            

        foreach ($importDataNamespace->data as $subscriberNum => $subscriber)
        {
            foreach ($fieldsMap as $fieldNumber => $fieldTag) {
                if (isset($subscriber[$fieldNumber])) {
                    $subscriberData[$fieldTag] = $subscriber[$fieldNumber]; 
                } else {
                    $subscriberData[$fieldTag] = '';
                }
            }
            $query = 'INSERT INTO subscribers (list_id, account_id, email_address, first_name, last_name, date_added, last_changed) ' .
                'VALUES (' . 52 . ', ' . 29 . ', \'' .  mysql_real_escape_string($subscriberData['EMAIL']) . '\', \'' . mysql_real_escape_string($subscriberData['FNAME']) .
                '\', \'' . mysql_real_escape_string($subscriberData['LNAME']) . '\', \'' . $dateAdded . '\', \'' . $lastChanged . '\')';                    
            mysql_query($query);                                    
        }
        $result = mysql_query('SET autocommit = 1');
        $result = mysql_query('COMMIT;');

        $this->view->time = microtime(true) - $startTime;           

In the first case it took 14.8 seconds, in the second 3.7. Could you tell me why does it happend and what do you do wrong?

If I delete any quote for Zend_Db it took 12 seconds from 14 with quote, but it's still much more slower than with mysql_query:

        $startTime = microtime(true);
        $db = Zend_Db_Table::getDefaultAdapter();
        $db->beginTransaction();

        $dateAdded = date('Y-m-d H:i:s');
        $lastChanged = $dateAdded;                  

        foreach ($importDataNamespace->data as $subscriberNum => $subscriber)
        {
            foreach ($fieldsMap as $fieldNumber => $fieldTag) {
                if (isset($subscriber[$fieldNumber])) {
                    $subscriberData[$fieldTag] = $subscriber[$fieldNumber]; 
                } else {
                    $subscriberData[$fieldTag] = '';
                }
            }
            $query = 'INSERT INTO subscribers (list_id, account_id, email_address, first_name, last_name, date_added, last_changed) ' .
                     'VALUES (' . 52 . ', ' . 29 . ', \'' .  $subscriberData['EMAIL'] . '\', \'' . $subscriberData['FNAME'] .
                     '\', \'' . $subscriberData['LNAME'] . '\', \'' . $dateAdded . '\', \'' . $lastChanged . '\')';
            $db->query($query);                                                         
        }
        $db->commit();

        $this->view->time = microtime(true) - $startTime;

Thank you for any information about this issue.

This code takes about 0.065 seconds with mysql_query:

    $dateAdded = date('Y-m-d H:i:s');
    $lastChanged = $dateAdded;      

    $startTime = microtime(true);       
    $result = mysql_query('BEGIN');
    for ($i = 0; $i < 100; $i++) {
        $email = 'test_ ' . $i . '@gmail.com';
        $query = 'INSERT INTO subscribers (list_id, account_id, email_address, first_name, last_name, date_added, last_changed) ' .
                'VALUES (' . 52 . ', ' . 29 . ', \'' .  mysql_real_escape_string($email) . '\', \'' . mysql_real_escape_string($firstName) .
                '\', \'' . mysql_real_escape_string($lastName) . '\', \'' . mysql_real_escape_string($dateAdded) . '\', \'' . mysql_real_escape_string($lastChanged) . '\')';                                                                   
        mysql_query($query);            
    }               
    $result = mysql_query('COMMIT');
    $time = microtime(true) - $startTime;                           
    echo 'Using mysql_query: ' . $time  . '<br />';
    exit(); 

Code with benchmark of Zend_Db_Adapter (I didn't even use quote in this case):

    $db = Zend_Db_Table::getDefaultAdapter();
    $db->getProfiler()->setEnabled(true);
    $profiler = $db->getProfiler();          

    $startTime = microtime(true);
    $db->beginTransaction();        
    for ($i = 0; $i < 100; $i++)
    {

        $email = 'test_' . $i . '@gmail.com';
        $query = 'INSERT INTO subscribers (list_id, account_id, email_address, first_name, last_name, date_added, last_changed) ' .
             'VALUES (' . 52 . ', ' . 29 . ', \'' . $email . '\', \'' . $firstName .
             '\', \'' . $lastName . '\', \'' . $dateAdded . '\', \'' . $lastChanged . '\')';
        $db->query($query);                                 
    }           
    $db->commit();
    $time = microtime(true) - $startTime;                       
    echo 'Time of transaction Zend_Db_Adapter query: ' . $time . '<br />';

    echo 'Total time ' . $profiler->getTotalElapsedSecs() . '<br />';           
    $count = 0;
    $totalTime = 0; 
    foreach ($profiler->getQueryProfiles() as $query) {
        $count++;
        $elapsedTime = $query->getElapsedSecs();
        $totalTime += $elapsedTime;
        echo $count . ' ' . $elapsedTime  . ' ' . $query->getQuery() . '<br />';
    }
    echo 'Sum time: ' . $totalTime . '<br />'; 

Here are some results:

Time of transaction Zend_Db_Adapter query: 0.23094701767 Total time 0.0949234962463 1 0.00199699401855 connect 2 0.000336885452271 begin 3 0.000540018081665 INSERT INTO subscribers (list_id, account_id, email_address, first_name, last_name, date_added, last_changed) VALUES (52, 29, '[email protected]', 'John', 'Clinton', '2011-01-28 15:25:21', '2011-01-28 15:25:21') 4 0.000504016876221 INSERT INTO subscribers (list_id, account_id, email_address, first_name, last_name, date_added, last_changed) VALUES (52, 29, '[email protected]', 'John', 'Clinton', '2011-01-28 15:25:21', '2011-01-28 15:25:21')

It's very strange. The time of transacation to insert 100 records is 2.5 times more than executing of all queries. If I try to meach the time of forming strings in the loop it (if we delete query) doesn't take so much time.

2
  • If you have a big system with lots of data, you may consider doctrine2 as an ORM Jan 28, 2011 at 8:07
  • Yes, it's going to have rather much data. But the problem with Zend_Db_Adapter is still unclear, what cause it to take so long.
    – Oleg
    Jan 28, 2011 at 12:20

3 Answers 3

3

I think that one reason is that you execute $db->quote() too many times, which is unnecessary. Do you know that $db->quote() can take an array as its argument and you can basically reduce calls to $db->quote() to only one. In addition in your mysql_query version you do not escape $dateAdded and $lastChanged, while in zend_db version you do.

EDIT: Added an example below

    $db = Zend_Db_Table::getDefaultAdapter();

    $input = array(
        'a' => "asd'lfksd",
        'b' => "asdlfk'sdfasdf",
        'c' => "asd fds f saf'sdfsd",
        'd' => "asd fds f saf'sdfsd"
    );


    // separate calls to quote
    $startTime = microtime(true);
    $db->quote($input['a']);
    $db->quote($input['b']);
    $db->quote($input['c']);
    $db->quote($input['d']);
    $totalTime1 = microtime(true) - $startTime;



    // one call to quote
    $startTime = microtime(true);
    $db->quote($input);
    $totalTime2 = microtime(true) - $startTime;

    // show results
    var_dump("Sperate calls are ". $totalTime1/$totalTime2 . " times slower");
    //output: string 'Sperate calls are 3.0875831485588 times slower' (length=46)
18
  • Do you think quote can take some much time?
    – Oleg
    Jan 27, 2011 at 10:50
  • I didn't think that quote can take array as argument. I didn't find it in the documentation. I used many mysqli_real_escape_string in the second case too.
    – Oleg
    Jan 27, 2011 at 10:54
  • @user366534. I added an example code that you can use to check the time difference between single call to quote vs. many calls to quote.
    – Marcin
    Jan 27, 2011 at 11:07
  • I added code whne I delete any quote for zend_db it take now 12 seconds (from 14 seconds) which is much slower than mysql_query (3.8) even when I added quote for date fields too.
    – Oleg
    Jan 27, 2011 at 11:56
  • @user366534. In this case I think you need to do small tests to localize what is so slow. You can, for example, just measure time for only execution of $db->query($query) and mysql_query($query) and see if this is the source of the problem. You can remove transactions and see what happens. Also, are you sure that in both cases your data (i.e. $importDataNamespace->data is identical). If you find something interesting, please let me know.
    – Marcin
    Jan 27, 2011 at 12:15
0

mysql_* functions are natives to PHP , so they are very fast.

Zend_Db_Adapter is working with PDO so you have a first absraction layer with PDO and a second with Zend_DB_Adapter.

More layer abstraction you 've got , more the code is slowing down.

That why the MVC framework in general are more slower than procedural code.

Try again your benchmark with prepared statement and a cache mechanism and you should be close of the mysql_* functions

5
  • Yes, I expected it to be a little slower, but several time slower, it seems too much. I
    – Oleg
    Jan 27, 2011 at 10:05
  • 1
    And it sure highlights the benefits of a good server-side caching strategy to avoid those costly db connections and queries wherever possible. Jan 27, 2011 at 12:34
  • I don't quite understand how caching could help inserting multiple rows.
    – Oleg
    Jan 27, 2011 at 14:49
  • Caching will not improve the database connection itself but the computing time of the abstraction layers will be reduce.
    – grunk
    Jan 31, 2011 at 12:29
  • As far as I can judge it can help mostly with reading not writing.
    – Oleg
    Feb 1, 2011 at 17:37
0

This looks like you don't have a metadata cache in Zend_Db.

3
  • Could you please, explain what you mean there?
    – Oleg
    Sep 9, 2011 at 5:14
  • Could you please explain what you mean about a metadata cache in Zend_Db?
    – Oleg
    Sep 9, 2011 at 5:15
  • You can read more here framework.zend.com/manual/en/…. Hopefully this will help with your problem though it could be a red herring. I make it a habit of always using a metadata cache in Zend_Db to prevent useless queries from being sent to the DB.
    – Julian
    Sep 12, 2011 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.