23

Need help with MySQL as it's not really my forte. So any help is appreciated.

I have issues on my site where UPDATE or INSERT were done with missing values. This caused some issues on other functions on the site, but I am not able to find where the UPDATE or INSERT were done in any of the classes.

Is there any way, maybe a MySQL trigger, that I could add to these tables that would allow me to store the original or full query of the UPDATE or INSERT. I have tried logging but that applies to the whole database and it takes up too much diskspace.

Thanks in advance for any replies.

PS: At the moment, the PHP classes are a bit messy as we're still in the development stage, so adding exceptions to the updates or inserts functions will take too much time. So please focus the answer to the question. Thanks again.

5
  • 1
    unless we see some code and error I am sure we will not be able to provide much of help. So, please post your code / error
    – Satya
    May 17, 2012 at 0:58
  • Satya @ The error is not related to MySQL but more on our core PHP classes. So its not relevant. I am just looking for ways to log UPDATE or INSERT SQL statement on 2 tables in the database. Thanks.
    – asyadiqin
    May 18, 2012 at 7:47
  • 1
    What do you use on the code side? PHP? If yes, I'll provide you with a quick, dirty and fully working solution.
    – oxygen
    May 21, 2012 at 14:53
  • Tiberiu-Ionut Stan @ Yes, Using PHP with CodeIgniter.
    – asyadiqin
    May 22, 2012 at 23:32
  • I have posted one good solution here: stackoverflow.com/a/26604340/1328818
    – John F
    Oct 28, 2014 at 9:07

8 Answers 8

45
+50

You can get the current SQL query as a string with the following statement:

SELECT info FROM INFORMATION_SCHEMA.PROCESSLIST WHERE id = CONNECTION_ID()

So what you have to do is to create a TRIGGER which runs on insert and/or update operations on your table which should (i) get the current sql statement and (ii) insert it into another table, like so:

DELIMITER |

CREATE TRIGGER log_queries_insert BEFORE INSERT ON `your_table`
FOR EACH ROW
BEGIN
    DECLARE original_query VARCHAR(1024);
    SET original_query = (SELECT info FROM INFORMATION_SCHEMA.PROCESSLIST WHERE id = CONNECTION_ID());
    INSERT INTO `app_sql_debug_log`(`query`) VALUES (original_query);
END;
|
DELIMITER ;

You will have to create two triggers - one for updates and one for inserts. The trigger inserts the new query as a string in the app_sql_debug_log table in the query column.

10
  • Thanks for that. Being a MySQL novice user, are the two triggers you wrote are meant for the two tables, eg. TableA and TableB. So i ahev to add the triggers for each table? Sorry if this question sounds silly but I never actually do much work on MySQL and triggers are totally new to me. Hope to hear from you.
    – asyadiqin
    May 20, 2012 at 11:16
  • I am trying very hard to understand the table PROCESSLIST. At this point of time, I have not done anything on the tables. What I did was to simply INSERT a new record, as well as UPDATE on the 2 tables, but when I query the table PROCESSLIST, there is nothing in that table to show the query that I just did. Am I missing anything here? Thanks again.
    – asyadiqin
    May 20, 2012 at 11:19
  • Triggers are triggered by only one event - SELECT OR DELETE for example. In your case you want triggers for the INSERT AND UPDATE operations so you have to have two triggers for each table, the one is triggered on INSERT and the other one on UPDATE statements. NOTICE: "BEFORE INSERT ON your_table" Change the INSERT keyword to UPDATE for the second trigger. Good Luck! May 20, 2012 at 11:22
  • The PROCESSLIST tables shows the currently working queries. While the Trigger is working your query is still there, that's it can be accessed. May 20, 2012 at 11:23
  • 4
    I always get the SELECT info FROM... as last query.. any idea why? Jul 16, 2015 at 9:31
4

I think you need to check General Query Log of your db server.

The server ... ... logs each SQL statement received from clients. ... ... Since MySQL 5.1.6 log can be a file or a table.

4
  • Ravinder @ I think it logs ALL SQL statements on the database and you have the options to either log it in a file otr a table. As I mentioned, I have tried this but this consume a lot of diskspace as it logs everything. I am looking at just getting the SQL statements if an INSERT or UPDATE is done on 2 tables rather than the whole database. If I am wrong, can you please provide me an example on how to get the General Query Log to log the SQL statements when changes are made on 2 specific tables, eg. TableA and TableB
    – asyadiqin
    May 17, 2012 at 10:11
  • 3
    This could work. Set the general query log to save into a log_table and add an INSERT trigger that allows only queries that are LIKE '%INSERT %' and LIKE '%tableA%' to be actually inserted into the log_table (so you save space) May 19, 2012 at 8:29
  • 2
    @ypercube - Good idea! But not sure what the overheads are. May 19, 2012 at 9:08
  • This works great if there are no stored procedures involved. We are looking for the way to get the logs for even inside SPs to see what exact queries caused the data change.
    – psuhas
    Mar 1, 2017 at 17:44
1

You don't exactly have to log it into a database.

Use file_put_contents with FILE_APPEND and store it in a text file using the tab delimiters or as a csv. That way you can easily import to a database when you need to or view the file in Excel or Numbers (in Mac) whenever you need them.

You can log the text file by dates, ie sql_queries_2012-05-24, and just store it in a folder. Text files shouldn't take as more space than storing it in a database. A simple 'if else' statement, with php date functions, should get the logging sorted by dates.

Using text files is certainly more cost-effective in resources than storing them in a database. Moreover, using php fgetcsv to index and INSERT from text files is much faster than using mysql UPDATE.

1

If you don't want some value missing on the database, can you use constraint to enforce that?

0

Simple PHP-based Solution

Use override_function in PHP to override mysql_query with your own function that you can use to profile and/or log any of the queries in question.

rename_function('mysql_query', 'mysql_query_orig');
override_function('mysql_query', '$query', 'return override_mysql_query($query);');

function override_mysql_query($query)
{
    $result = mysql_query_orig($query);

    if (stripos($query, "mytablename") !== FALSE) {
        // log, echo, etc.
    }

    return $result;
}
0

If you neither can modify the application to log what you send to MySQL (normally, solution #1), nor you can enable general query log (solution #2, but yes, not for production) - you may want to use the MySQL Proxy. See this: https://github.com/mysql/mysql-proxy

MySQL Proxy is lightweight program which you can put between your client (PHP scripts) and the MySQL server. You can run MySQL Proxy on the same server you run the MySQL server itself, and make your clients connect to the proxy port instead of MySQL server directly (if you cannot modify client, you can move MySQL server to another port, and setup MySQL Proxy to the port previously used by MySQL Server).

Then, MySQL proxy can be extended with custom scripts, where your custom script may catch various types of events: new connections, queries sent to server, etc. For queries you have the flexibility to log them, or block them, or modify the query, or add new queries in addition to those actually sent, whatever.

Bad news - custom scripts are implemented in Lua ( http://dev.mysql.com/doc/refman/5.5/en/mysql-proxy-scripting.html). Good news - you can find lots of sample Lua scripts for MySQL proxy. For example here: http://forge.mysql.com/wiki/Lua_Scripts_For_MySQL_Proxy_Examples. In particular you may want to check "Blocking unwanted queries" example and modify it to your needs (you won't actually block anything, but will print certain queries to the log).

MySQL Proxy adds some overhead, but generally it is pretty lightweight. If you keep your Lua scripts light as well - it should work all right.

EDIT:

The repository was last updated on 2014, and currently, the version archives say this:

MySQL Proxy is not GA, and is not recommended for Production use.

We recommend MySQL Router for production use. Download MySQL Router »

MySQL Router doesn't look like it allows the same kind of functionality as MySQL Proxy sadly.

-1

If your server has PHP 5.3.0 or above installed then this might be helpful http://www.php.net/manual/en/mysqlnd.plugin.php

-2

You probably want to turn on binary logging so you can look at every update (note: not every query). You just need to add the --log-bin option, then look through the resulting log using mysqlbinlog. If necessary, you can start a copy of your database from a backup, start in the log at the position of the backup (saved in the backup using something like the --master-data option to mysqldump), and run the updates one at a time until you get a bad row. You then know which update is the culprit. You can script that last part and/or use binary search on the length of the log to make it go faster.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.