25

Is there any program IDE or not that can format MySQL code inside PHP string e.g. I use PHPStorm IDE and it cannot do it.

It does that for PHP and MYSQL but not for MYSQL inside php string. I am ready to use new IDE because now i have to manually format hundreds of database requests that are one line and not readable. Only criteria for my choice is that ide can do that automatically.

<?php
...
$request1 = "select * from tbl_admin where admin_id= {$_SESSION['admin_id']} and active= 1 order By admin_id Asc";
...
?>

should become

<?php
...
$request1 = "SELECT * 
               FROM tbl_admin 
                  WHERE admin_id = {$_SESSION['admin_id']}
                  AND active = 1
                      ORDER BY admin_id ASC";
...
?>
11
  • You mean you want the software to indent the code like your second example? Jun 12, 2012 at 21:39
  • 1
    I believe that is the question. Jun 12, 2012 at 21:40
  • 1
    Yes i do want software to automatically reformat the code.
    – JohnA
    Jun 12, 2012 at 21:41
  • 1
    because in php your sql query is in string format so no ID will think that it is an sql query or normal query so its not possible because ist a string for that IDE Jun 20, 2012 at 10:55
  • 9
    BTW: You should urgently prevent SQL injection with prepared statements or with mysql_real_string_escape().
    – glglgl
    Jun 22, 2012 at 22:49

17 Answers 17

17
+150

The best way to do this in my opinion is to use Regular Expressions or SED/AWK to format everything, it gives us the bonus of replacement maps on the fly. The chance that you might have code errors though is high, so it's kind of tough.

let me work on it a bit and I can see if I can come up with a good solution. Is it guaranteed that you are encapsulating all SQL in double quotes?

EDIT

Try this

cd {{directory}} && find . -type f -print0 |
  xargs -0 perl -i.bak -pe 's/select/SELECT/g ; s/from/\n\tFROM/g ; s/where/\n\t\tWHERE/g ; s/and/\n\t\tAND/g ; s/order by/\n\t\tORDER BY/g ; s/asc/ASC/g ; s/desc/DESC/g ;'

Here's an example

$ printf '"select * from whatever where this = that and active = 1 order by something asc";\n' |
> perl -pe 's/select/SELECT/g ; s/from/\n\tFROM/g ; s/where/\n\t\tWHERE/g ; s/and/\n\t\tAND/g ; s/order by/\n\t\tORDER BY/g ; s/asc/ASC/g ; s/desc/DESC/g ;'

"SELECT * 
    FROM whatever 
        WHERE this = that 
        AND active = 1 
        ORDER BY something ASC";

Is it pretty? no, not at all, does it work.... Yeup.

I'll try creating a filter file and maybe a little bash program or something as i get time to run this hot mess.

EDIT

Here's some revised code, looks prettier (sorta)

printf '$request1 = "select * from whatever where this = that and active = 1 order by something asc";\n' | 
perl -pe 's/select/SELECT/gi ; s/from/\n  FROM/gi ; s/where/\n    WHERE/gi ; s/and/\n    AND/gi ; s/order by/\n      ORDER BY/gi ; s/asc/ASC/gi ; s/desc/DESC/gi ;' | 
awk 'NR == 1 {pad = length($0)/2; print} NR > 1 {gsub(/\r/,""); printf "%*s%s\n", pad, " ", $0}'

__OUTPUTS__
$request1 = "SELECT * 
             FROM whatever 
               WHERE this = that 
               AND active = 1 
                 ORDER BY something ASC";
3
  • 2
    it's one step close to asking the OP to write his own Netbeans plugin. Jun 26, 2012 at 3:45
  • 1
    Nope, if we were to accept the metaphor that UNIX/Linux operating systems are basically one huge IDE, then this would actually be a plugin (just not a Netbeans plugin). Jun 26, 2012 at 9:34
  • I agree with Mihai, that this is pretty much a standalone that can parse anything its pointed at. *nix systems are pretty much there own IDE's
    – ehime
    Jun 26, 2012 at 16:48
13

As fas as I know PhpStorm can do it, if you use the heredoc syntax

$request1 = <<<SQL
   SELECT * 
           FROM tbl_admin 
              WHERE admin_id = {$_SESSION['admin_id']}
              AND active = 1
                  ORDER BY admin_id ASC
SQL;
8
  • 3
    That would mean i have to change hundreds of requests still... its very big project i got from other developer and i cannot manually change every request. P.S. I dont want to use heredoc syntax.
    – JohnA
    Jun 12, 2012 at 21:44
  • 1
    If your queries don't all adhere to a certain syntax it's would be impossible to know if a string is SQL or not. Imagine the case where an sql string is broken over multiple lines with concatenation 'SELECT'.' * '.'FROM'. With no strict standard everywhere guaranteeing that parameters are inserted the same way you're not going anywhere. I know heredoc is ugly but it gets the job done. Jun 24, 2012 at 0:45
  • @MihaiStancu: I don't think that's entirely true. For example, an engine could evaluate each string statement then attempt to parse for SQL; if the string contains SQL, it could backtrack to the underlying literals and format appropriately. However, I very much doubt there's anything out there that will do this...?
    – eggyal
    Jun 26, 2012 at 4:27
  • @eggyal In fact PhpStorm does (something like) this. On the one hand it parses strings and compare it against known languages. This (of course) doesn't work very well for constructed strings, but works fine for example with substitution ("SELECT * FROM $table WHERE id = :id;"). On the other hand when you use the heredoc syntax you can give it a hint with the heredoc identifier (see <<<SQL above). This works quite good for continuos strings.
    – KingCrunch
    Jun 26, 2012 at 6:28
  • @eggyal PhpStorm allows you to do this provided that you use heredoc and a fixed delimiter <<<SQL (as @KingCrunch mentioned) but my point was that no editor can do this regardless of syntax. If you grab the final query you sent to the database (at runtime) and format that then look for the original string generator and find something like SELECT <? foreach($fields AS $field): echo $field; ?> you'll have a thrilling time formatting it. Jun 26, 2012 at 9:12
5

The answer is NO.

As an alternative solution, I can suggest copying the text (ALL THE PHP CODE in the *.php file) into the query editor of Heidisql, clicking Re-Format and copying back to NetBeans and pressing Alt+Shift+F. This will format sql and format php code after, preserving sql formatting. SO instead of a plug in, you're copying, formatting a whole file, copying back and formatting again.

Instead of HeidiSQL, you can use this tool, which is focused on formatting if you are comfortable with web tools like these.

If you want more automation, you can use regular expression support of netbeans to format with backslash n = \n for new line befre each "insert, update delete, select, where and values" terms running this 6 times will replace all " insert" <- blank plus insert, in your projects with new lines. you can write a regular expression for searching all, "blank plus reserved word", combinations and replace them with new lines.

You can also use it like search for " insert" and repalce with "\n\t insert", which would create a new line an a tab

enter image description here

4

Considering the question: Is there an IDE that will automatically format sql queries, I must concur with ugurcode. The answer is simple: there isn't any. And with good reason.

Basically an SQL query is nothing but a string in any IDE. Since strings can contain intentional whitespaces and tabs, it would be very bad if an IDE would reformat string content.

The only option that would work for any IDE would be to detect INSERT, UPDATE, DELETE at the start of a string and then format the rest of the string.. ofcourse it would also try to do the same with a string saying "insert quote here..", which once again would be a bad thing.

Since i don't want to leave you with just nothing i will give you advice on a good IDE to choose which does in fact do proper code formatting, code completion, good indentation, plugins etc.. and basically is the best IDE for php development (atleast its the best i've found in my years of professional php development).

Netbeans would be your IDE of choice. It's faster then i.e. Eclipse/PHP, more stable then i.e. codelobster etc.

4
  1. Log, profile your SQL statements.
  2. Use an external service such as this one - SQLFormat API - in order to format your SQL statements
  3. You can also run it on your machine dowloading the Python sources.
4

Since your problem is to format an existing body of code, you don't need an IDE. You need a script to batch-process all your files once, and then you can forget about it. This is what the script needs to do:

  1. Correctly parse PHP strings, including strings with all sorts of embedded or escaped quotes. This should be bullet-proof, and should only care about PHP syntax.
  2. Check each PHP string with an algorithm that detects SQL commands. This can be made as smart as you need it to be (no need to blindly accept every string containing the word "insert", for example).
  3. Feed the identified strings through an SQL pretty-printer.
  4. Edit the input file, substituting the formatted string for the original one.

Ideally, parts 1 and 3 should be handled by off-the-shelf modules. The rest should be easy to put together yourself, right?

Update: Explaining this made it sound so simple, I decided to do it myself. Here's a quick solution. It's in python, but if you were willing to switch IDEs you can deal with installing python, right?

Drag-and-drop any number of php files onto the script, or call it from the commandline, and it will filter the SQL bits through the SQLFormat API that @Parahat suggested. It edits the files in place, so keep a copy!

"""Format sql strings embedded in php code, editing source IN PLACE"""
import sys, re, urllib, urllib2

def processfile(fname):
    with open(fname) as fp:
        text = fp.read()

    with open(fname, "w") as out:
        for token in chunk(text):
            if is_sql_string(token):
                token = token[0] + sqlformat(token[1:-1]) + token[0]
            out.write(token)

def sqlformat(query):
    sqlapi = 'http://sqlformat.appspot.com/format/?keyword_case=upper&reindent=1&n_indents=4&'
    return urllib2.urlopen(sqlapi+urllib.urlencode({'data':query})).read()

php_mode = False # global, so that is_sql_string() can examine it
def chunk(src):
    """Chunk the php file into interesting units"""
    global php_mode
    while src:
        if not php_mode: # Read up to the next php group, if any
            m = re.match(r".*?<\?php", src, re.S)
            if m:
                tok, src = _usematch(m, src)
                yield tok
                php_mode = True
            else: # No more php groups: EOF
                yield src
                return

        else:  # Reading php code
            # PHP ends without another string?
            m = re.match(r"[^'\"]*?\?>", src, re.S)
            if m:
                tok, src = _usematch(m, src)
                yield tok
                php_mode = False
                continue

            # at non-string material?
            m = re.match(r"[^'\"]+", src) 
            if m:
                tok, src = _usematch(m, src)
                yield tok
                continue

            # Parse a string: Smallest quote-delimited sequence,
            # where closing quote is not preceded by backslash
            m = re.match(r'".*?(?<!\\)"|' + r"'.*?(?<!\\)'", src, re.S)
            if m:
                tok, src = _usematch(m, src)
                yield tok
                continue

            # Something I can't parse: advance one char and hope for the best
            tok, src = src[0], src[1:]
            yield tok

def _usematch(m, inp):
    return m.group(), inp[m.end():] # Matched chunk & remaining input

# To recognize an sql command, it MUST begin with one of these words
sql_commands = set("select insert update delete create drop alter declare".split())
def is_sql_string(tok):
    if not php_mode or len(tok) < 3 or tok[0] not in set("\"'"):
        return False    
    tokwords = tok[1:-1].split()
    return tokwords and tokwords[0].lower() in sql_commands

for fname in sys.argv[1:]:
    processfile(fname)
4

In PHPStorm you can specify SQL dialect for your project: http://www.jetbrains.com/phpstorm/webhelp/sql-dialects.html

Next, you must setup Data Source: http://www.jetbrains.com/phpstorm/webhelp/data-sources-2.html

After that your IDE will support SQL syntax and autocomplete for tables (fields).


Also, there is a page, where you can specify SQL code-style: http://www.jetbrains.com/phpstorm/webhelp/code-style-sql.html

2

There is no such IDE. Maybe some SQL is broken into multiple string concatenation, or maybe some SQL is generated using some iterator. Or maybe it contains invalid SQL such as {$_SESSION['admin_id']}.

Your only chance is writing a small script, tuned to match the coding style and possible invalid SQL stuff, which will repair all application source code files. It will take you hours to get it right, but in the end you won't need an IDE which doesn't exist, and you will have a prettyfiend stonified SQL and better source code.

(The above is a solution, thinking you have hundreds of SQL statement throughout the app; if not, just repair them by hand).

Edit: make sure you log all changes in a comparison table, so you can review all which was formatted by your script.

2
  • I doubt it will beautify some scrambled SQL. And its expensive. For 100€ I'll write that script to fix the entire project.
    – oxygen
    Jun 21, 2012 at 19:57
  • I only posted that link because you said in your answer, "There is no such IDE." I'd be willing to write a script for that price too. ;)
    – Robert K
    Jun 21, 2012 at 20:37
2

may be nuSphere debugger somehow help you for doing this here is the link nuSphere

2

Write your queries in MySQL Workebench then edit->format->beautify query

Then paste the query into netbeans, select the test and tab in.

2

I always used SQLyog to format my SQL statements and paste them back to the IDE, because i didn't find an IDE either, which is able to format sql inside php blocks.

1

I built a tool to do this for MS SQL Server and C# code. I'd be happy to alter the tool for this purpose for a small fee. http://www.softfrontiers.com/Downloads/ReIndenter.shtml

An hour or two of my time will save you many hours of frustration.

1

You can use the tokenizer extension to write a script that would format the SQL.

The following script can be invoked from the command-line, it reads from standard input and writes to standard output. It handles both quoting styles (double or single quotes).

All strings are scanned for possible SQL grammar, based on the starting word, and line breaks are inserted before every keyword; both starting word and keywords are extensible, so go wild :)

<?php

$tokens = token_get_all(file_get_contents('php://stdin'));

function token_value($token)
{
        return is_array($token) ? $token[1] : $token;
}

function sql_format($s)
{
        if (preg_match("/^(?:select|insert|update|delete)/i", $s)) {
                // prefix newline and tab before every keyword
                return preg_replace('/\b(from|where|and|order|group by)/i', "\n\t\\0", $s);
        }
        return $s;
}

$target = '';
$i = 0; $n = count($tokens);
while ($i < $n) {
        $token = $tokens[$i];
        if ($token === '"') {
                $s = ''; ++$i;
                while ($i < $n && $tokens[$i] !== '"') {
                        $s .= token_value($tokens[$i]);
                        ++$i;
                }
                if ($i < $n) {
                        ++$i;
                }
                $target .= '"' . sql_format($s) . '"';
        } elseif (is_array($token) && $token[0] === T_CONSTANT_ENCAPSED_STRING) {
                $quote_style = $token[1][0];
                $target .= $quote_style . sql_format(trim($token[1], "'\"")) . $quote_style;
                ++$i;
        } else {
                $target .= token_value($token);
                ++$i;
        }
}

echo $target;
1

You asked:

Is there any program IDE or not that can format MySQL code inside PHP string

with specifically this PHP string definition:

$request1 = "select * from tbl_admin where admin_id= {$_SESSION['admin_id']} and active= 1 order By admin_id Asc";

And the answer is no. There doesn't exist a well use-able settop on a PHP parser in IDEs or on the commandline that works PHP's token_get_all.

With token_get_all you should be able to first of all extract that part that makes sense in your case to be edited. Here with context:

<309:T_VARIABLE> "$request1" <371:T_WHITESPACE> " " "=" <371:T_WHITESPACE> " " """ <314:T_ENCAPSED_AND_WHITESPACE> "select * from tbl_admin where admin_id= " <375:T_CURLY_OPEN> "{" <309:T_VARIABLE> "$_SESSION" "[" <315:T_CONSTANT_ENCAPSED_STRING> "'admin_id'" "]" "}" <314:T_ENCAPSED_AND_WHITESPACE> " and active= 1 order By admin_id Asc" """ ";"

As these tokens show, there is a lot of additional work going to extract that what you call string from it:

$request1 = "select * from tbl_admin where admin_id= {$_SESSION['admin_id']} and active= 1 order By admin_id Asc"

This need to be managed and identified in tokens:

<314:T_ENCAPSED_AND_WHITESPACE> "select * from tbl_admin where admin_id= " <375:T_CURLY_OPEN> "{" <309:T_VARIABLE> "$_SESSION" "[" <315:T_CONSTANT_ENCAPSED_STRING> "'admin_id'" "]" "}" <314:T_ENCAPSED_AND_WHITESPACE> " and active= 1 order By admin_id Asc"

As this example shows, this is not true SQL. So you not only need to find a SQL parser (which exists in PHP, see Parsing sql query PHP), but you also need to make that SQL parser aware of PHP variable substitution like {$_SESSION['admin_id']} in your case. For you it is a variable, for an SQL parser this is just a freaky syntax soup if not an error.

Your small code example is a good one: It already shows that there is no information about the nature of the string that will be substituted. As this information is hidden, a SQL parser will never be able to deal with this string as something well-formed.

So to make my answer "No" more profound: Because no general tools on how to deal with multiple outcome on the higher languages level (the PHP string has multiple probably correct representations in SQL, only the author of the variable content will can tell which one is correct) that already exists, there is no general solution available.

You might want to just fire up some regular expressions on your codebase and blitzshit together an outcome you can feel happy with, and indeed the tools ehime lined-up are worth to consider.

But probably it's worth to invest the time and first extract the strings from the PHP token stream, reformat the string and then proceed outputting the result. That's something perl and regexes can't give you because they don't have the PHP tokenizer.

0

If there is no support in all the many IDEs for such a thing, there is a reason in most cases. Please consider prepared statements oder object wrappers for your work. I know both things can't be implemented in every system within a few minutes, but your code will become maintainable.

In short: IDEs cant correct design errors. They just make them look nice and colored.

0

I have used PHPDesigner. i think this will be fulfill what you expecting.

0

Late to the party, but...

function formatSQL($txt){
    $txt = str_replace('SELECT ', "\nSELECT", $txt);
    $txt = str_replace(' MATCH', "\nMATCH", $txt);
    $txt = str_replace(' AGAINST', "\nAGAINST", $txt);
    $txt = str_replace(' OR ', "\nOR", $txt);
    $txt = str_replace(' AND ', "\n\tAND ", $txt);
    $txt = str_replace(' FROM ', "\nFROM", $txt);
    $txt = str_replace(' WHERE ', "\nWHERE", $txt);
    $txt = str_replace(' ORDER ', "\nORDER", $txt);
    return $txt;
}
1
  • While this code may answer the question, providing additional context regarding why and/or how this code answers the question improves its long-term value. Nov 1, 2021 at 9:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.