135

I'm new to regular expressions and would appreciate your help. I'm trying to put together an expression that will split the example string using all spaces that are not surrounded by single or double quotes. My last attempt looks like this: (?!") and isn't quite working. It's splitting on the space before the quote.

Example input:

This is a string that "will be" highlighted when your 'regular expression' matches something.

Desired output:

This
is
a
string
that
will be
highlighted
when
your
regular expression
matches
something.

Note that "will be" and 'regular expression' retain the space between the words.

2
  • Are you actually using the "split" method, or would looping with the "find" method on Matcher be sufficient?
    – erickson
    Dec 14, 2008 at 5:31
  • 13
    "and now he has two problems"
    – user3850
    Dec 14, 2008 at 5:44

16 Answers 16

284
+200

I don't understand why all the others are proposing such complex regular expressions or such long code. Essentially, you want to grab two kinds of things from your string: sequences of characters that aren't spaces or quotes, and sequences of characters that begin and end with a quote, with no quotes in between, for two kinds of quotes. You can easily match those things with this regular expression:

[^\s"']+|"([^"]*)"|'([^']*)'

I added the capturing groups because you don't want the quotes in the list.

This Java code builds the list, adding the capturing group if it matched to exclude the quotes, and adding the overall regex match if the capturing group didn't match (an unquoted word was matched).

List<String> matchList = new ArrayList<String>();
Pattern regex = Pattern.compile("[^\\s\"']+|\"([^\"]*)\"|'([^']*)'");
Matcher regexMatcher = regex.matcher(subjectString);
while (regexMatcher.find()) {
    if (regexMatcher.group(1) != null) {
        // Add double-quoted string without the quotes
        matchList.add(regexMatcher.group(1));
    } else if (regexMatcher.group(2) != null) {
        // Add single-quoted string without the quotes
        matchList.add(regexMatcher.group(2));
    } else {
        // Add unquoted word
        matchList.add(regexMatcher.group());
    }
} 

If you don't mind having the quotes in the returned list, you can use much simpler code:

List<String> matchList = new ArrayList<String>();
Pattern regex = Pattern.compile("[^\\s\"']+|\"[^\"]*\"|'[^']*'");
Matcher regexMatcher = regex.matcher(subjectString);
while (regexMatcher.find()) {
    matchList.add(regexMatcher.group());
} 
7
  • 1
    Jan, thanks for your response. BTW, I'm a big fan of EditPad.
    – carlsz
    Dec 14, 2008 at 16:05
  • What if I want to allow escaped quotes in the strings \"?
    – Monstieur
    May 6, 2014 at 14:19
  • 3
    The problem with this answer is with unmatched quote: John's mother results splited in [John, s, mother]
    – leonbloy
    May 16, 2014 at 20:10
  • 2
    To fix the issue leonbloy outlines, you can re-order the operands a bit and omit the quotes from the whitespace-group: "([^"]*)"|'([^']*)'|[^\s]+. Sep 15, 2014 at 1:26
  • 2
    Building upon this and other answers, the following regex allows escaping characters inside the quotes: "([^"\\]*(?:\\.[^"\\]*)*)"|'([^'\\]*(?:\\.[^'\\]*)*)'|[^\s]+. See stackoverflow.com/questions/5695240/…
    – Limnic
    May 23, 2019 at 19:45
20

There are several questions on StackOverflow that cover this same question in various contexts using regular expressions. For instance:

UPDATE: Sample regex to handle single and double quoted strings. Ref: How can I split on a string except when inside quotes?

m/('.*?'|".*?"|\S+)/g 

Tested this with a quick Perl snippet and the output was as reproduced below. Also works for empty strings or whitespace-only strings if they are between quotes (not sure if that's desired or not).

This
is
a
string
that
"will be"
highlighted
when
your
'regular expression'
matches
something.

Note that this does include the quote characters themselves in the matched values, though you can remove that with a string replace, or modify the regex to not include them. I'll leave that as an exercise for the reader or another poster for now, as 2am is way too late to be messing with regular expressions anymore ;)

2
  • I think your regex allows mismatched quotes, e.g. "will be' and 'regular expressions". Dec 14, 2008 at 7:05
  • @Zach - you're right, it does...updated it to fix that just in case
    – Jay
    Dec 14, 2008 at 7:36
6

If you want to allow escaped quotes inside the string, you can use something like this:

(?:(['"])(.*?)(?<!\\)(?>\\\\)*\1|([^\s]+))

Quoted strings will be group 2, single unquoted words will be group 3.

You can try it on various strings here: http://www.fileformat.info/tool/regex.htm or http://gskinner.com/RegExr/

3

The regex from Jan Goyvaerts is the best solution I found so far, but creates also empty (null) matches, which he excludes in his program. These empty matches also appear from regex testers (e.g. rubular.com). If you turn the searches arround (first look for the quoted parts and than the space separed words) then you might do it in once with:

("[^"]*"|'[^']*'|[\S]+)+
2
(?<!\G".{0,99999})\s|(?<=\G".{0,99999}")\s

This will match the spaces not surrounded by double quotes. I have to use min,max {0,99999} because Java doesn't support * and + in lookbehind.

0
1

It'll probably be easier to search the string, grabbing each part, vs. split it.

Reason being, you can have it split at the spaces before and after "will be". But, I can't think of any way to specify ignoring the space between inside a split.

(not actual Java)

string = "This is a string that \"will be\" highlighted when your 'regular expression' matches something.";

regex = "\"(\\\"|(?!\\\").)+\"|[^ ]+"; // search for a quoted or non-spaced group
final = new Array();

while (string.length > 0) {
    string = string.trim();
    if (Regex(regex).test(string)) {
        final.push(Regex(regex).match(string)[0]);
        string = string.replace(regex, ""); // progress to next "word"
    }
}

Also, capturing single quotes could lead to issues:

"Foo's Bar 'n Grill"

//=>

"Foo"
"s Bar "
"n"
"Grill"
1
  • Your solution doesn't handle single-quoted strings, which are part of Carl's example. Dec 14, 2008 at 13:08
1

String.split() is not helpful here because there is no way to distinguish between spaces within quotes (don't split) and those outside (split). Matcher.lookingAt() is probably what you need:

String str = "This is a string that \"will be\" highlighted when your 'regular expression' matches something.";
str = str + " "; // add trailing space
int len = str.length();
Matcher m = Pattern.compile("((\"[^\"]+?\")|('[^']+?')|([^\\s]+?))\\s++").matcher(str);

for (int i = 0; i < len; i++)
{
    m.region(i, len);

    if (m.lookingAt())
    {
        String s = m.group(1);

        if ((s.startsWith("\"") && s.endsWith("\"")) ||
            (s.startsWith("'") && s.endsWith("'")))
        {
            s = s.substring(1, s.length() - 1);
        }

        System.out.println(i + ": \"" + s + "\"");
        i += (m.group(0).length() - 1);
    }
}

which produces the following output:

0: "This"
5: "is"
8: "a"
10: "string"
17: "that"
22: "will be"
32: "highlighted"
44: "when"
49: "your"
54: "regular expression"
75: "matches"
83: "something."
1

I liked Marcus's approach, however, I modified it so that I could allow text near the quotes, and support both " and ' quote characters. For example, I needed a="some value" to not split it into [a=, "some value"].

(?<!\\G\\S{0,99999}[\"'].{0,99999})\\s|(?<=\\G\\S{0,99999}\".{0,99999}\"\\S{0,99999})\\s|(?<=\\G\\S{0,99999}'.{0,99999}'\\S{0,99999})\\s"
1

Jan's approach is great but here's another one for the record.

If you actually wanted to split as mentioned in the title, keeping the quotes in "will be" and 'regular expression', then you could use this method which is straight out of Match (or replace) a pattern except in situations s1, s2, s3 etc

The regex:

'[^']*'|\"[^\"]*\"|( )

The two left alternations match complete 'quoted strings' and "double-quoted strings". We will ignore these matches. The right side matches and captures spaces to Group 1, and we know they are the right spaces because they were not matched by the expressions on the left. We replace those with SplitHere then split on SplitHere. Again, this is for a true split case where you want "will be", not will be.

Here is a full working implementation (see the results on the online demo).

import java.util.*;
import java.io.*;
import java.util.regex.*;
import java.util.List;

class Program {
public static void main (String[] args) throws java.lang.Exception  {

String subject = "This is a string that \"will be\" highlighted when your 'regular expression' matches something.";
Pattern regex = Pattern.compile("\'[^']*'|\"[^\"]*\"|( )");
Matcher m = regex.matcher(subject);
StringBuffer b= new StringBuffer();
while (m.find()) {
    if(m.group(1) != null) m.appendReplacement(b, "SplitHere");
    else m.appendReplacement(b, m.group(0));
}
m.appendTail(b);
String replaced = b.toString();
String[] splits = replaced.split("SplitHere");
for (String split : splits) System.out.println(split);
} // end main
} // end Program
1

If you are using c#, you can use

string input= "This is a string that \"will be\" highlighted when your 'regular expression' matches <something random>";

List<string> list1 = 
                Regex.Matches(input, @"(?<match>\w+)|\""(?<match>[\w\s]*)""|'(?<match>[\w\s]*)'|<(?<match>[\w\s]*)>").Cast<Match>().Select(m => m.Groups["match"].Value).ToList();

foreach(var v in list1)
   Console.WriteLine(v);

I have specifically added "|<(?[\w\s]*)>" to highlight that you can specify any char to group phrases. (In this case I am using < > to group.

Output is :

This
is
a
string
that
will be
highlighted
when
your
regular expression 
matches
something random
0
1

1st one-liner using String.split()

String s = "This is a string that \"will be\" highlighted when your 'regular expression' matches something.";
String[] split = s.split( "(?<!(\"|').{0,255}) | (?!.*\\1.*)" );

[This, is, a, string, that, "will be", highlighted, when, your, 'regular expression', matches, something.]

don't split at the blank, if the blank is surrounded by single or double quotes
split at the blank when the 255 characters to the left and all characters to the right of the blank are neither single nor double quotes

adapted from original post (handles only double quotes)

4
  • How do I handle this case ?: str = "First Second=\"This is Second\" Third=3 \"this is Four\"" . I want is [First] [Second="This is Second"] [Third=3] [this is Four] Nov 18, 2022 at 5:18
  • 1
    @Nobel With Your example obviously the limits of split() are reached and you better use eg. replaceAll() as an alternative: String[] arr = s.split("(?<!\"[^\"]*)\\x20|(?<=\"|((?<==\\w+)\\x20))\\x20");
    – Kaplan
    Nov 18, 2022 at 20:08
  • 2
    actually this worked : String[] str1= str.split("\\s(?=(([^\"]*\"){2})*[^\"]*$)\\s*");. This regex matches spaces ONLY if it is followed by even number of double quotes. Nov 21, 2022 at 4:31
  • @Nobel Normally I would have handled the OP's problem by parsing the input. This is more flexible for changes in the syntax of the input and able to catch errors at an early stage when eg. a quote is missing, to prevent complete rubbish. I started my solution with the motivation in mind to return "this is Four" without quotes. That would certainly be difficult or impossible to implement in your indisputably clever solution.
    – Kaplan
    Nov 21, 2022 at 10:31
1

When you come across this pattern like this :

String str = "2022-11-10 08:35:00,470    RAV=REQ YIP=02.8.5.1 CMID=caonaustr CMN=\"Some Value Pyt Ltd\"";
//this helped
String[] str1= str.split("\\s(?=(([^\"]*\"){2})*[^\"]*$)\\s*");
System.out.println("Value of split string is "+ Arrays.toString(str1));

This results in :[2022-11-10, 08:35:00,470, PLV=REQ, YIP=02.8.5.1, CMID=caonaustr, CMN="Some Value Pyt Ltd"]

This regex matches spaces ONLY if it is followed by even number of double quotes.

1
  • 1
    Your solution worked best (at least for me). I changed the leading \\s to , since I wanted to break on commas, not blanks and this worked like a charm.
    – mmo
    Dec 23, 2023 at 16:14
0

I'm reasonably certain this is not possible using regular expressions alone. Checking whether something is contained inside some other tag is a parsing operation. This seems like the same problem as trying to parse XML with a regex -- it can't be done correctly. You may be able to get your desired outcome by repeatedly applying a non-greedy, non-global regex that matches the quoted strings, then once you can't find anything else, split it at the spaces... that has a number of problems, including keeping track of the original order of all the substrings. Your best bet is to just write a really simple function that iterates over the string and pulls out the tokens you want.

2
  • It's possible with a regex, see some of the samples I linked to. There's a few variations on this, and I've seen several similar questions on SO that address this via regular expressions.
    – Jay
    Dec 14, 2008 at 6:39
  • 1
    Know when not to use regex is more helpful knowledge then to be able to create a (?:(['"])(.*?)(?<!\)(?>\\\)*\1|([^\s]+))
    – Rene
    Dec 14, 2008 at 15:43
0

A couple hopefully helpful tweaks on Jan's accepted answer:

(['"])((?:\\\1|.)+?)\1|([^\s"']+)
  • Allows escaped quotes within quoted strings
  • Avoids repeating the pattern for the single and double quote; this also simplifies adding more quoting symbols if needed (at the expense of one more capturing group)
1
  • This breaks words with apostrophes in them, like you're Mar 20, 2019 at 17:55
0

You can also try this:

    String str = "This is a string that \"will be\" highlighted when your 'regular expression' matches something";
    String ss[] = str.split("\"|\'");
    for (int i = 0; i < ss.length; i++) {
        if ((i % 2) == 0) {//even
            String[] part1 = ss[i].split(" ");
            for (String pp1 : part1) {
                System.out.println("" + pp1);
            }
        } else {//odd
            System.out.println("" + ss[i]);
        }
    }
1
  • You should really add some explanation as to why this should work - you can also add code as well as the comments in the code itself - in its current form, it does not provide any explanation which can help the rest of the community to understand what you did to solve/answer the question. This is especially important for questions that already have answers. Sep 12, 2016 at 14:13
0

The following returns an array of arguments. Arguments are the variable 'command' split on spaces, unless included in single or double quotes. The matches are then modified to remove the single and double quotes.

using System.Text.RegularExpressions;

var args = Regex.Matches(command, "[^\\s\"']+|\"([^\"]*)\"|'([^']*)'").Cast<Match>
().Select(iMatch => iMatch.Value.Replace("\"", "").Replace("'", "")).ToArray();
1
  • 2
    Can you add a little bit of explanation to your answer so that others can more easily understand it? Ideally we want to avoid code-only answers.
    – Jaquez
    Jul 22, 2019 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.