7

Following my previous question: Why `strchr` seems to work with multibyte characters, despite man page disclaimer?, I figured out that strchr was a bad choice.

Instead I am thinking about using strstr to look for a single character (multi-byte not char):

const char str[] = "This string contains é which is a multi-byte character";
char * pos = strstr(str, "é"); // 'é' = 0xC3A9: 2 bytes 
printf("%s\n", pos);

Ouput:

é which is a multi-byte character

Which is what I expect: the position of the 1st byte of my multi-byte character.

A priori, this is not the canonical use of strstr but it seems to work well.
Is this workaround safe ? Can you think about any side-effect or special case that would cause a bug ?

[EDIT]: I should precise that I do not want to use wchar_t type and that strings I handle are UTF-8 encoded (I am aware this choice can be discussed but this an irrelevant debate)

8

2 Answers 2

8

Edit
Based on updated question from OP that "can such false positive exist in an UTF-8 context" So the answer is UTF-8 is designed in such a way that it is immune to partial mismatch of character as shown above and cause any false positive. So it is completely safe to use strstr with UTF-8 coded multibyte characters.

Original Answer
No strstr is not suitable for strings containing multi-byte characters.

If you are searching for a string that doesn't contain multi-byte character inside a string that contains multi-byte character, it may give false positive. (While using shift-jis encoding in japanese locale, strstr("掘something", "@some") may give false positive)

+---------+----+----+----+
|   c1    | c2 | c3 | c4 |  <--- string
+---------+----+----+----+

     +----+----+----+
     | c5 | c2 | c3 |  <--- string to search
     +----+----+----+

If trailing part of c1 (accidentally) matches with c5, you may get incorrect result. I would suggest using unicode with unicode substring check function or multibyte substring check functions. (_mbsstr for example)

6
  • Thanks, I had the intuition I was missing something. But now the question is: can such false positive exist in an UTF-8 context ?
    – n0p
    Aug 29, 2014 at 16:03
  • 5
    You can't get false positives with UTF-8 because the initial byte of character is always different than any of the possible subsequent characters.
    – Ross Ridge
    Aug 29, 2014 at 16:09
  • 4
    As Ross has already mentioned use of strstr for utf-8 and completely safe. UTF-8 codes are generated in a way that false positives are not possible between characters of UTF-8 character set.
    – Mohit Jain
    Aug 29, 2014 at 16:11
  • Based on OP's additional information that strings are UTF-8, this answer is wrong. It should at least be updated with additional information to make it clear that strstr is perfectly safe for OP's usage and that the concerns in the answer only apply to legacy encodings like Shift_JIS. Aug 29, 2014 at 16:42
  • Helpful answer ! What about strcmp ?
    – Virus721
    May 18, 2015 at 13:59
1

Modern systems use UTF-8 (or ASCII) as their multibyte encoding, where the use of this function is safe.

To be strictly conforming and make your code work even on old/exotic platforms, you need to take additional problems into account.

First, the good news: In every multibyte encoding, a 0-byte indicates the end of a string, regardless of state. This means, your strstr won’t cause a crash or something, but the result may be wrong.

As an example, consider UTF-7, a 7-bit clean way to encode Unicode. UTF-7 is a multibyte encoding having a shift state, which means how a byte is interpreted may depend on the context where it appears. E.g. (cf. Wikipedia) “£1AKM” is encoded as +AKM-AKM in UTF-7, where the + sign changes the state and the interpretation of letters like A. Doing strstr(str, "AKM") would match the first AKM portion (after the +), although this is part of the encoding of £ and actually should match the AKM portion after the - (setting the shift state back to the initial state).

1
  • I forgot to precise that I use UTF-8 encoding, but thanks for the tips anyway.
    – n0p
    Aug 29, 2014 at 18:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.