5

I need to search through a chunk of memory for a string of characters, but several of these strings have every character null separated, like this:
"I. .a.m. .a. .s.t.r.i.n.g"
with all of the '.'s being null characters. My problem comes from actually getting this into memory. I've tried several ways, for instance:

 char* str2; 
 str2 = (char*)malloc(sizeof(char)*40);   
 memcpy((void*)str2, "123\0567\09abc", 12);    

Will put the following into the memory that str2 points to: 123.7.9abc..
Something like
str2 = "123456789\0abcde\054321";
Will have str2 pointing to a block of memory that looks like 123456789.abcde,321 , wherein the '.' is a null character, and the ',' is an actual comma.

So clearly inserting null characters into cstrings doesn't work as easily as I thought it did, like inserting a newline character. I encountered similar difficulties trying this with the string library as well. I could do separate assignments, something like:

 char* str;    
 str = (char*)malloc(sizeof(char)*40);  
 strcpy(str, "123");  
 strcpy(str+4, "abc");  
 strcpy(str+8, "ABC");  

But that is certainly not preferable, and I believe the problem lies in my understanding of how c-style strings are stored in memory. Clearly "abc\0123" doesn't actually go into memory as 61 62 63 00 31 32 33 (in hex). How is it stored, and how can I store what I need to?

(I also apologize for not having set the code in blocks, this is my first time posting a question, and somehow "four spaced" is more difficult than I can handle apparently. Thank you, Luchian. I see more newlines were needed.)

1
  • Use the {} button above the edit window, or ctrl-K, to create code blocks.
    – Jim Balter
    Jun 13, 2012 at 22:08

4 Answers 4

6

If every other char contains a null, then almost certainly you actually have UTF-16 encoded strings. Process them accordingly and your problems will disappear.

Assuming you are on Windows, where UTF-16 is common, you would use wchar_t* rather than char* to hold such strings. And you would use wide char string processing functions to operate on such data. For example, use wcscpy rather than strcpy and so on.

1
  • This definitely solved my practical problem, and lets me implement the search easily. This is the first time I've programmed on windows, and I hadn't encountered wide characters before, so when looking at the binary dumps that never occurred to me. And I kept thinking how weird it was to null-terminate every character... =P Refp below also helped me figure out what was causing the problem I was having doing it the old way.
    – Fulluphigh
    Jun 13, 2012 at 20:34
3

\0 is the starting sequence of an escaped character in octets, it's not just a "null character" (even though the use of it's own will result in one).


The easiest way to define a string containing a null-character followed by something that could also be treated as a part of an escaped characer in octet (such as "\012"1) is to split it up using this below feature of C:

char const * p = "123456789" "\0" "abcde" "\0" "54321";

1. "\012" will result in the character with the equivalent hex value of 0x0A, not three characters; 0x00, '1' and '2'.

2
  • Excellent answer. David's answer above works perfectly for what I need to do, but I still didn't know what was causing this behavior. I was unaware of octets being something you could even escape in this manner. Thank you, excellent answer. I don't have enough rep to upvote, but yeah.
    – Fulluphigh
    Jun 13, 2012 at 20:31
  • @Joshua It's a good idea to familiarize yourself with the basic syntax of your programming language. Here's a useful reference for C: ieng9.ucsd.edu/~cs30x/Std.C/syntax.html
    – Jim Balter
    Jun 13, 2012 at 22:07
2

First off, every second character being a NULL is a clear hallmark of a widestring - a string that's composed of two-byte characters, really an array of unsigned shorts. Depending on your compiler and settings, you might be better off using datatype wchar_t instead of char and wcsxxx() family of functions instead of strxxx().

On Windows, 2-byte widestrings (UTF-16, technically) is the native string format of the OS, so they're all around the place.

That said, strxxx() functions all assume that the string is null-terminated. So plan accordingly. Sometimes memxxx() will come to the rescue.

"abc\0123" does not go into memory the way you expect because \012 is being interpreted by the compiler as a single octal escape sequence - the character with octal code 12 (that's 0a hex). To avoid, use one of the following literals:

"abc\000123"
"abc\x00123"
"abc\0""123"

The snippet where you generate a string from chunks is mostly correct. It's just that I'd rather use

strcpy(str+strlen(str)+1, "123");

that guarantees that the next chunk will be written past the null character of the previous chunk.

1
  • 2
    \054 is being interpreted by the compiler as a single octal escape sequence. Octal 54 is hex 2c. To make a true string with an embedded null, use the following literal: "abcde\00054321" or "abcde\0""54321", or "abcde\x0054321". The escape sequence parser is matching greedily, obviously. Stick to string literals that don't allow for ambiguous interpretation. Jun 13, 2012 at 20:29
0

I am a bit confused by your question. But let me guess what is going on. You are looking at 16 bit wchat_t string and not a normal c string. wchar getting ascii characters may look like null separated between letters but actually this is normal.

simply (wchar_t *)XXX where XXX is a pointer to that region of memory and lookup wchar_t operations like wcscpy etc... as for the nulls between strings, this may actually be a known method to pass multiple string construct. You can simply iterate after your read each string until normally you encounter 2 consecutive nulls.

Hope I have answered your question. Good luck!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.