0

I receive data as a vector<char>, from which I need to create a string. Vector may contain utf-16 characters (i.e. null bytes) and is a fixed size. Actual data is padded with null bytes to this fixed sized. So, for example, I can have the following vector:

\0 a \0 b \0 c \0 d \0 \0 \0 \0

Fixed size is 12 and the vector contains utf-16 string "abcd" padded with 4 null chars to size.

From this, I need to actually extract this string. I already have the code for converting from utf-16 to string, the thing where I got myself confused is find the number of characters (bytes) in the vector without the padding. In the example above, the number is 8.

I started by doing something like:

std::string CrmxFile::StringFromBytes(std::vector<char> data, int fixedsize) {

    std::vector<char>iterator it = data.rbegin();
    while(it != data.rend() && *it == '\0') {
        it++;
    }

    return std::string(&data[0], fixedsize - (it - data.rbegin());
}

However in the full context, the vector contains a lot of data and I need to do the above manipulation with only a specified part of it. For example, the vector may contain 1000 elements and I need to get the string that starts at position 30 and goes for a max of 12 chars. Of course, I can create another vector and copy the required 21 characters into it before applying the above logic, but I feel that I should be able to do something directly on the given vector. Yet, I can't grasp what iterators I am comparing with what. Any help is appreciated.

8
  • Is it an option to use a conversion function that does NOT require a terminated string (and instead takes a base pointer and wide-char count to convert)? More specifically, two iterators (first and last, which is one past the last char to convert)
    – WhozCraig
    Jan 9, 2013 at 12:39
  • @WhozCraig Not really. The data is coming from reading a binary file from a legacy system. It has a certain format that I had to read. The string is not necessarily UTF-16. I have to determine on the fly whether the data is utf-16 or 8-bit (I asked another question on that recently). The format is a mess, but unfortunately I cannot do anything about it.
    – Aleks G
    Jan 9, 2013 at 12:46
  • 1
    If you are not familiar with STL iterator don't use it, it's not required for your specific needs. You can handle it with plain old C pointers: const char* pc = &(*data.begin());
    – 9dan
    Jan 9, 2013 at 13:45
  • @9dan I am familiar with iterators all right. I'm just can't think of a way to compare iterator with a reverse_iterator. And, by the way, your code will not work, as the string may be UTF-16, that is contain null characters in the middle.
    – Aleks G
    Jan 9, 2013 at 14:51
  • -You are aware that UTF-16 can also be little endian, in which case your resulting string would contain an odd number of chars?
    – dhavenith
    Jan 9, 2013 at 15:00

2 Answers 2

0

Now, this is embarrassing: vector<char>::iterator is obviously a random access iterator, therefore I can decrement it. Hence my method now looks like this:

std::string CrmxFile::StringFromBytes(std::vector<char> data, int fixedsize) {
    std::vector<char>::iterator begin = data.begin() + start;
    std::vector<char>::iterator end = start + length - 1;
    while(it >= begin  && *it == '\0') {
        it--;
    }

    if(it >= begin) {
        int len = it - begin + 1;
        if(IsUtf8Heuristic(begin, begin + len) {
            return std::string(begin, begin + len);
        }
        else {  //(heuristically this is utf-16)
            len = ((len + 1) >> 1) << 1;
            std::string res;
            ConvertUtf16To8(begin, begin + len, std::back_inserter(res));
            return res;
        }
    }
    else {
        return "";
    }
}
-1

As I understand the question, you want to extract a part of max fixedsize from data, and erase all trailing zeroes. And from the comments you want the optimal solution.

For me, your code is overly complicated if the data will always be in array form. Use indices, they are more self describing.

std::vector<char> data = ...;
int fixedsize = ...;
int start = ...;

int i = start + fixedsize - 1; // last character that can be in the string
while(i >= start && data[i] == 0) i--; // 'remove' the trailing zeroes
std::string result(&data[start], i - start + 1);

This is the optimal algorithm, there are no 'more optimal' algorithm (there is a micro-optimization that consists in testing with ints rather than chars, ie 4 chars in a row).

3
  • I don't have a problem determining/converting strings. You're missing the question: the string is fixed width \0-padded. The question is about the optimal way to find the actual ending of the string.
    – Aleks G
    Jan 9, 2013 at 20:21
  • Ok then... Your question does not make it clear, you should edit it. I will edit my answer after.
    – Synxis
    Jan 9, 2013 at 20:29
  • Just re-read it: it's very clear, I can't see how it can be improved.
    – Aleks G
    Jan 9, 2013 at 20:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.