234
string s = "おはよう";
wstring ws = FUNCTION(s, ws);

How would i assign the contents of s to ws?

Searched google and used some techniques but they can't assign the exact content. The content is distorted.

6
  • 8
    I don't think strings accepts >8-bit characters. Is it already encoded in UTF-8?
    – kennytm
    Commented Apr 4, 2010 at 7:36
  • 4
    What's your system encoding that it would make "おはよう" a system-encoded string?
    – sbi
    Commented Apr 4, 2010 at 7:42
  • I believe MSVC will accept that and make it some multibyte encoding, maybe UTF-8. Commented Apr 4, 2010 at 7:47
  • 2
    @Potatoswatter: MSVC doesn't use UTF-8 by default for ANYTHING. If you enter those characters, it asks which encoding to convert the file to, and defaults to codepage 1252. Commented Sep 3, 2013 at 16:58
  • 3
    @Samir: more important is what is the encoding of the file? Can you move that string to the beginning of the file and show a hexdump of that part? We can probably identify it from that. Commented Sep 3, 2013 at 16:59

20 Answers 20

300

NOTE! See Note (2023-10-05) at the bottom!

Assuming that the input string in your example (おはよう) is a UTF-8 encoded (which it isn't, by the looks of it, but let's assume it is for the sake of this explanation :-)) representation of a Unicode string of your interest, then your problem can be fully solved with the standard library (C++11 and newer) alone.

The TL;DR version:

#include <locale>
#include <codecvt>
#include <string>

std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;
std::string narrow = converter.to_bytes(wide_utf16_source_string);
std::wstring wide = converter.from_bytes(narrow_utf8_source_string);

Longer online compilable and runnable example:

(They all show the same example. There are just many for redundancy...)

Note (old):

As pointed out in the comments and explained in https://stackoverflow.com/a/17106065/6345 there are cases when using the standard library to convert between UTF-8 and UTF-16 might give unexpected differences in the results on different platforms. For a better conversion, consider std::codecvt_utf8 as described on http://en.cppreference.com/w/cpp/locale/codecvt_utf8

Note (new):

Since the codecvt header is deprecated in C++17, some worry about the solution presented in this answer were raised. However, the C++ standards committee added an important statement in http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0618r0.html saying

this library component should be retired to Annex D, along side , until a suitable replacement is standardized.

So in the foreseeable future, the codecvt solution in this answer is safe and portable.

Note (2023-10-05):

Proposal to remove the deprecated codecvt and wstring_convert in C++26:

26
  • 2
    Check what encoding you save VS files with Commented Nov 8, 2013 at 10:39
  • 9
    Be aware that this is C++11-only!
    – bk138
    Commented Jan 15, 2014 at 13:58
  • 36
    Please note that <codecvt> is deprecated since C++17.
    – tambre
    Commented Apr 9, 2017 at 11:01
  • 2
    @tambre - thanks for pointing this out, I added the Note (new) paragraph to address this. Commented Jul 28, 2017 at 7:32
  • 6
    If I were the sole and all powerful ruler of this world, I would decree, that UTF16 is outlawed and only UTF8 and UTF32 are legal and usable without the danger of severe punishment. ;) I mean seriously - what is UTF16 good for if it still is multi-code point?! All this hassle with conversions has the root cause that UTF16 is flawed, IMHO.
    – BitTickler
    Commented Mar 7, 2020 at 20:23
58
int StringToWString(std::wstring &ws, const std::string &s)
{
    std::wstring wsTmp(s.begin(), s.end());

    ws = wsTmp;

    return 0;
}
10
  • 121
    This only works if all the characters are single byte, i.e. ASCII or ISO-8859-1. Anything multi-byte will fail miserably, including UTF-8. The question clearly contains multi-byte characters. Commented Sep 3, 2013 at 16:22
  • 31
    This answer is clearly insufficient and does nothing but copy narrow characters as is into wide characters. See the other answers, particularly the one by Johann Gerell, for how to properly go from a multi-byte or utf8 encoded string to a utf16 wstring.
    – DLRdave
    Commented Oct 13, 2013 at 11:29
  • 11
    this answer is dangerous and will probably break on non-ascii system. i.e. an arabic filename will get mangled by this hack.
    – Stephen
    Commented Apr 18, 2014 at 19:50
  • 14
    This answer is useful if you ignore the nuance of the question's body and focus on the question title, which is what brought me here from Google. As is, the question's title is extremely misleading and should be altered to reflect the true question being asked
    – Anne Quinn
    Commented Dec 17, 2015 at 7:37
  • 3
    This works only for 7-bit ASCII characters. For latin1, it works only if char is configured as unsigned. If the type char is signed (which is most of time the case), characters > 127 will give wrong results.
    – huyc
    Commented May 16, 2016 at 18:32
42

Your question is underspecified. Strictly, that example is a syntax error. However, std::mbstowcs is probably what you're looking for.

It is a C-library function and operates on buffers, but here's an easy-to-use idiom, courtesy of Mooing Duck:

std::wstring ws(s.size(), L' '); // Overestimate number of code points.
ws.resize(std::mbstowcs(&ws[0], s.c_str(), s.size())); // Shrink to fit.
17
  • 1
    string s = "おはよう"; wchar_t* buf = new wchar_t[ s.size() ]; size_t num_chars = mbstowcs( buf, s.c_str(), s.size() ); wstring ws( buf, num_chars ); // ws = distorted
    – Samir
    Commented Apr 4, 2010 at 8:23
  • 1
    @Samir: You have to make sure the runtime encoding is the same as the compile-time encoding. You might need to setlocale or adjust compiler flags. I don't know because I don't use Windows, but this is why it's not a common feature. Consider the other answer if possible. Commented Apr 4, 2010 at 9:30
  • 2
    std::string ws(s.size()); ws.resize(mbstowcs(&ws[0], s.c_str(), s.size()); RAII FTW Commented Sep 3, 2013 at 17:01
  • 2
    @WaffleSouffle That's out of date. Contiguous implementations have been required since 2011 and implementations quit such tricks long before that. Commented Sep 22, 2014 at 23:53
  • 1
    and some environments like mingw still don't have the codecvt header so some of the 'better' solutions earlier up don't work meaning this problem still has no good solutions in mingw even as of Dec 2014
    – Brian Jack
    Commented Dec 11, 2014 at 19:54
26

If you are using Windows/Visual Studio and need to convert a string to wstring you could use:

#include <AtlBase.h>
#include <atlconv.h>
...
string s = "some string";
CA2W ca2w(s.c_str());
wstring w = ca2w;
printf("%s = %ls", s.c_str(), w.c_str());

Same procedure for converting a wstring to string (sometimes you will need to specify a codepage):

#include <AtlBase.h>
#include <atlconv.h>
...
wstring w = L"some wstring";
CW2A cw2a(w.c_str());
string s = cw2a;
printf("%s = %ls", s.c_str(), w.c_str());

You could specify a codepage and even UTF8 (that's pretty nice when working with JNI/Java). A standard way of converting a std::wstring to utf8 std::string is showed in this answer.

// 
// using ATL
CA2W ca2w(str, CP_UTF8);

// 
// or the standard way taken from the answer above
#include <codecvt>
#include <string>

// convert UTF-8 string to wstring
std::wstring utf8_to_wstring (const std::string& str) {
    std::wstring_convert<std::codecvt_utf8<wchar_t>> myconv;
    return myconv.from_bytes(str);
}

// convert wstring to UTF-8 string
std::string wstring_to_utf8 (const std::wstring& str) {
    std::wstring_convert<std::codecvt_utf8<wchar_t>> myconv;
    return myconv.to_bytes(str);
}

If you want to know more about codepages there is an interesting article on Joel on Software: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets.

These CA2W (Convert Ansi to Wide=unicode) macros are part of ATL and MFC String Conversion Macros, samples included.

Sometimes you will need to disable the security warning #4995', I don't know of other workaround (to me it happen when I compiled for WindowsXp in VS2012).

#pragma warning(push)
#pragma warning(disable: 4995)
#include <AtlBase.h>
#include <atlconv.h>
#pragma warning(pop)

Edit: Well, according to this article the article by Joel appears to be: "while entertaining, it is pretty light on actual technical details". Article: What Every Programmer Absolutely, Positively Needs To Know About Encoding And Character Sets To Work With Text.

3
  • Probably the fact that it promotes non-portable code. Commented Aug 24, 2015 at 23:11
  • Yes, that's why I stated that this works only in Windows/Visual Studio. But at least this solution is correct, and not this one: char* str = "hello worlddd"; wstring wstr (str, str+strlen(str));
    – lmiguelmh
    Commented Aug 25, 2015 at 23:07
  • Additional note: CA2W is under namespace of ATL. (ATL::CA2W)
    – Val
    Commented Mar 22, 2016 at 8:55
25

Windows API only, pre C++11 implementation, in case someone needs it:

#include <stdexcept>
#include <vector>
#include <windows.h>

using std::runtime_error;
using std::string;
using std::vector;
using std::wstring;

wstring utf8toUtf16(const string & str)
{
   if (str.empty())
      return wstring();

   size_t charsNeeded = ::MultiByteToWideChar(CP_UTF8, 0, 
      str.data(), (int)str.size(), NULL, 0);
   if (charsNeeded == 0)
      throw runtime_error("Failed converting UTF-8 string to UTF-16");

   vector<wchar_t> buffer(charsNeeded);
   int charsConverted = ::MultiByteToWideChar(CP_UTF8, 0, 
      str.data(), (int)str.size(), &buffer[0], buffer.size());
   if (charsConverted == 0)
      throw runtime_error("Failed converting UTF-8 string to UTF-16");

   return wstring(&buffer[0], charsConverted);
}
2
  • You can optimize it. There's no need to do double copy of the string by using a vector. Simply reserve the characters in the string by doing wstring strW(charsNeeded + 1); and then use it as buffer for conversion: &strW[0]. Lastly ensure last null is present after conversion by doing strW[charsNeeded] = 0;
    – c00000fd
    Commented Feb 6, 2017 at 3:35
  • 3
    @c00000fd, as far as I know, the std::basic_string internal buffer is required to be continuous only since C++11 standard. My code is pre C++11, as noted on the top of the post. Therefore, the &strW[0] code would be not standard compliant and might legitimately crash at runtime.
    – Alex Che
    Commented Feb 6, 2017 at 7:03
23

Here's a way to combining string, wstring and mixed string constants to wstring. Use the wstringstream class.

This does NOT work for multi-byte character encodings. This is just a dumb way of throwing away type safety and expanding 7 bit characters from std::string into the lower 7 bits of each character of std:wstring. This is only useful if you have a 7-bit ASCII strings and you need to call an API that requires wide strings.

#include <sstream>

std::string narrow = "narrow";
std::wstring wide = L"wide";

std::wstringstream cls;
cls << " abc " << narrow.c_str() << L" def " << wide.c_str();
std::wstring total= cls.str();
6
  • The answer seems interesting. Could you please explain a bit: will this work for multi-byte encodings, and why/how?
    – wh1t3cat1k
    Commented Nov 14, 2015 at 8:23
  • 1
    encoding schemes are orthogonal to the storage class. string stores 1 byte characters and wstring stores 2 byte characters. something like utf8 stores mulitbyte characters as a series of 1 byte values, i.e. in a string. the string classes don't help with encoding. I'm not an expert on encoding classes in c++. Commented Nov 14, 2015 at 16:40
  • 2
    Any reason why this one is not the best answer, given how short and simple it is? Any cases that it does not cover?
    – Ryuu
    Commented May 4, 2018 at 9:56
  • @MarkLakata, I read your answer to the first comment but am still not sure. Will it work for multi-byte characters? In other words, is it not prone to the same pitfall as this answer?
    – Marc.2377
    Commented Sep 10, 2019 at 6:42
  • 1
    @Marc.2377 This does NOT work for multi-byte character encodings. This is just a dumb way of throwing away type safety and expanding 7 bit characters from std::string into the lower 7 bits of each character of std:wstring. This is only useful if you have a 7-bit ASCII strings and you need to call an API that requires wide strings. Look at stackoverflow.com/a/8969776/3258851 if you need something more sophisticated. Commented Sep 10, 2019 at 22:42
19

From char* to wstring:

char* str = "hello worlddd";
wstring wstr (str, str+strlen(str));

From string to wstring:

string str = "hello worlddd";
wstring wstr (str.begin(), str.end());

Note this only works well if the string being converted contains only ASCII characters.

4
  • 9
    Because this only works if the encoding is Windows-1252, which can't even hold the letters in the question. Commented Sep 4, 2013 at 16:54
  • 3
    this is least error prone way of doing it, when you know you dealing with ASCII. Which is a prominent usecase when porting apps to newer api's. Commented Feb 25, 2014 at 19:45
  • This is not the way. If you are using Visual Studio you should use atlconv.h. Check the other answers.
    – lmiguelmh
    Commented Nov 5, 2014 at 22:50
  • Points for simplicity!
    – UserX
    Commented Oct 7, 2022 at 23:40
12

This variant of it is my favourite in real life. It converts the input, if it is valid UTF-8, to the respective wstring. If the input is corrupted, the wstring is constructed out of the single bytes. This is extremely helpful if you cannot really be sure about the quality of your input data.

std::wstring convert(const std::string& input)
{
    try
    {
        std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;
        return converter.from_bytes(input);
    }
    catch(std::range_error& e)
    {
        size_t length = input.length();
        std::wstring result;
        result.reserve(length);
        for(size_t i = 0; i < length; i++)
        {
            result.push_back(input[i] & 0xFF);
        }
        return result;
    }
}
1
9

using Boost.Locale:

ws = boost::locale::conv::utf_to_utf<wchar_t>(s);
0
8

You can use boost path or std path; which is a lot more easier. boost path is easier for cross-platform application

#include <boost/filesystem/path.hpp>

namespace fs = boost::filesystem;

//s to w
std::string s = "xxx";
auto w = fs::path(s).wstring();

//w to s
std::wstring w = L"xxx";
auto s = fs::path(w).string();

if you like to use std:

#include <filesystem>
namespace fs = std::filesystem;

//The same

c++ older version

#include <experimental/filesystem>
namespace fs = std::experimental::filesystem;

//The same

The code within still implement a converter which you dont have to unravel the detail.

0
6

For me the most uncomplicated option without big overhead is:

Include:

#include <atlbase.h>
#include <atlconv.h>

Convert:

char* whatever = "test1234";
std::wstring lwhatever = std::wstring(CA2W(std::string(whatever).c_str()));

If needed:

lwhatever.c_str();
4

String to wstring

std::wstring Str2Wstr(const std::string& str)
{
    int size_needed = MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), NULL, 0);
    std::wstring wstrTo(size_needed, 0);
    MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), &wstrTo[0], size_needed);
    return wstrTo;
}

wstring to String

std::string Wstr2Str(const std::wstring& wstr)
{
    typedef std::codecvt_utf8<wchar_t> convert_typeX;
    std::wstring_convert<convert_typeX, wchar_t> converterX;
    return converterX.to_bytes(wstr);
}
1
  • 1
    This Str2Wstr has a problem with 0 termination. It is not possible to concatenate the generated wstrings anymore via "+" (like in wstring s3 = s1 + s2). I will post a answer soon solving this problem. Have to do some testing for memory leaks first. Commented Jan 6, 2020 at 18:51
4

If you have QT and if you are lazy to implement a function and stuff you can use

std::string str;
QString(str).toStdWString()
3
  • 1
    Almost, but you should just start out with a QString, because the QString constructor can't accept a string for some reason. Commented May 20, 2020 at 10:56
  • 1
    You can use doc.qt.io/qt-5/qstring.html#fromStdString Commented May 20, 2020 at 11:00
  • This is nice. Also, you can use .c_str() to let QString accept your string in the constructor.
    – miep
    Commented Jul 1, 2020 at 9:46
2

Here is my super basic solution that might not work for everyone. But would work for a lot of people.

It requires usage of the Guideline Support Library. Which is a pretty official C++ library that was designed by many C++ committee authors:

    std::string to_string(std::wstring const & wStr)
    {
        std::string temp = {};

        for (wchar_t const & wCh : wStr)
        {
            // If the string can't be converted gsl::narrow will throw
            temp.push_back(gsl::narrow<char>(wCh));
        }

        return temp;
    }

All my function does is allow the conversion if possible. Otherwise throw an exception.

Via the usage of gsl::narrow (https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#es49-if-you-must-use-a-cast-use-a-named-cast)

1

method s2ws works well. Hope helps.

std::wstring s2ws(const std::string& s) {
    std::string curLocale = setlocale(LC_ALL, ""); 
    const char* _Source = s.c_str();
    size_t _Dsize = mbstowcs(NULL, _Source, 0) + 1;
    wchar_t *_Dest = new wchar_t[_Dsize];
    wmemset(_Dest, 0, _Dsize);
    mbstowcs(_Dest,_Source,_Dsize);
    std::wstring result = _Dest;
    delete []_Dest;
    setlocale(LC_ALL, curLocale.c_str());
    return result;
}
2
  • 9
    What is with all of these answers allocating dynamic memory in an unsafe way, and then copying the data from the buffer to the string? Why does nobody get rid of the unsafe middleman? Commented Sep 4, 2013 at 16:56
  • hahakubile, can you help please with something similar for ws2s ?
    – cristian
    Commented Jun 9, 2016 at 15:32
1

Based upon my own testing (On windows 8, vs2010) mbstowcs can actually damage original string, it works only with ANSI code page. If MultiByteToWideChar/WideCharToMultiByte can also cause string corruption - but they tends to replace characters which they don't know with '?' question marks, but mbstowcs tends to stop when it encounters unknown character and cut string at that very point. (I have tested Vietnamese characters on finnish windows).

So prefer Multi*-windows api function over analogue ansi C functions.

Also what I've noticed shortest way to encode string from one codepage to another is not use MultiByteToWideChar/WideCharToMultiByte api function calls but their analogue ATL macros: W2A / A2W.

So analogue function as mentioned above would sounds like:

wstring utf8toUtf16(const string & str)
{
   USES_CONVERSION;
   _acp = CP_UTF8;
   return A2W( str.c_str() );
}

_acp is declared in USES_CONVERSION macro.

Or also function which I often miss when performing old data conversion to new one:

string ansi2utf8( const string& s )
{
   USES_CONVERSION;
   _acp = CP_ACP;
   wchar_t* pw = A2W( s.c_str() );

   _acp = CP_UTF8;
   return W2A( pw );
}

But please notice that those macro's use heavily stack - don't use for loops or recursive loops for same function - after using W2A or A2W macro - better to return ASAP, so stack will be freed from temporary conversion.

1

utf-8 implementation

Assuming that your std::string is utf8-encoded, this is a platform-independent implementation of wstring-string conversion functions:

#include <codecvt>
#include <locale>
#include <string>
#include <type_traits>

std::string wstring_to_utf8(std::wstring const& str)
{
  std::wstring_convert<std::conditional<
        sizeof(wchar_t) == 4,
        std::codecvt_utf8<wchar_t>,
        std::codecvt_utf8_utf16<wchar_t>>::type> converter;
  return converter.to_bytes(str);
}

std::wstring utf8_to_wstring(std::string const& str)
{
  std::wstring_convert<std::conditional<
        sizeof(wchar_t) == 4,
        std::codecvt_utf8<wchar_t>,
        std::codecvt_utf8_utf16<wchar_t>>::type> converter;
  return converter.from_bytes(str);
}

The currently most upvoted answer looks similar, but produces incorrect results for non-BMP characters (i.e. Emojis 🚒) on non-Windows platforms. wchar_t is UTF-16 on windows, but UTF-32 everywhere else. The std::conditional takes care of that distinction.

MSVC Deprecation Warning

On msvc this might generate some deprecation warnings. You can disable these by wrapping the functions in

#pragma warning(push)
#pragma warning(disable : 4996)
<the two functions>
#pragma warning(pop)

Johann Gerell's answer explains why it's ok to disable that warning.

Getting utf-8 on msvc

Note that when you write a normal string in your source (like std::string s = "おはよう";), it won't be utf-8 encoded per default on msvc. I would strongly recommend setting your msvc character set to utf-8 to address this: https://learn.microsoft.com/en-us/cpp/build/reference/utf-8-set-source-and-executable-character-sets-to-utf-8?view=msvc-170

0

std::string -> wchar_t[] with safe mbstowcs_s function:

auto ws = std::make_unique<wchar_t[]>(s.size() + 1);
mbstowcs_s(nullptr, ws.get(), s.size() + 1, s.c_str(), s.size());

This is from my sample code

-1

use this code to convert your string to wstring

std::wstring string2wString(const std::string& s){
    int len;
    int slength = (int)s.length() + 1;
    len = MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, 0, 0); 
    wchar_t* buf = new wchar_t[len];
    MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, buf, len);
    std::wstring r(buf);
    delete[] buf;
    return r;
}

int main(){
    std::wstring str="your string";
    std::wstring wStr=string2wString(str);
    return 0;
}
2
  • 6
    Note that the question has no mentioning of Windows and this answer is Windows-only. Commented Aug 27, 2015 at 14:11
  • CP_ACP is most certainly the wrong argument. All of a sudden, the executing thread's environment state has an effect on the behavior of the code. Not advisable. Specify a fixed character encoding in your conversion. (And consider handling errors.) Commented Jan 12, 2016 at 0:56
-3

string s = "おはよう"; is an error.

You should use wstring directly:

wstring ws = L"おはよう";
6
  • 1
    That's not going to work either. You'll have to convert those non-BMP characters to C escape sequences. Commented Apr 4, 2010 at 7:49
  • 3
    @Dave: it does work if your compiler supports unicode in source files, and all the ones in the last decade do (visual studio, gcc, ...) Commented Apr 4, 2010 at 7:52
  • Hi, regardless of the default system encoding (I may have Arabic as my default system encoding for example), what should the encoding of the source code file for L"おはよう" to work? should it be in UTF-16, or can I have UTF-8 without BOM for the .cpp file encoding? Commented Aug 12, 2010 at 4:26
  • 2
    @afriza: it doesn't really matter as long as your compile supports it Commented Aug 12, 2010 at 14:00
  • 2
    It is not an error; extended characters in a "narrow" string are defined to map to multibyte sequences. The compiler should support it as long as the OS does, which is the least you can ask. Commented Oct 13, 2013 at 0:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.