271

The question is how to convert wstring to string?

I have next example :

#include <string>
#include <iostream>

int main()
{
    std::wstring ws = L"Hello";
    std::string s( ws.begin(), ws.end() );

  //std::cout <<"std::string =     "<<s<<std::endl;
    std::wcout<<"std::wstring =    "<<ws<<std::endl;
    std::cout <<"std::string =     "<<s<<std::endl;
}

the output with commented out line is :

std::string =     Hello
std::wstring =    Hello
std::string =     Hello

but without is only :

std::wstring =    Hello

Is anything wrong in the example? Can I do the conversion like above?

EDIT

New example (taking into account some answers) is

#include <string>
#include <iostream>
#include <sstream>
#include <locale>

int main()
{
    setlocale(LC_CTYPE, "");

    const std::wstring ws = L"Hello";
    const std::string s( ws.begin(), ws.end() );

    std::cout<<"std::string =     "<<s<<std::endl;
    std::wcout<<"std::wstring =    "<<ws<<std::endl;

    std::stringstream ss;
    ss << ws.c_str();
    std::cout<<"std::stringstream =     "<<ss.str()<<std::endl;
}

The output is :

std::string =     Hello
std::wstring =    Hello
std::stringstream =     0x860283c

therefore the stringstream can not be used to convert wstring into string.

21
  • 5
    How can you ask this question without specifying also the encodings? Jan 26, 2011 at 12:15
  • 5
    @tenfour: Why use std::wstring at all? stackoverflow.com/questions/1049947/…
    – dalle
    Jan 26, 2011 at 13:14
  • 19
    @dalle If you have data that is already encoded with UTF-16, whether or not UTF-16 is considered harmful is somewhat moot. And for what it's worth, I don't think any transformation form is harmful; what is harmful is people thinking they understand Unicode when in fact they don't. Jan 26, 2011 at 13:17
  • 2
    Does it have to be a cross-platform solution?
    – ali_bahoo
    Jan 26, 2011 at 13:18
  • 2
    @dalle c++ standard doesn't mention utf in any way (utf-8 or utf-16). Got a link where it says why utf-16 can't encoded with wstring? Jan 26, 2011 at 14:08

19 Answers 19

386

As Cubbi pointed out in one of the comments, std::wstring_convert (C++11) provides a neat simple solution (you need to #include <locale> and <codecvt>):

std::wstring string_to_convert;

//setup converter
using convert_type = std::codecvt_utf8<wchar_t>;
std::wstring_convert<convert_type, wchar_t> converter;

//use converter (.to_bytes: wstr->str, .from_bytes: str->wstr)
std::string converted_str = converter.to_bytes( string_to_convert );

I was using a combination of wcstombs and tedious allocation/deallocation of memory before I came across this.

http://en.cppreference.com/w/cpp/locale/wstring_convert

update(2013.11.28)

One liners can be stated as so (Thank you Guss for your comment):

std::wstring str = std::wstring_convert<std::codecvt_utf8<wchar_t>>().from_bytes("some string");

Wrapper functions can be stated as so: (Thank you ArmanSchwarz for your comment)

std::wstring s2ws(const std::string& str)
{
    using convert_typeX = std::codecvt_utf8<wchar_t>;
    std::wstring_convert<convert_typeX, wchar_t> converterX;

    return converterX.from_bytes(str);
}

std::string ws2s(const std::wstring& wstr)
{
    using convert_typeX = std::codecvt_utf8<wchar_t>;
    std::wstring_convert<convert_typeX, wchar_t> converterX;

    return converterX.to_bytes(wstr);
}

Note: there's some controversy on whether string/wstring should be passed in to functions as references or as literals (due to C++11 and compiler updates). I'll leave the decision to the person implementing, but it's worth knowing.

Note: I'm using std::codecvt_utf8 in the above code, but if you're not using UTF-8 you'll need to change that to the appropriate encoding you're using:

http://en.cppreference.com/w/cpp/header/codecvt

14
171

An older solution from: http://forums.devshed.com/c-programming-42/wstring-to-string-444006.html

std::wstring wide( L"Wide" ); 
std::string str( wide.begin(), wide.end() );

// Will print no problemo!
std::cout << str << std::endl;

Update (2021): However, at least on more recent versions of MSVC, this may generate a wchar_t to char truncation warning. The warning can be quieted by using std::transform instead with explicit conversion in the transformation function, e.g.:

std::wstring wide( L"Wide" );

std::string str;
std::transform(wide.begin(), wide.end(), std::back_inserter(str), [] (wchar_t c) {
    return (char)c;
});

Or if you prefer to preallocate and not use back_inserter:

std::string str(wide.length(), 0);
std::transform(wide.begin(), wide.end(), str.begin(), [] (wchar_t c) {
    return (char)c;
});

See example on various compilers here.


Beware that there is no character set conversion going on here at all. What this does is simply to assign each iterated wchar_t to a char - a truncating conversion. It uses the std::string c'tor:

template< class InputIt >
basic_string( InputIt first, InputIt last,
              const Allocator& alloc = Allocator() );

As stated in comments:

values 0-127 are identical in virtually every encoding, so truncating values that are all less than 127 results in the same text. Put in a chinese character and you'll see the failure.

the values 128-255 of windows codepage 1252 (the Windows English default) and the values 128-255 of unicode are mostly the same, so if that's teh codepage you're using most of those characters should be truncated to the correct values. (I totally expected á and õ to work, I know our code at work relies on this for é, which I will soon fix)

And note that code points in the range 0x80 - 0x9F in Win1252 will not work. This includes , œ, ž, Ÿ, ...

13
  • 2
    Bizarrely, this works on Visual Studio 10. What is going on? This should cause a truncating assigment from wchar_t to char for all elements of the original string. Jan 4, 2013 at 17:41
  • 6
    ...when it goes to any non-latin characters.
    – JavaRunner
    May 31, 2013 at 21:14
  • 10
    @PedroLamarão: values 0-127 are identical in virtually every encoding, so truncating values that are all less than 127 results in the same text. Put in a chinese character and you'll see the failure. Sep 4, 2013 at 20:20
  • 3
    @PedroLamarão: the values 128-255 of windows codepage 1252 (the Windows English default) and the values 128-255 of unicode are mostly the same, so if that's teh codepage you're using most of those characters should be truncated to the correct values. (I totally expected á and õ to work, I know our code at work relies on this for é, which I will soon fix) Sep 5, 2013 at 16:30
  • 1
    This used to work for me until I upgraded to MSVC 2019 and v142 toolset - now it craps out with a warning (which I always treat as errors): warning C4244: 'argument': conversion from 'const wchar_t' to 'const _Elem', possible loss of data
    – YePhIcK
    Apr 14, 2019 at 13:26
38

Here is a worked-out solution based on the other suggestions:

#include <string>
#include <iostream>
#include <clocale>
#include <locale>
#include <vector>

int main() {
  std::setlocale(LC_ALL, "");
  const std::wstring ws = L"ħëłlö";
  const std::locale locale("");
  typedef std::codecvt<wchar_t, char, std::mbstate_t> converter_type;
  const converter_type& converter = std::use_facet<converter_type>(locale);
  std::vector<char> to(ws.length() * converter.max_length());
  std::mbstate_t state;
  const wchar_t* from_next;
  char* to_next;
  const converter_type::result result = converter.out(state, ws.data(), ws.data() + ws.length(), from_next, &to[0], &to[0] + to.size(), to_next);
  if (result == converter_type::ok or result == converter_type::noconv) {
    const std::string s(&to[0], to_next);
    std::cout <<"std::string =     "<<s<<std::endl;
  }
}

This will usually work for Linux, but will create problems on Windows.

11
  • @Phillip: which part of the code depend on the c-locale ? is the std::setlocale(LC_ALL, ""); really needed ?
    – smerlin
    Jan 26, 2011 at 14:44
  • 2
    using std::wcout.imbue(locale) should do the job aswell, and it has the benefit that it does not change any global state.
    – smerlin
    Jan 26, 2011 at 15:22
  • 33
    The std::wstring_convert from C++11 wraps up a lot of this noise.
    – Cubbi
    Sep 27, 2011 at 19:34
  • 8
    @Philipp, what do you mean "will create problems on Windows"? What kind of problems?
    – Gili
    Nov 23, 2011 at 21:45
  • 2
    The above code gives (as copied) gives me a *** glibc detected *** test: malloc(): smallbin double linked list corrupted: 0x000000000180ea30 *** on linux 64-bit (gcc 4.7.3). Anybody else experiencing this?
    – hogliux
    Nov 10, 2013 at 12:22
17

Default encoding on:

  • Windows UTF-16.
  • Linux UTF-8.
  • MacOS UTF-8.

My solution Steps, includes null chars \0 (avoid truncated). Without using functions on windows.h header:

  1. Add Macros to detect Platform. Windows/Linux and others
  1. Create function to convert std::wstring to std::string and inverse std::string to std::wstring
  1. Create function for print
  1. Print std::string/ std::wstring

Check RawString Literals. Raw String Suffix.

Linux Code. Print directly std::string using std::cout, Default Encoding on Linux is UTF-8, no need extra functions.

On Windows if you need to print unicode. We can use WriteConsole for print unicode chars from std::wstring.

Finally on Windows. You need a powerfull and complete view support for unicode chars in console. I recommend Windows Terminal

QA

  • Tested on Microsoft Visual Studio 2019 with VC++; std=c++17. (Windows Project)
  • Tested on repl.it using Clang compiler; std=c++17.

Q. Why you not use <codecvt> header functions and classes?.
A. Deprecate Removed or deprecated features impossible build on VC++, but no problems on g++. I prefer 0 warnings and headaches.

Q. std ::wstring is cross platform?
A. No. std::wstring uses wchar_t elements. On Windows wchar_t size is 2 bytes, each character is stored in UTF-16 units, if character is bigger than U+FFFF, the character is represented in two UTF-16 units(2 wchar_t elements) called surrogate pairs. On Linux wchar_t size is 4 bytes each character is stored in one wchar_t element, no needed surrogate pairs. Check Standard data types on UNIX, Linux, and Windowsl.

Q. std ::string is cross platform?
A. Yes. std::string uses char elements. char type is guaranted that is same byte size in most compilers. char type size is 1 byte. Check Standard data types on UNIX, Linux, and Windowsl.

Full example code


#include <iostream>
#include <set>
#include <string>
#include <locale>

// WINDOWS
#if (_WIN32)
#include <Windows.h>
#include <conio.h>
#define WINDOWS_PLATFORM 1
#define DLLCALL STDCALL
#define DLLIMPORT _declspec(dllimport)
#define DLLEXPORT _declspec(dllexport)
#define DLLPRIVATE
#define NOMINMAX

//EMSCRIPTEN
#elif defined(__EMSCRIPTEN__)
#include <emscripten/emscripten.h>
#include <emscripten/bind.h>
#include <unistd.h>
#include <termios.h>
#define EMSCRIPTEN_PLATFORM 1
#define DLLCALL
#define DLLIMPORT
#define DLLEXPORT __attribute__((visibility("default")))
#define DLLPRIVATE __attribute__((visibility("hidden")))

// LINUX - Ubuntu, Fedora, , Centos, Debian, RedHat
#elif (__LINUX__ || __gnu_linux__ || __linux__ || __linux || linux)
#define LINUX_PLATFORM 1
#include <unistd.h>
#include <termios.h>
#define DLLCALL CDECL
#define DLLIMPORT
#define DLLEXPORT __attribute__((visibility("default")))
#define DLLPRIVATE __attribute__((visibility("hidden")))
#define CoTaskMemAlloc(p) malloc(p)
#define CoTaskMemFree(p) free(p)

//ANDROID
#elif (__ANDROID__ || ANDROID)
#define ANDROID_PLATFORM 1
#define DLLCALL
#define DLLIMPORT
#define DLLEXPORT __attribute__((visibility("default")))
#define DLLPRIVATE __attribute__((visibility("hidden")))

//MACOS
#elif defined(__APPLE__)
#include <unistd.h>
#include <termios.h>
#define DLLCALL
#define DLLIMPORT
#define DLLEXPORT __attribute__((visibility("default")))
#define DLLPRIVATE __attribute__((visibility("hidden")))
#include "TargetConditionals.h"
#if TARGET_OS_IPHONE && TARGET_IPHONE_SIMULATOR
#define IOS_SIMULATOR_PLATFORM 1
#elif TARGET_OS_IPHONE
#define IOS_PLATFORM 1
#elif TARGET_OS_MAC
#define MACOS_PLATFORM 1
#else

#endif

#endif



typedef std::string String;
typedef std::wstring WString;

#define EMPTY_STRING u8""s
#define EMPTY_WSTRING L""s

using namespace std::literals::string_literals;

class Strings
{
public:
    static String WideStringToString(const WString& wstr)
    {
        if (wstr.empty())
        {
            return String();
        }
        size_t pos;
        size_t begin = 0;
        String ret;

#if WINDOWS_PLATFORM
        int size;
        pos = wstr.find(static_cast<wchar_t>(0), begin);
        while (pos != WString::npos && begin < wstr.length())
        {
            WString segment = WString(&wstr[begin], pos - begin);
            size = WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, &segment[0], segment.size(), NULL, 0, NULL, NULL);
            String converted = String(size, 0);
            WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, &segment[0], segment.size(), &converted[0], converted.size(), NULL, NULL);
            ret.append(converted);
            ret.append({ 0 });
            begin = pos + 1;
            pos = wstr.find(static_cast<wchar_t>(0), begin);
        }
        if (begin <= wstr.length())
        {
            WString segment = WString(&wstr[begin], wstr.length() - begin);
            size = WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, &segment[0], segment.size(), NULL, 0, NULL, NULL);
            String converted = String(size, 0);
            WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, &segment[0], segment.size(), &converted[0], converted.size(), NULL, NULL);
            ret.append(converted);
        }
#elif LINUX_PLATFORM || MACOS_PLATFORM || EMSCRIPTEN_PLATFORM
        size_t size;
        pos = wstr.find(static_cast<wchar_t>(0), begin);
        while (pos != WString::npos && begin < wstr.length())
        {
            WString segment = WString(&wstr[begin], pos - begin);
            size = wcstombs(nullptr, segment.c_str(), 0);
            String converted = String(size, 0);
            wcstombs(&converted[0], segment.c_str(), converted.size());
            ret.append(converted);
            ret.append({ 0 });
            begin = pos + 1;
            pos = wstr.find(static_cast<wchar_t>(0), begin);
        }
        if (begin <= wstr.length())
        {
            WString segment = WString(&wstr[begin], wstr.length() - begin);
            size = wcstombs(nullptr, segment.c_str(), 0);
            String converted = String(size, 0);
            wcstombs(&converted[0], segment.c_str(), converted.size());
            ret.append(converted);
        }
#else
        static_assert(false, "Unknown Platform");
#endif
        return ret;
    }

    static WString StringToWideString(const String& str)
    {
        if (str.empty())
        {
            return WString();
        }

        size_t pos;
        size_t begin = 0;
        WString ret;
#ifdef WINDOWS_PLATFORM
        int size = 0;
        pos = str.find(static_cast<char>(0), begin);
        while (pos != std::string::npos) {
            std::string segment = std::string(&str[begin], pos - begin);
            std::wstring converted = std::wstring(segment.size() + 1, 0);
            size = MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, &segment[0], segment.size(), &converted[0], converted.length());
            converted.resize(size);
            ret.append(converted);
            ret.append({ 0 });
            begin = pos + 1;
            pos = str.find(static_cast<char>(0), begin);
        }
        if (begin < str.length()) {
            std::string segment = std::string(&str[begin], str.length() - begin);
            std::wstring converted = std::wstring(segment.size() + 1, 0);
            size = MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, segment.c_str(), segment.size(), &converted[0], converted.length());
            converted.resize(size);
            ret.append(converted);
        }

#elif LINUX_PLATFORM || MACOS_PLATFORM || EMSCRIPTEN_PLATFORM
        size_t size;
        pos = str.find(static_cast<char>(0), begin);
        while (pos != String::npos)
        {
            String segment = String(&str[begin], pos - begin);
            WString converted = WString(segment.size(), 0);
            size = mbstowcs(&converted[0], &segment[0], converted.size());
            converted.resize(size);
            ret.append(converted);
            ret.append({ 0 });
            begin = pos + 1;
            pos = str.find(static_cast<char>(0), begin);
        }
        if (begin < str.length())
        {
            String segment = String(&str[begin], str.length() - begin);
            WString converted = WString(segment.size(), 0);
            size = mbstowcs(&converted[0], &segment[0], converted.size());
            converted.resize(size);
            ret.append(converted);
        }
#else
        static_assert(false, "Unknown Platform");
#endif
        return ret;
    }
};

enum class ConsoleTextStyle
{
    DEFAULT = 0,
    BOLD = 1,
    FAINT = 2,
    ITALIC = 3,
    UNDERLINE = 4,
    SLOW_BLINK = 5,
    RAPID_BLINK = 6,
    REVERSE = 7,
};

enum class ConsoleForeground
{
    DEFAULT = 39,
    BLACK = 30,
    DARK_RED = 31,
    DARK_GREEN = 32,
    DARK_YELLOW = 33,
    DARK_BLUE = 34,
    DARK_MAGENTA = 35,
    DARK_CYAN = 36,
    GRAY = 37,
    DARK_GRAY = 90,
    RED = 91,
    GREEN = 92,
    YELLOW = 93,
    BLUE = 94,
    MAGENTA = 95,
    CYAN = 96,
    WHITE = 97
};

enum class ConsoleBackground
{
    DEFAULT = 49,
    BLACK = 40,
    DARK_RED = 41,
    DARK_GREEN = 42,
    DARK_YELLOW = 43,
    DARK_BLUE = 44,
    DARK_MAGENTA = 45,
    DARK_CYAN = 46,
    GRAY = 47,
    DARK_GRAY = 100,
    RED = 101,
    GREEN = 102,
    YELLOW = 103,
    BLUE = 104,
    MAGENTA = 105,
    CYAN = 106,
    WHITE = 107
};

class Console
{
private:
    static void EnableVirtualTermimalProcessing()
    {
#if defined WINDOWS_PLATFORM
        HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE);
        DWORD dwMode = 0;
        GetConsoleMode(hOut, &dwMode);
        if (!(dwMode & ENABLE_VIRTUAL_TERMINAL_PROCESSING))
        {
            dwMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
            SetConsoleMode(hOut, dwMode);
        }
#endif
    }

    static void ResetTerminalFormat()
    {
        std::cout << u8"\033[0m";
    }

    static void SetVirtualTerminalFormat(ConsoleForeground foreground, ConsoleBackground background, std::set<ConsoleTextStyle> styles)
    {
        String format = u8"\033[";
        format.append(std::to_string(static_cast<int>(foreground)));
        format.append(u8";");
        format.append(std::to_string(static_cast<int>(background)));
        if (styles.size() > 0)
        {
            for (auto it = styles.begin(); it != styles.end(); ++it)
            {
                format.append(u8";");
                format.append(std::to_string(static_cast<int>(*it)));
            }
        }
        format.append(u8"m");
        std::cout << format;
    }
public:
    static void Clear()
    {

#ifdef WINDOWS_PLATFORM
        std::system(u8"cls");
#elif LINUX_PLATFORM || defined MACOS_PLATFORM
        std::system(u8"clear");
#elif EMSCRIPTEN_PLATFORM
        emscripten::val::global()["console"].call<void>(u8"clear");
#else
        static_assert(false, "Unknown Platform");
#endif
    }

    static void Write(const String& s, ConsoleForeground foreground = ConsoleForeground::DEFAULT, ConsoleBackground background = ConsoleBackground::DEFAULT, std::set<ConsoleTextStyle> styles = {})
    {
#ifndef EMSCRIPTEN_PLATFORM
        EnableVirtualTermimalProcessing();
        SetVirtualTerminalFormat(foreground, background, styles);
#endif
        String str = s;
#ifdef WINDOWS_PLATFORM
        WString unicode = Strings::StringToWideString(str);
        WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), unicode.c_str(), static_cast<DWORD>(unicode.length()), nullptr, nullptr);
#elif defined LINUX_PLATFORM || defined MACOS_PLATFORM || EMSCRIPTEN_PLATFORM
        std::cout << str;
#else
        static_assert(false, "Unknown Platform");
#endif

#ifndef EMSCRIPTEN_PLATFORM
        ResetTerminalFormat();
#endif
    }

    static void WriteLine(const String& s, ConsoleForeground foreground = ConsoleForeground::DEFAULT, ConsoleBackground background = ConsoleBackground::DEFAULT, std::set<ConsoleTextStyle> styles = {})
    {
        Write(s, foreground, background, styles);
        std::cout << std::endl;
    }

    static void Write(const WString& s, ConsoleForeground foreground = ConsoleForeground::DEFAULT, ConsoleBackground background = ConsoleBackground::DEFAULT, std::set<ConsoleTextStyle> styles = {})
    {
#ifndef EMSCRIPTEN_PLATFORM
        EnableVirtualTermimalProcessing();
        SetVirtualTerminalFormat(foreground, background, styles);
#endif
        WString str = s;

#ifdef WINDOWS_PLATFORM
        WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), str.c_str(), static_cast<DWORD>(str.length()), nullptr, nullptr);
#elif LINUX_PLATFORM || MACOS_PLATFORM || EMSCRIPTEN_PLATFORM
        std::cout << Strings::WideStringToString(str);
#else
        static_assert(false, "Unknown Platform");
#endif

#ifndef EMSCRIPTEN_PLATFORM
        ResetTerminalFormat();
#endif
    }

    static void WriteLine(const WString& s, ConsoleForeground foreground = ConsoleForeground::DEFAULT, ConsoleBackground background = ConsoleBackground::DEFAULT, std::set<ConsoleTextStyle> styles = {})
    {
        Write(s, foreground, background, styles);
        std::cout << std::endl;
    }

    static void WriteLine()
    {
        std::cout << std::endl;
    }

    static void Pause()
    {
        char c;
        do
        {
            c = getchar();
            std::cout << "Press Key " << std::endl;
        } while (c != 64);
        std::cout << "KeyPressed" << std::endl;
    }

    static int PauseAny(bool printWhenPressed = false, ConsoleForeground foreground = ConsoleForeground::DEFAULT, ConsoleBackground background = ConsoleBackground::DEFAULT, std::set<ConsoleTextStyle> styles = {})
    {
        int ch;
#ifdef WINDOWS_PLATFORM
        ch = _getch();
#elif LINUX_PLATFORM || MACOS_PLATFORM || EMSCRIPTEN_PLATFORM
        struct termios oldt, newt;
        tcgetattr(STDIN_FILENO, &oldt);
        newt = oldt;
        newt.c_lflag &= ~(ICANON | ECHO);
        tcsetattr(STDIN_FILENO, TCSANOW, &newt);
        ch = getchar();
        tcsetattr(STDIN_FILENO, TCSANOW, &oldt);
#else
        static_assert(false, "Unknown Platform");
#endif
        if (printWhenPressed)
        {
            Console::Write(String(1, ch), foreground, background, styles);
        }
        return ch;
    }
};



int main()
{
    std::locale::global(std::locale(u8"en_US.UTF8"));
    auto str = u8"🐶\0Hello\0🐶123456789也不是可运行的程序123456789日本"s;//
    WString wstr = L"🐶\0Hello\0🐶123456789也不是可运行的程序123456789日本"s;
    WString wstrResult = Strings::StringToWideString(str);
    String strResult = Strings::WideStringToString(wstr);
    bool equals1 = wstr == wstrResult;
    bool equals2 = str == strResult;

    Console::WriteLine(u8"█ Converted Strings printed with Console::WriteLine"s, ConsoleForeground::GREEN);
    Console::WriteLine(wstrResult, ConsoleForeground::BLUE);//Printed OK on Windows/Linux.
    Console::WriteLine(strResult, ConsoleForeground::BLUE);//Printed OK on Windows/Linux.
    
    Console::WriteLine(u8"█ Converted Strings printed with std::cout/std::wcout"s, ConsoleForeground::GREEN);
    std::cout << strResult << std::endl;//Printed OK on Linux. BAD on Windows.
    std::wcout << wstrResult << std::endl; //Printed BAD on Windows/Linux.
    Console::WriteLine();
    Console::WriteLine(u8"Press any key to exit"s, ConsoleForeground::DARK_GRAY);
    Console::PauseAny();

}

You cant test this code on https://repl.it/@JomaCorpFX/StringToWideStringToString#main.cpp


**Screenshots**

Using Windows Terminal WindowsTerminal

Using cmd/powershell enter image description here

Repl.it capture
enter image description here

0
13

Instead of including locale and all that fancy stuff, if you know for FACT your string is convertible just do this:

#include <iostream>
#include <string>

using namespace std;

int main()
{
  wstring w(L"bla");
  string result;
  for(char x : w)
    result += x;

  cout << result << '\n';
}

Live example here

5
  • 2
    +1 because it's a simple solution that works for some scenarios (for a loose definition of "works", I might add).
    – raven
    Aug 14, 2012 at 9:43
  • 2
    Almost the same thing as namar0x0309's solution, which is much more elegant IMHO. But that's just me.
    – onitake
    Jun 17, 2013 at 13:57
  • I spiffied up your code to actually work with minimal modification ;-)
    – rubenvb
    Aug 22, 2013 at 8:05
  • 12
    -1 If you have a wstring, it's likely you're dealing with multibyte characters. If you could know the string is trivially convertible, you wouldn't be handling a wstring in the first place. More likely, you're dealing with another library that expects you to handle the wstring properly. Truncating the wchars is just begging for a hard to trace bug later on. Also, you should use "string result( w.begin(), w.end() );" if you were going to do it, to avoid a loop that could trigger many reallocations.
    – Kian
    May 22, 2014 at 19:35
  • @Kian, Plain and simple, and perfect for some limited use cases.
    – Devolus
    Feb 14, 2021 at 16:17
8

I believe the official way is still to go thorugh codecvt facets (you need some sort of locale-aware translation), as in

resultCode = use_facet<codecvt<char, wchar_t, ConversionState> >(locale).
  in(stateVar, scratchbuffer, scratchbufferEnd, from, to, toLimit, curPtr);

or something like that, I don't have working code lying around. But I'm not sure how many people these days use that machinery and how many simply ask for pointers to memory and let ICU or some other library handle the gory details.

0
8

There are two issues with the code:

  1. The conversion in const std::string s( ws.begin(), ws.end() ); is not required to correctly map the wide characters to their narrow counterpart. Most likely, each wide character will just be typecast to char.
    The resolution to this problem is already given in the answer by kem and involves the narrow function of the locale's ctype facet.

  2. You are writing output to both std::cout and std::wcout in the same program. Both cout and wcout are associated with the same stream (stdout) and the results of using the same stream both as a byte-oriented stream (as cout does) and a wide-oriented stream (as wcout does) are not defined.
    The best option is to avoid mixing narrow and wide output to the same (underlying) stream. For stdout/cout/wcout, you can try switching the orientation of stdout when switching between wide and narrow output (or vice versa):

    #include <iostream>
    #include <stdio.h>
    #include <wchar.h>
    
    int main() {
        std::cout << "narrow" << std::endl;
        fwide(stdout, 1); // switch to wide
        std::wcout << L"wide" << std::endl;
        fwide(stdout, -1); // switch to narrow
        std::cout << "narrow" << std::endl;
        fwide(stdout, 1); // switch to wide
        std::wcout << L"wide" << std::endl;
    }
    
1
  • Yes, that fixes the problem with using cout and wcout. Jan 26, 2011 at 16:21
8

Besides just converting the types, you should also be conscious about the string's actual format.

When compiling for Multi-byte Character set Visual Studio and the Win API assumes UTF8 (Actually windows encoding which is Windows-28591 ).
When compiling for Unicode Character set Visual studio and the Win API assumes UTF16.

So, you must convert the string from UTF16 to UTF8 format as well, and not just convert to std::string.
This will become necessary when working with multi-character formats like some non-latin languages.

The idea is to decide that std::wstring always represents UTF16.
And std::string always represents UTF8.

This isn't enforced by the compiler, it's more of a good policy to have. Note the string prefixes I use to define UTF16 (L) and UTF8 (u8).

To convert between the 2 types, you should use: std::codecvt_utf8_utf16< wchar_t>

#include <string>

#include <codecvt>

int main()
{

    std::string original8 = u8"הלו";

    std::wstring original16 = L"הלו";

    //C++11 format converter
    std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> convert;

    //convert to UTF8 and std::string
    std::string utf8NativeString = convert.to_bytes(original16);

    std::wstring utf16NativeString = convert.from_bytes(original8);

    assert(utf8NativeString == original8);
    assert(utf16NativeString == original16);

    return 0;
}
7

At the time of writing this answer, the number one google search for "convert string wstring" would land you on this page. My answer shows how to convert string to wstring, although this is NOT the actual question, and I should probably delete this answer but that is considered bad form. You may want to jump to this StackOverflow answer, which is now higher ranked than this page.


Here's a way to combining string, wstring and mixed string constants to wstring. Use the wstringstream class.

#include <sstream>

std::string narrow = "narrow";
std::wstring wide = "wide";

std::wstringstream cls;
cls << " abc " << narrow.c_str() << L" def " << wide.c_str();
std::wstring total= cls.str();
3
  • 14
    This is not a wstring to string conversion
    – poitroae
    Sep 6, 2012 at 12:09
  • 1
    @Michael Can you please explain? What about this is incorrect? Your comment is not helpful without more details.
    – Nate
    Oct 3, 2012 at 20:48
  • 1
    this is a string to wstring conversion. i.e. the opposite of the question. Feb 1, 2017 at 22:24
7

If you are dealing with file paths (as I often am when I find the need for wstring-to-string) you can use filesystem::path (since C++17):

#include <filesystem>

const std::wstring wPath = GetPath(); // some function that returns wstring
const std::string path = std::filesystem::path(wPath).string();
1
6

You might as well just use the ctype facet's narrow method directly:

#include <clocale>
#include <locale>
#include <string>
#include <vector>

inline std::string narrow(std::wstring const& text)
{
    std::locale const loc("");
    wchar_t const* from = text.c_str();
    std::size_t const len = text.size();
    std::vector<char> buffer(len + 1);
    std::use_facet<std::ctype<wchar_t> >(loc).narrow(from, from + len, '_', &buffer[0]);
    return std::string(&buffer[0], &buffer[len]);
}
6

This solution is inspired in dk123's solution, but uses a locale dependent codecvt facet. The result is in locale encoded string instead of UTF-8 (if it is not set as locale):

std::string w2s(const std::wstring &var)
{
   static std::locale loc("");
   auto &facet = std::use_facet<std::codecvt<wchar_t, char, std::mbstate_t>>(loc);
   return std::wstring_convert<std::remove_reference<decltype(facet)>::type, wchar_t>(&facet).to_bytes(var);
}

std::wstring s2w(const std::string &var)
{
   static std::locale loc("");
   auto &facet = std::use_facet<std::codecvt<wchar_t, char, std::mbstate_t>>(loc);
   return std::wstring_convert<std::remove_reference<decltype(facet)>::type, wchar_t>(&facet).from_bytes(var);
}

I was searching for it, but I can't find it. Finally I found that I can get the right facet from std::locale using the std::use_facet() function with the right typename. Hope this helps.

2
  • Vizor, what are the advantages (if any) of converting with the locale dependent facet?
    – Marc.2377
    Sep 12, 2019 at 23:33
  • If you work with strings from system, from console input for example.
    – Vizor
    Dec 10, 2019 at 17:31
6

I spent many sad days trying to come up with a way to do this for C++17, which deprecated code_cvt facets, and this is the best I was able to come up with by combining code from a few different sources:

setlocale( LC_ALL, "en_US.UTF-8" ); //Invoked in main()

std::string wideToMultiByte( std::wstring const & wideString )
{
     std::string ret;
     std::string buff( MB_CUR_MAX, '\0' );

     for ( wchar_t const & wc : wideString )
     {
         int mbCharLen = std::wctomb( &buff[ 0 ], wc );

         if ( mbCharLen < 1 ) { break; }

         for ( int i = 0; i < mbCharLen; ++i ) 
         { 
             ret += buff[ i ]; 
         }
     }

     return ret;
 }

 std::wstring multiByteToWide( std::string const & multiByteString )
 {
     std::wstring ws( multiByteString.size(), L' ' );
     ws.resize( 
         std::mbstowcs( &ws[ 0 ], 
             multiByteString.c_str(), 
             multiByteString.size() ) );

     return ws;
 }

I tested this code on Windows 10, and at least for my purposes, it seems to work fine. Please don't lynch me if this doesn't consider some crazy edge cases that you might need to handle, I'm sure someone with more experience can improve on this! :-)

Also, credit where it's due:

Adapted for wideToMultiByte()

Copied for multiByteToWide

3

In my case, I have to use multibyte character (MBCS), and I want to use std::string and std::wstring. And can't use c++11. So I use mbstowcs and wcstombs.

I make same function with using new, delete [], but it is slower then this.

This can help How to: Convert Between Various String Types

EDIT

However, in case of converting to wstring and source string is no alphabet and multi byte string, it's not working. So I change wcstombs to WideCharToMultiByte.

#include <string>

std::wstring get_wstr_from_sz(const char* psz)
{
    //I think it's enough to my case
    wchar_t buf[0x400];
    wchar_t *pbuf = buf;
    size_t len = strlen(psz) + 1;

    if (len >= sizeof(buf) / sizeof(wchar_t))
    {
        pbuf = L"error";
    }
    else
    {
        size_t converted;
        mbstowcs_s(&converted, buf, psz, _TRUNCATE);
    }

    return std::wstring(pbuf);
}

std::string get_string_from_wsz(const wchar_t* pwsz)
{
    char buf[0x400];
    char *pbuf = buf;
    size_t len = wcslen(pwsz)*2 + 1;

    if (len >= sizeof(buf))
    {
        pbuf = "error";
    }
    else
    {
        size_t converted;
        wcstombs_s(&converted, buf, pwsz, _TRUNCATE);
    }

    return std::string(pbuf);
}

EDIT to use 'MultiByteToWideChar' instead of 'wcstombs'

#include <Windows.h>
#include <boost/shared_ptr.hpp>
#include "string_util.h"

std::wstring get_wstring_from_sz(const char* psz)
{
    int res;
    wchar_t buf[0x400];
    wchar_t *pbuf = buf;
    boost::shared_ptr<wchar_t[]> shared_pbuf;

    res = MultiByteToWideChar(CP_ACP, 0, psz, -1, buf, sizeof(buf)/sizeof(wchar_t));

    if (0 == res && GetLastError() == ERROR_INSUFFICIENT_BUFFER)
    {
        res = MultiByteToWideChar(CP_ACP, 0, psz, -1, NULL, 0);

        shared_pbuf = boost::shared_ptr<wchar_t[]>(new wchar_t[res]);

        pbuf = shared_pbuf.get();

        res = MultiByteToWideChar(CP_ACP, 0, psz, -1, pbuf, res);
    }
    else if (0 == res)
    {
        pbuf = L"error";
    }

    return std::wstring(pbuf);
}

std::string get_string_from_wcs(const wchar_t* pcs)
{
    int res;
    char buf[0x400];
    char* pbuf = buf;
    boost::shared_ptr<char[]> shared_pbuf;

    res = WideCharToMultiByte(CP_ACP, 0, pcs, -1, buf, sizeof(buf), NULL, NULL);

    if (0 == res && GetLastError() == ERROR_INSUFFICIENT_BUFFER)
    {
        res = WideCharToMultiByte(CP_ACP, 0, pcs, -1, NULL, 0, NULL, NULL);

        shared_pbuf = boost::shared_ptr<char[]>(new char[res]);

        pbuf = shared_pbuf.get();

        res = WideCharToMultiByte(CP_ACP, 0, pcs, -1, pbuf, res, NULL, NULL);
    }
    else if (0 == res)
    {
        pbuf = "error";
    }

    return std::string(pbuf);
}
2
  • How can I use "wcstombs_s" with gcc 4.8 ? Because I see that is C++ 11 feature.
    – cristian
    Jun 9, 2016 at 15:07
  • @cristian You can use the "unsafe" version of this function wcstombs().
    – Vizor
    Nov 9, 2017 at 10:01
2
#include <boost/locale.hpp>
namespace lcv = boost::locale::conv;

inline std::wstring fromUTF8(const std::string& s)
{ return lcv::utf_to_utf<wchar_t>(s); }

inline std::string toUTF8(const std::wstring& ws)
{ return lcv::utf_to_utf<char>(ws); }
1
  • Very easy solution as always from BOOST.... however it cannot handle special symbols.
    – Mecanik
    Sep 5, 2022 at 4:08
1

In case anyone else is interested: I needed a class that could be used interchangeably wherever either a string or wstring was expected. The following class convertible_string, based on dk123's solution, can be initialized with either a string, char const*, wstring or wchar_t const* and can be assigned to by or implicitly converted to either a string or wstring (so can be passed into a functions that take either).

class convertible_string
{
public:
    // default ctor
    convertible_string()
    {}

    /* conversion ctors */
    convertible_string(std::string const& value) : value_(value)
    {}
    convertible_string(char const* val_array) : value_(val_array)
    {}
    convertible_string(std::wstring const& wvalue) : value_(ws2s(wvalue))
    {}
    convertible_string(wchar_t const* wval_array) : value_(ws2s(std::wstring(wval_array)))
    {}

    /* assignment operators */
    convertible_string& operator=(std::string const& value)
    {
        value_ = value;
        return *this;
    }
    convertible_string& operator=(std::wstring const& wvalue)
    {
        value_ = ws2s(wvalue);
        return *this;
    }

    /* implicit conversion operators */
    operator std::string() const { return value_; }
    operator std::wstring() const { return s2ws(value_); }
private:
    std::string value_;
};
1
  • 1
    I would rather store an std::wstring in the class, than to store std::string and do a conversion to std::wstring when needed to get an std::wstring. Because std::wstring is somewhat faster than std::string and it's better compatible. Even it consumes more memory than std::string.
    – 0xAA55
    May 7, 2020 at 13:21
0
std::string
convert_str(const std::wstring &s) {
  //not determinate how many bytes to place all wide characters.
  std::string res(2 * s.size(), '\0');
  while(1) {
    size_t num_used_char = std::wcstombs(res.data(), s.data(), res.size());
    if (num_used_char == (size_t) -1) {
      //do your error handling.
    } 
    else if (num_used_char < res.size()) {
      res.resize(num_used_char);
      break;
    }
    res.resize(2 * res.size());
  }
  return res;
}
-1

I am using below to convert wstring to string.

std::string strTo;
char *szTo = new char[someParam.length() + 1];
szTo[someParam.size()] = '\0';
WideCharToMultiByte(CP_ACP, 0, someParam.c_str(), -1, szTo, (int)someParam.length(), NULL, NULL);
strTo = szTo;
delete szTo;
1
  • You seem to be missing a standard header (<string>) and a definition for WideCharToMultiByte() - is that some wrapper around std::wctomb()? Apr 5, 2018 at 13:39
-3
// Embarcadero C++ Builder 

// convertion string to wstring
string str1 = "hello";
String str2 = str1;         // typedef UnicodeString String;   -> str2 contains now u"hello";

// convertion wstring to string
String str2 = u"hello";
string str1 = UTF8string(str2).c_str();   // -> str1 contains now "hello"
2
  • 3
    please explain what ure doing there in your answer, else it may get deletet Nov 11, 2014 at 12:32
  • 1
    Where does UTF8string function come from ? Mar 16, 2016 at 20:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.