0

I'm trying to open a file which path has non-ascii characters. The user drags the file into the cmd and I get the path with getline. When I try to open the file using the path provided by the user it doesn't work.

#include <iostream>
#include <string>
#include <fstream>
#include <cstdlib>

using namespace std;

int main() {
    string userInput;   //Saves the user input
    string autoInput = "C:\\Espolón\\file.txt"; //Specifying the path like this works
    ifstream file1; //For opening the file with the userInput
    ifstream file2; //For opening the file with autoInput

    getline(cin, userInput);

    system("CLS");  //Clears the CMD

    file1.open(userInput);  //This throws an error. Note that I didn't use is_open for cleaner code but im actually using it in my tests
    file2.open(autoInput);  //This works perfectly


    cout << "User input: " + userInput << endl<<"Auto input: " + autoInput << endl; //Both show correctly in the CMD
    system("pause");
}

Although the cout shows everything correctly, while debugging I found out the userInput non-ascii character 'ó' is being changed with a '¢' ("C:\Espol¢n\file.txt") but the autoInput is correctly stored ("C:\Espolón\file.txt"): screen capture. Therefore, I can't open a file using the path provided by the user if it has special characters.

I tried using wide characters as I've read in other similar questions:

#include <iostream>
#include <string>
#include <fstream>
#include <cstdlib>

using namespace std;

int main() {
    wstring userInput;  //Saves the user input
    string autoInput = "C:\\Espolón\\file.txt"; //Specifying the path like this works
    ifstream file1; //For opening the file with the userInput
    ifstream file2; //For opening the file with a fix string

    getline(wcin, userInput);

    system("CLS");  //Clears the CMD

    file1.open(userInput);  //This throws an error. Note that I didn't use is_open for cleaner code but im actually using it in my tests
    file2.open(autoInput);  //This works perfectly


    wcout << L"User input: " + userInput << endl;
    cout<<"Auto input: " + autoInput << endl; //Both show correctly in the CMD
    system("pause");
}

But the issue persists. I also tried changing the encoding to Unicode but nothing changed.

Any help would be welcomed! (Sry for bad eglish)

6
  • Look at std::locale
    – n314159
    Commented Nov 18, 2019 at 14:53
  • Have you tried using wide character versions of std::string, cout and cin? Commented Nov 18, 2019 at 14:56
  • For me, your code works. (even the first version)
    – n314159
    Commented Nov 18, 2019 at 15:02
  • @ThomasMatthews Yes! Thats in the second code block.
    – matsrom
    Commented Nov 18, 2019 at 15:33
  • @n314159 Have you tried a path with non ascii characters? Which IDE are you using? I'm using Visual Studio Community 2019.
    – matsrom
    Commented Nov 18, 2019 at 15:36

1 Answer 1

0

Windows APIs expect Unicode in UTF16LE format, not UTF8. Wide string characters and functions are treated as UTF16 in Windows.

The letter ó is ANSI, not ASCII. Windows also supports ANSI. You can use ANSI generally if your program runs on your own computer, but even then, you can run in to problems. If you wish to use ANSI, then try not to mix it with UTF16 and wide string format (std::wstring)

Windows 8 and 10 do have limited console support for UTF8. But it is recommended to use UTF16 if you want to avoid the pitfalls with UTF8.

Here is UTF16 version:

#include <iostream>
#include <string>
#include <fstream>
#include <cstdlib>
#include <io.h> 
#include <fcntl.h> 

using namespace std;

int main()
{
    _setmode(_fileno(stdout), _O_U16TEXT);
    _setmode(_fileno(stdin), _O_U16TEXT);

    wstring userInput;  
    wstring autoInput = L"C:\\Espolón\\file.txt";
    ifstream file1, file2;

    getline(wcin, userInput);

    file1.open(userInput);
    file2.open(autoInput);
    wcout << L"file1: " << (file1.good() ? L"good\n" : L"bad\n");
    wcout << L"file2: " << (file1.good() ? L"good\n" : L"bad\n");

    wcout << L"User input: " + userInput << endl;
    wcout << L"Auto input: " + autoInput << endl;
    system("pause");

    return 0;
}


Unix based systems use UTF8. Note that these systems don't easily get along with UTF16. They use std::wstring is used for UTF32. For the most part you can simply use std::string for UTF8 in Linux, Mac, etc.
The text file may contain text in ASCII format, ANSI, UTF8, or UTF16. It is recommended to use UTF8 for compatibility.

In Windows, store the text file in UTF8 format. Use these functions to convert back and forth between UTF8 and UTF16:

//need at least c++11 for writable std::string::data()
std::string get_u8(const std::wstring u16)
{
    if(u16.empty()) return std::string();
    int size = WideCharToMultiByte(CP_UTF8, 0, u16.c_str(), -1, 0, 0, 0, 0);
    std::string u8(size, 0);
    WideCharToMultiByte(CP_UTF8, 0, u16.c_str(), -1, u8.data(), size, 0, 0);
    return u8;
}

std::wstring get_u16(const std::string u8)
{
    if(u8.empty()) return std::wstring();
    int size = MultiByteToWideChar(CP_UTF8, 0, u8.c_str(), -1, 0, 0);
    std::wstring u16(size, 0);
    MultiByteToWideChar(CP_UTF8, 0, u8.c_str(), -1, u16.data(), size);
    return u16;
}
2
  • It works! Thanks for your complete and informative answer :)
    – matsrom
    Commented Nov 19, 2019 at 8:13
  • You are welcome, but my answer is far from complete. This topic is a bit of a mess when it comes to Windows and Unicode. See updated answer and other resources. Commented Nov 19, 2019 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.