0

It looks lengthy, but it is easier to read. Please keep patience.

Here is a simple C++ program, from the book C++ the complete reference: 4th edition (chapter 21, page 543), which writes a text file:

#include <iostream>
#include <fstream>
using namespace std;

int main()
{
  ofstream out("INVNTRY.txt"); // output, normal file

  if(!out) {
    cout << "Cannot open INVENTORY file.\n";
    return 1;
  }

  out << "Radios " << 39.95 << endl;
  out << "Toasters " << 19.95 << endl;
  out << "Mixers " << 24.80 << endl;

  out.close();
  system("pause");
  return 0;
}

and another program that reads the data just written:

#include <iostream>
#include <fstream>
using namespace std;

int main()
{
  ifstream in("INVNTRY.txt"); // input

  if(!in) {
    cout << "Cannot open INVENTORY file.\n";
    return 1;
  }

  char item[20];
  float cost;

  in >> item >>  cost;
  cout << item << " " << cost << "\n";
  in >> item >> cost;
  cout << item << " " << cost << "\n";
  in >> item >> cost;
  cout << item << " " << cost << "\n";

  in.close();
  system("pause");
  return 0;
}

Everything is fine, till now. Whatever was written, is easily readable.

Now the problem is, I have written another text file "response.txt" using Visual Basic. I tried to read the file using the above program. But in console window i got " " at the very beginning!
I thought something was wrong with my "response.txt" file, I copied the content of "INVNTRY.txt" and pasted in "response.txt", but still got " " at the beginning. How is this possible. At this point of time, two files have the same content, same program was used to read them, still different output ??

I am trying to show some pictures for clarification:

enter image description here

But if I try to read "INVNTRY.txt", look the difference in output, there is no " " character at the beginning:

enter image description here

How is this possible ??

8
  • 8
    It's Byte Order Mark. Some text editors tend to leave them at the beginning of a file. Jun 3, 2014 at 19:18
  • 3
    Select "Save as..." from the Notepad menu, under Encoding (down next to the Save and Cancel buttons), select ANSI, then save the file and try again (cursing at MS while doing so is encouraged).
    – WhozCraig
    Jun 3, 2014 at 19:20
  • Thanks @WhozCraig...I wasted my whole day on this! I love this website, and I love all you guys, thanks again for helping me!
    – aniliitb10
    Jun 3, 2014 at 19:25
  • 2
    The BOM is added by Notepad when saving a file using any of the unicode encodings (Unicode, UTF-8 etc.).
    – Felix Glas
    Jun 3, 2014 at 19:25
  • 2
    If you're writing the fie with Visual Basic and all you want is the ASCII data (no BOM) make sure you open the StreamWriter with the proper encoding : System.Text.ASCIIEncoding. The default is System.Text.UTF8Encoding, and is responsible for your BOM woes.
    – WhozCraig
    Jun 3, 2014 at 19:37

1 Answer 1

3

As I already mentioned, it's the Byte Order Mark, which is only one of the issues in the twisted world of encodings.

There are too many complications in byte order marks, code pages, and encodings, and this means that:

You can't reliably process the file without knowing its encoding.

So what can you do about it?

  • ignore the problem. (probably not a good idea)
  • require specific encoding from user (simple enough - depending on your user)
  • support multiple encodings (this requires some input - metadata, user selection, whatever)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.