83

I have a website, and I can send my Turkish characters with jQuery in Firefox, but Internet Explorer doesn't send my Turkish characters. I looked at my source file in notepad, and this file's code page is ANSI.

When I convert it to UTF-8 without BOM and close the file, the file is again ANSI when I reopen.

How can I convert my file from ANSI to UTF-8?

2
  • 1
    You can use the tool I wrote for that, I also suffered from same problem and made my own way out. github.com/srcnalt/ANSI-to-UTF8 Commented Feb 3, 2014 at 8:33
  • I agree that – in a strict sense – this question is not on-topic for Stack Overflow. ~ * ~ But it's very much ON-topic for Super User. After more than 9 years (!), it still hasn't been migrated to Super User. Such a pity. Commented Sep 28, 2022 at 11:20

3 Answers 3

80

Regarding this part:

When I convert it to UTF-8 without bom and close file, the file is again ANSI when I reopen.

The easiest solution is to avoid the problem entirely by properly configuring Notepad++.

Try Settings -> Preferences -> New document -> Encoding -> choose UTF-8 without BOM, and check Apply to opened ANSI files.

notepad++ UTF-8 apply to opened ANSI files

That way all the opened ANSI files will be treated as UTF-8 without BOM.

For explanation what's going on, read the comments below this answer.

To fully learn about Unicode and UTF-8, read this excellent article from Joel Spolsky.

Sign up to request clarification or add additional context in comments.

7 Comments

That helped me a lot. Thanks. I do not understand the behavior though. Because I open an existing file and not a new one.
The Apply to opened ANSI files is relevant in your situation: when you have a file that contains only plain ASCII characters (without accents etc.), and you don't have BOM at the beginning of the file, then the editor by default treats it as an ANSI file, because there's nothing in this file to indicate that you might want to treat is an UTF-8 file. However when you add, say, Ö and save it as UTF-8 w/o BOM, even though there's no BOM at the beginning of the file, from the presence of two-byte combo behind Ö (0xC396 in this case) the editor learns "this has to be UTF-8".
In other words, when you save ANSI plain file as UTF-8, the output is identical as you were to save it as ANSI. You have to tell the editor to treat it as UTF-8 when you open it. For the file to be UTF-8, it either has to start with a BOM, or contain certain two-bytes sequences. The behavior of the editor when you input Ö in an ANSI file is configuration dependent.
AFAIK the only encoding that you can enforce from the editor on read time, by putting certain characters in the file, is UTF-8 with BOM.
@SanderdeJong works for me with 7.8.5 32-bit - I posted an image
|
47

Maybe this is not the answer you needed, but I encountered similar problem, so I decided to put it here.

I needed to convert 500 xml files to UTF8 via Notepad++. Why Notepad++? When I used the option "Encode in UTF8" (many other converters use the same logic) it messed up all special characters, so I had to use "Convert to UTF8" explicitly.


Here some simple steps to convert multiple files via Notepad++ without messing up with special characters (for ex. diacritical marks).

  1. Run Notepad++ and then open menu Plugins->Plugin Manager->Show Plugin Manager
  2. Install Python Script. When plugin is installed, restart the application.
  3. Choose menu Plugins->Python Script->New script.
  4. Choose its name, and then past the following code:

convertToUTF8.py

import os
import sys
from Npp import notepad # import it first!

filePathSrc="C:\\Users\\" # Path to the folder with files to convert
for root, dirs, files in os.walk(filePathSrc):
    for fn in files: 
        if fn[-4:] == '.xml': # Specify type of the files
            notepad.open(root + "\\" + fn)      
            notepad.runMenuCommand("Encoding", "Convert to UTF-8")
            # notepad.save()
            # if you try to save/replace the file, an annoying confirmation window would popup.
            notepad.saveAs("{}{}".format(fn[:-4], '_utf8.xml')) 
            notepad.close()

After all, run the script

8 Comments

Great solution. Since I use notepad++ localization I've had to translate 'Encoding' and 'Convert to UTF-8' options, weird.
I wonder how to run the python script? I run it in a command line and it says that notepad cannot be found.
Hi flexwang, you should run it from Notepad++
I got error meassage because of the Chinese characters. dropbox.com/s/f2efnzt9cd2i5or/…
doesnt work anymore :(
|
16

If you don't have non-ASCII characters (codepoints 128 and above) in your file, UTF-8 without BOM is the same as ASCII, byte for byte - so Notepad++ will guess wrong.

What you need to do is to specify the character encoding when serving the AJAX response - e.g. with PHP, you'd do this:

header('Content-Type: application/json; charset=utf-8');

The important part is to specify the charset with every JS response - else IE will fall back to user's system default encoding, which is wrong most of the time.

2 Comments

Why isn't this the accepted answer? It's the only answer that explains what is happening, and what the real solution to the question is.
so Notepad++ will guess wrong – Maybe just a matter of wording, but I wouldn't say that Notepad++ guesses it wrong. I'd rather say that – for a file containing only ASCII characters – encoding as ANSI or UTF-8 is equally correct. And the only way to make Notepad++ choose UTF-8 over ANSI is to make UTF-8 the default encoding (for pure ASCII files). This answer shows how to do just that.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.