1

I have a text in utf-8 and I want to decode it, using utf8_decode()
But when I do that I lose a part of the text, utf8_decode() decodes the string until it finds a character –
Any idea to solve this problem ?

3
  • 2
    utf8_decode converts from UTF-8 to ISO 8859-1. You can loose the characters that are not in ISO 8859-1.
    – Gumbo
    Nov 25, 2010 at 14:28
  • I know I can lose characters. But you find it normal that half the text after a particular character is lost though it does not contain special chars ?
    – Serty Oan
    Nov 25, 2010 at 14:31
  • With iconv I get this even with //IGNORE : Notice: iconv(): Detected an illegal character in input string Any idea ?
    – Serty Oan
    Nov 27, 2010 at 13:24

5 Answers 5

2

Maybe iconv can help you

0
2
†= E2 80 = 1110 0010 1000 0000

If that's literally what was in your UTF-8 text, then it might not be UTF-8. It would need to be followed by one more octet starting 10 to be valid.

That's because an octet starting 1110 introduces a three octet sequence, with the following octets starting 10, to deliver a total of 16 bytes of 'payload' to give the Unicode code point.

EDIT: You've provided the next char as 0x93 = 1001 0011 which would be valid. The UTF-8 sequence 0xE28093 = 0010 00 0000 01 0011 = 0x2013 which is an EN DASH. So, it looks like plausible UTF-8 after all!

2
  • S***... Problem may come from the way data were saved (I try to import from another database which stored utf8 encoded strings in utf8_general_ci tables but not using MySQL SET NAMES 'utf8' on connection, when I look via phpmyadmin I see the characters like this : équipes, all goes well until I found this case with – sequence...)
    – Serty Oan
    Nov 25, 2010 at 14:39
  • To precise, what I have in the text is : –
    – Serty Oan
    Nov 25, 2010 at 14:47
1

Perhaps – are not in ISO-8859-1? utf8_decode eats only utf8-characters which also exist in ISO-8859-1.

1

You'll probably want something similar to this:

$string = iconv("UTF-8", "ISO-8859-1//TRANSLIT", $string);

You can read more about iconv in the documentation. Depending on your use, IGNORE might be more useful than TRANSLIT.

1
  • Accoding to the comments in the documentation an option of //TRANSLIT//IGNORE is also possible, and for some of people solved the problem. I haven't tested it myself, but might be worth trying. Nov 25, 2010 at 14:48
1

Are you sure that EdoDodo's code is not working?

Try to force the browser to handle the output as iso-8859-1. To do this, you need an utf8 encoded file with the string in it (you need this, because text editors may use an invisible UTF-8 BOM, and the browser may switch to UTF-8 against the defined ISO-8859-2), and an other one with the php code in ansi encoding (I am using Notepad++ just to be sure that the encoding is proper - it detects the file's encoding and shows it in the lower right corner, and you can convert between the encodings too).

So create a file in utf-8 encoding called utf8.txt with just the string:

–

And create an ANSI encoded index.php file with this content:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> 
<html> 
<head> 
 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> 
</head> 

<body>

<?php
$str = file_get_contents('utf8.txt');

echo "iconv(//IGNORE//TRANSLIT): " . iconv("UTF-8", "ISO-8859-1//IGNORE//TRANSLIT", $str) . "<br>\n";

For webpages, I strongly recommend to always use UTF-8 encoding, even if it is in English.

6
  • @Serty You can try iconv("UTF-8", "UTF-8//IGNORE", $str); to filter out bad chars. See this article for further info. Nov 26, 2010 at 8:29
  • I just tried this. And it does not remove the problematic characters. And a decoding after cuts the sentence again :/
    – Serty Oan
    Nov 26, 2010 at 9:14
  • @Serty I have successfully replicated the error on an other PHP installation, so edited the answer with the possible solution. Nov 26, 2010 at 9:18
  • @Serty just to notice that this is not the one that EdoEdo suggested on his answer's comment and which you tried before, because that was //TRANSLIT//IGNORE, and this one is //IGNORE//TRANSLIT, and it works for me this way. Nov 26, 2010 at 9:30
  • Yes, it's strange, because my first answer worked on my 5.2.x installation, but not on 5.3.3, and now I tried //TRANSLIT//IGNORE, and it is working on 5.3.3 too (and //IGNORE//TRANSLIT as well). According to phpinfo(), my libiconv version is 1.11, and the iconv.input_encoding, iconv.internal_encoding and iconv.output_encoding are all set to ISO-8859-1. Nov 26, 2010 at 9:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.