7

i have a few directories/files with Japanese characters. If i try to read a filename (not the contents) containing (as example) a ク i receive a String containing a �. If i try to create a file/directory containing an ク a file/directory appears containing a ?.

As example: I list the files with.

File file = new File(".");  
String[] filesAndDirs = file.list();

the filesAndDirs array now contains the directories this the special characters. The String now only contains ����. It seams there is nothing to decode because the a getbytes shows only "-17 -65 -67" for every char in the filename even for different chars.

I use MacOS 10.8.2 Java 7_10 and Netbeans.

Any ideas?

Thank You in advance :)

4
  • 2
    It's not clear how you're showing those filenames or whether you're providing an encoding for String.getBytes() (which you always should). You should dump out the UTF-16 code point for each character in the filename, to see what's really going on. Also, it's not clear how you were getting the input data when trying to create the file.
    – Jon Skeet
    Commented Jan 5, 2013 at 12:05
  • getBytes returns what seems to be valid UTF8 Commented Jan 5, 2013 at 12:12
  • 1
    @Jan Yes it seems like valid UT8 but getBytes returns "-17 -65 -67" for every char. But not all chars in the file/directory name are the same. It seems as if i loose all information between the OS and the JavaVM. "-17 -65 -67" is repeated for every Special char. [a-zA-Z ...] are returned as expected. I would at least expect different bytes for each char.
    – uti.devel
    Commented Jan 5, 2013 at 12:36
  • Perhaps it's valid UTF8 for �? Commented Jan 5, 2013 at 13:17

2 Answers 2

4

Those bytes are 0xef 0xbf 0xbd, which is the UTF-8-encoded form of the \ufffd character you're seeing instead of the Japanese characters. It appears whatever OS function Java is using to list the files is in fact returning those incorrect characters.

Perhaps Files.newDirectoryStream will be more reliable. Try this instead:

try (DirectoryStream<Path> dir = Files.newDirectoryStream(Paths.get("."))) {
    for (Path child : dir) {
        String filename = child.getFileName().toString();

        System.out.println("name=" + filename);
        for (char c : filename.toCharArray()) {
            System.out.printf("%04x ", (int) c);
        }
        System.out.println();
    }
}
2
  • 1
    working like a charm :). Now i only have to find out how to create files and folders with special chars. Thank You very much!
    – uti.devel
    Commented Jan 5, 2013 at 12:53
  • Have a look at the javadoc for the Files class. It has methods to do all of those things.
    – VGR
    Commented Jan 5, 2013 at 16:56
2

It's a bug in the old java File api (maybe just on a mac). Anyway, it's all fixed in the new java.nio.

I have several files containing unicode characters in the filename and content that failed to load using java.io.File and related classes. After converting all my code to use java.nio.Path EVERYTHING started working. And I replaced org.apache.commons.io.FileUtils (which has the same problem) with java.nio.Files...

...and be sure to read and write the content of file using an appropriate charset, for example: Files.readAllLines(myPath, StandardCharsets.UTF_8)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.