29

I am trying to read a file in as either UTF-8 or Windows-1252 depending on the output of this method:

public Charset getCorrectCharsetToApply() {
    // Returns a Charset for either UTF-8 or Windows-1252.
}

So far, I have:

String fileName = getFileNameToReadFromUserInput();
InputStream is = new ByteArrayInputStream(fileName.getBytes());
InputStreamReader isr = new InputStreamReader(is, getCorrectCharsetToApply());
BufferedReader buffReader = new BufferedReader(isr);

The problem I'm having is converting the BufferedReader instance to a FileReader.

Furthermore:

  • The name of the file itself (fileName) cannot be trusted to be a particular Charset; sometime the file name will contain UTF-8 characters, and sometimes Windows-1252. Same goes for the file's content (however if file name and file content will always have matching charsets).
  • Only the logic inside getCorrectCharsetToApply() can select the charset to apply, so attempting to read a file by its name prior to calling this method could very well result with, Java trying to read the file name with the wrong encoding...which causes it to die!

Thanks in advance!

4 Answers 4

35

So, first, as a heads up, do realize that fileName.getBytes() as you have there gets the bytes of the filename, not the file itself.

Second, reading inside the docs of FileReader:

The constructors of this class assume that the default character encoding and the default byte-buffer size are appropriate. To specify these values yourself, construct an InputStreamReader on a FileInputStream.

So, sounds like FileReader actually isn't the way to go. If we take the advice in the docs, then you should just change your code to have:

String fileName = getFileNameToReadFromUserInput();
FileInputStream is = new FileInputStream(fileName);
InputStreamReader isr = new InputStreamReader(is, getCorrectCharsetToApply());
BufferedReader buffReader = new BufferedReader(isr);

and not try to make a FileReader at all.

6
  • Thanks @Dennis Meng (+1) - I'm comfortable going with the FileInputStream/FileReader as you recommend, but then how do I use those objects in conjunction with specifying the Charset (UTF-8 vs Windows-1252)? Thanks again! Aug 23, 2012 at 17:31
  • 1
    So, the idea that I got from the docs is to create the FileInputStream, and then wrap the InputStreamReader around it. Judging from what you had, you already knew to specify the encoding in the constructor for InputStreamReader, which is why my suggestion only changes the line above it. Aug 23, 2012 at 17:33
  • Thanks again (+1)! I'm not trying to be difficult here, but I'm still not connecting the dots. You say to "wrap the FileInputStream around the InputStreamReader, so to me that means something like FileInputStream fis = new FileInputStream(myInputStreamReader), where myInputStreamReader is the InputStreamReader that is already set with the correct charset. However, after looking at thee Java 6 docs for FileInputStream, I see no constructors that accept an InputStreamReader as an arg. Can you update with a code example of what you mean? Thanks again! Aug 23, 2012 at 17:37
  • 1
    Alright, I'll modify my answer to contain the full four lines then. Aug 23, 2012 at 17:39
  • Ahhh, I'm terribly sorry - I misread your comment! I had the FileInputStream/InputStreamReader wrapping inverted. Thanks again! Aug 23, 2012 at 17:43
6

With Java 7+, you can create the Reader in one line:

BufferedReader buffReader = Files.newBufferedReader(Paths.get(fileName), getCorrectCharsetToApply());

4

Note that if you are using Google Guava, you can use Files.newReader:

final BufferedReader reader =
        Files.newReader(new File(filename), getCorrectCharsetToApply());
1
  • 2
    Also, since using Guava, you can choose your charset from their Charsets constant.
    – Papsicle
    Jul 15, 2015 at 20:07
0

Java does not die if you pick the wrong charset. The code is perfectly fine like this, but you may want to read the file content instead of the file name:

String fileName = getFileNameToReadFromUserInput();
InputStream is = new ByteArrayInputStream(fileName.getBytes());
InputStreamReader isr = new InputStreamReader(is, getCorrectCharsetToApply());
BufferedReader buffReader = new BufferedReader(isr);

Just consider what exactly will happen if you pick the wrong encoding: Java will throw an Exception. All you need to do is handle this situation, then you can iterate over all the charsets Java knows about:

void processFile(String fileName, Charset charset) {
    try (
        InputStream is = new FileInputStream(fileName);
        InputStreamReader isr = new InputStreamReader(is, charset);
        BufferedReader buffReader = new BufferedReader(isr);
    ) {
        ... (do whatever you have to do here by reading buffReader)
    }
}

void main(String[]) {
    String fileName = getFileNameToReadFromUserInput();

    // try all the known charsets
    for (Charset charset: Charset.availableCharsets().values()) {
        try {
            processFile(fileName, charset);
            // still here? then it was successful
            return;
        } catch (Exception e) {
            System.out.println("Reading " + fileName + " with " + charset + " resulted in");
            e.printStackTrace();
            System.out.println("Trying next...");
        }
    }
}

The try-with-resources block inside processFile ensures all files/streams/buffers get closed when they are no longer needed. The try/catch block in the main method ensures the program does not die in case we picked the wrong charset.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.