4

We're streaming a CSV file from a web service. It appears that we're losing the new line characters when streaming - the client gets the file all on a single line. Any idea what we're doing wrong?

Code:

 public static void writeFile(OutputStream out, File file) throws IOException {
    BufferedReader input = new BufferedReader(new FileReader(file)); //File input stream 
    String line;
    while ((line = input.readLine()) != null) { //Read file
        out.write(line.getBytes());  //Write to output stream 
        out.flush();
    }
    input.close();
} 

6 Answers 6

11

Don't use BufferedReader. You already have an OutputStream at hands, so just get an InputStream of the file and pipe the bytes from input to output it the usual Java IO way. This way you also don't need to worry about newlines being eaten by BufferedReader:

public static void writeFile(OutputStream output, File file) throws IOException {
    InputStream input = null;
    byte[] buffer = new byte[10240]; // 10KB.
    try {
        input = new FileInputStream(file);
        for (int length = 0; (length = input.read(buffer)) > 0;) {
            output.write(buffer, 0, length);
        }
    } finally {
        if (input != null) try { input.close(); } catch (IOException logOrIgnore) {}
    }
}

Using a Reader/Writer would involve character encoding problems if you don't know/specify the encoding beforehand. You actually also don't need to know about them here. So just leave it aside.

To improve performance a bit more, you can always wrap the InputStream and OutputStream in an BufferedInputStream and BufferedOutputStream respectively.

3
  • 10KB is kinda big and will inhibit scalability in a popular/much-used usage scenario. 1k-2k is just fine. And, yes. Use a BufferedOutputStream for the writing. Dec 30, 2009 at 18:43
  • @Stu: BufferedInputStream/BufferedOutputStream uses 8KB under the hoods and are configureable by 2nd constructor argument. I just use 10KB all the time, it's certainly enough with cheap RAM nowadays.
    – BalusC
    Dec 30, 2009 at 18:47
  • @BalusC: Is that not JVM implementation and/or Java version dependent? Anyway, yes, RAM is cheap. I guess the fact that my 'baby' (a heavy IO webapp with many users) has the need to keep memory down is influencing my perspective. A 5x or 10x improvement in that narrow aspect can be important for some, like myself. A last thought: Unless one has a good reason to, and understands what they are doing, using the default is almost always best. Jan 2, 2010 at 16:45
7

The readline method uses the newline chars to delimit what gets read, so the newlines themselves are not returned by readLine.

Don't use readline, you can use a BufferedInputStream and read the file one byte at a time if you want, or pass your own buffer into OutputStream.write.

Note that, like BalusC and Michael Borgwardt say, Readers and Writers are for text, if you just want to copy the file you should use InputStream and OutputStream, you are only concerned with bytes.

4

Any idea what we're doing wrong?

Yes. This line drops the "new line character"

while ((line = input.readLine()) != null) {

And then you write it without it:

 out.write(line.getBytes());

This this related question.

3

There are several things wrong with that code. It may also mutilate any NON-ASCII text since it converts via the platform default encoding twice - and for no good reason at all.

Don't use a Reader to read the file, use a FileInputStream and transfer bytes, avoiding the unnecessary and potentially destructive charset conversions. The line break problem will also be gone.

0
1

BufferedReader.ReadLine() does not preserve the newline. Thus you'll have to add it when writing it out

0

You can use a PrintWriter which offers a prinln() method. This will also save you from converting the string into an array of chars.

public static void writeFile(OutputStream o, File file) throws IOException {
   PrintWriter out = new PrintWriter(new OutputStreamWriter(o));
   BufferedReader input = new BufferedReader(new FileReader(file)); //File input stream 
   String line;
   while ((line = input.readLine()) != null) { //Read file
       out.println(line);  //Write to output stream 
       out.flush();
   }
   input.close();
}  
1
  • He's converting it to an array of bytes (after first converting it from bytes to String) - and the PrintWriter will do the same thing. Dec 30, 2009 at 18:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.