I have a question about the Data.ByteString.Lazy.Char8 library in the bytestring library. Specifically, my question concerns the readFile function, which is documented as follows:
Read an entire file lazily into a ByteString. Use 'text mode' on Windows to interpret newlines
I'm interested in the claim that this function will 'use text mode on Windows to interpret newlines'. The source code for the function is as follows:
-- | Read an entire file /lazily/ into a 'ByteString'. Use 'text mode'
-- on Windows to interpret newlines
readFile :: FilePath -> IO ByteString
readFile f = openFile f ReadMode >>= hGetContents
and we see that, in one sense, the claim in the documentation is perfectly true: the openFile
function (as opposed to openBinaryFile
) has been used, and so newline conversion will be enabled for the file.
But, the file will then be passed to hGetContents. This will call Data.ByteString.hGetNonBlocking
(see the source code here and here), which is meant to be a non blocking version of Data.ByteString.hGet
(see the documentation); and (finally) Data.ByteString.hGet
calls GHC.IO.Handle.hGetBuf
(see the documentation or the source code). This function's documentation says that
hGetBuf ignores whatever TextEncoding the Handle is currently using, and reads bytes directly from the underlying IO device.
which suggests that the fact that we opened the file using readFile
rather than readBinaryFile
is irrelevant: the data will be read without transforming newlines, notwithstanding the claim in the documentation referred to at the beginning of the question.
So, the nub of the question: 1. Am I missing something? Is there some sense in which the statement 'that Data.ByteString.Lazy.Char8.readFile uses text mode on Windows to interpret newlines' is true? Or is the documentation just misleading?
P.S. Testing also indicates that this function, at least when used naively as I was using it, does no newline conversion on Windows.