30

I have a struct with a BufStream<T> where T: Read+Write. The BufStream can be a TcpStream and I'd like to read n bytes from it. Not a fixed amount of bytes in a predefined buffer, but I have a string/stream which indicates the number of bytes to read next.

Is there a nice way to do that?

2 Answers 2

34

Since Rust 1.6, Read::read_exact can be used to do this. If bytes_to_read is the number of bytes you need to read, possibly determined at runtime, and reader is the stream to read from:

let mut buf = vec![0u8; bytes_to_read];
reader.read_exact(&mut buf)?;

The part that wasn't clear to me from the read_exact documentation was that the target buffer can be a dynamically-allocated Vec.

Thanks to the Rust Gitter community for pointing me to this solution.

6
  • It's also not clear what happens if EOF is returned before bytes_to_read. Sep 5, 2018 at 3:01
  • 3
    That's specified in the documentation: "If this function encounters an "end of file" before completely filling the buffer, it returns an error of the kind ErrorKind::UnexpectedEof."
    – rspeer
    Sep 6, 2018 at 18:24
  • True, so I guess you can't actually do this. In the sense that it's not defined what happens to the buffer contents/impossible to recover. Sep 6, 2018 at 18:48
  • How can you use a Vec as the target, when I attempt to do this with a Vec::with_capacity(bytes_to_read) the vec "length" is zero and nothing gets read Oct 11, 2019 at 13:20
  • 4
    @Nanos Hopefully you've already figured this out elsewhere, but, the answer is you need to fill the Vec with some value first, (or unsafely set the length I guess). The vec![0u8; bytes_to_read] used above fills the Vec with bytes_to_read zeroes.
    – Ryan1729
    Nov 3, 2019 at 13:42
24

It sounds like you want Read::take and Read::read_to_end.

This will allow you to read data into a &mut Vec<u8>, which is useful when you want to reuse an existing buffer or don't have an appropriately sized slice already. This allows you to avoid initializing the data with dummy values before overwriting them with the newly-read information:

use std::{
    io::{prelude::*, BufReader},
    str,
};

fn read_n<R>(reader: R, bytes_to_read: u64) -> Vec<u8>
where
    R: Read,
{
    let mut buf = vec![];
    let mut chunk = reader.take(bytes_to_read);
    // Do appropriate error handling for your situation
    // Maybe it's OK if you didn't read enough bytes?
    let n = chunk.read_to_end(&mut buf).expect("Didn't read enough");
    assert_eq!(bytes_to_read as usize, n);
    buf
}

fn main() {
    let input_data = b"hello world";
    let mut reader = BufReader::new(&input_data[..]);

    let first = read_n(&mut reader, 5);
    let _ = read_n(&mut reader, 1);
    let second = read_n(&mut reader, 5);

    println!(
        "{:?}, {:?}",
        str::from_utf8(&first),
        str::from_utf8(&second)
    );
}

If you are worried that Read::take consumes the reader by reference, note that take comes from Read and Read is implemented for any mutable reference to a type that implements Read. You can also use Read::by_ref to create this mutable reference.

See also:

8
  • 1
    Thanks.. doesn't take() consume the reader? My usecase is that i have a tcpstream (part of a struct) from which i read lines. Sometimes in between i don't need to read a line but a specific amount of bytes. Afterwards i'll need to read lines again...
    – buster
    May 23, 2015 at 14:47
  • 2
    It does, but "the reader" can be a &mut R i.e a temporary mutable reference to another reader. Take uses a by-value API so that all use cases are supported. This model appears in some different corners of rust.
    – bluss
    May 23, 2015 at 14:49
  • @buster as bluss points out, &mut R also implements the Read trait. My example uses that to good effect - that's why I pass in &mut reader in the main method of the example.
    – Shepmaster
    May 23, 2015 at 14:50
  • @buster @Shepmaster how did you guys figure out that Read::take also accepts a &mut reader? Wondering how I would be able to figure that out myself next time. :)
    – Kaeros
    Sep 5, 2019 at 20:00
  • 2

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.