6

I have binary files which contain data structures of various length. I would like to save these blocks of data into separate files. The size of each block is known. The split command can, well, split a file but it does not stop after the first block of data. It slices the file into pieces of equal size.

Therefore, my current solution is to split and cat the remainder of the file back together, iterating my way through the data. This is very clumsy and may even fail in certain circumstances.

What is the best way to slice a binary file precisely at certain positions?

2
  • 1
    It would be a bit more manual but you could use dd. Mar 4, 2015 at 15:03
  • If I see it correctly, dd lets me copy the first block of arbitrary size, but then I can skip only at multiples within the input and output files, respectively. I would need offset and block size as independant parameters.
    – Aziraphale
    Mar 4, 2015 at 15:28

2 Answers 2

14

You can use two independent dd commands. One to seek arbitrarily, and another to copy arbitrary lengths.

SEEK=501
BYTES=387
dd if=yourfile bs=$SEEK skip=1 | dd bs=$BYTES count=1 > lump.bin

Note: Although counter-intuitive to what you are actually try to do, keep the blocksize high and the count low for best performance. What I mean is, if you want 8192 bytes, use bs=8192 count=1 rather than bs=1 count=8192.

2
  • 1
    Thank you. It solves my problem. And it is not counter-intuitive at all.
    – Aziraphale
    Mar 6, 2015 at 6:27
  • Excellent! Saved a hell of a lot of time in my case May 13, 2020 at 3:50
1

I would need offset and block size as independant parameters.

Using the iflag operand may help you here. It is available in dd (coreutils) since 2012, and you can specify different units for bs, skip and count.

Note that skip and count must either be in bytes or match the bs value.

An example usage will be:

dd if=infile.bin of=outfile.bin bs=4K skip=1234 count=20 iflag=skip_bytes,count_bytes 

which would copy 20 bytes from input file, starting from byte 1235.

You can refer here for more detailed usage: https://askubuntu.com/a/1178771

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.