232

How do I create an unmodified hex dump of a binary file in Linux using bash? The od and hexdump commands both insert spaces in the dump and this is not ideal.

Is there a way to simply write a long string with all the hex characters, minus spaces or newlines in the output?

1

9 Answers 9

300
xxd -p file

Or if you want it all on a single line:

xxd -p file | tr -d '\n'
1
  • 19
    fyi To reverse the process: xxd -r -ps hexascii.txt file (it is ok with or without newlines) May 27, 2014 at 23:19
120

Format strings can make hexdump behave exactly as you want it to (no whitespace at all, byte by byte):

hexdump -ve '1/1 "%.2x"'

1/1 means "each format is applied once and takes one byte", and "%.2x" is the actual format string, like in printf. In this case: 2-character hexadecimal number, leading zeros if shorter.

6
  • 9
    You need a -v or it will drop repeated bytes and replace them with an asterisk. Apr 10, 2010 at 22:35
  • 2
    I wonder if hexdump itself is able to append the newline (only) to the end of output.. (obvious appendage of ; echo makes it impossible to use as bash alias)
    – mykhal
    Aug 22, 2015 at 17:57
  • 3
    My alias: alias to_hex="hexdump -ve '1/1 \"%.2x\"' && echo"
    – devstuff
    Feb 16, 2018 at 22:19
  • The iteration count and byte count default to one, so 1/1 may be omitted, leaving the hexdump -ve '"%.2x"'
    – Alex Che
    Sep 27, 2018 at 10:16
  • 1
    You can reverse this output back into a binary with xdd -p -r < dump > file
    – dagelf
    Nov 3, 2021 at 9:04
29

It seems to depend on the details of the version of od. On OSX, use this:

od -t x1 -An file |tr -d '\n '

(That's print as type hex bytes, with no address. And whitespace deleted afterwards, of course.)

1
12

Perl one-liner:

perl -e 'local $/; print unpack "H*", <>' file
7
  • 1
    Verified. Matches "xxd -p file | tr -d '\n'". May 27, 2014 at 22:56
  • 1
    fyi To reverse the process: perl -e 'local $/; print pack "H*", <>' <hexascii.txt >file May 27, 2014 at 23:17
  • The "local $/" is unnecessary. May 28, 2014 at 15:39
  • Update to last comment: The "local $/" is unnecessary for "pack". For unpack, you need it but can alternatively put "undef $/". $/ is the line separator (default NL). undefined puts it into slurp-mode. So <> referenced in a string context pulls the whole binary file without parsing it into lines. May 28, 2014 at 15:54
  • 1
    Alternate form 2 for reversal (hexascii to binary), removes any newlines (xxd -ps -r adds them): perl -pe 's/\n//g; $_=pack "H*", $_' <hexascii.txt >file May 28, 2014 at 16:11
3

The other answers are preferable, but for a pure Bash solution, I've modified the script in my answer here to be able to output a continuous stream of hex characters representing the contents of a file. (Its normal mode is to emulate hexdump -C.)

3

You can use Python for this purpose:

python -c "print(open('file.bin','rb').read().hex())"

...where file.bin is your filename.

Explaination:

  1. Open file.bin in rb (read binary) mode.
  2. Read contents (returned as bytes object).
  3. Use bytes method .hex(), which returns hex dump without spaces or new lines.
  4. Print output.
2

tldr;

$ od -t x1 -A n -v <empty.zip | tr -dc '[:xdigit:]' && echo 
504b0506000000000000000000000000000000000000
$

Explanation:

Use the od tool to print single hexadecimal bytes (-t x1) --- without address offsets (-A n) and without eliding repeated "groups" (-v) --- from empty.zip, which has been redirected to standard input. Pipe that to tr which deletes (-d) the complement (-c) of the hexadecimal character set ('[:xdigit:]'). You can optionally print a trailing newline (echo) as I've done here to separate the output from the next shell prompt.

References:

1

I think this is the most widely supported version (requiring only POSIX defined tr and od behavior):

cat "$file" | od -v -t x1 -A n | tr -d ' \n'

This uses od to print each byte as hex without address without skipping repeated bytes and tr to delete all spaces and linefeeds in the output. Note that not even the trailing linefeed is emitted here. (The cat is intentional to allow multicore processing where cat can wait for filesystem while od is still processing previously read part. Single core users may want replace that with < "$file" od ... to save starting one additional process.)

3
  • No need to use cat here, just pass the filename to od.
    – Martin
    Jun 27, 2022 at 18:21
  • I agree that historically connecting stdin is considered better and it's still the correct solution for single core CPUs. However, with modern systems with many idle cores in a single socket CPU, I think it's better to allow cat to read pipe buffer worth of input in parallel with the od at all times. This reduces the possibility of od stalling for reading the file. Jun 28, 2022 at 7:05
  • I did some microbenchmarks and it seems that using the cat at the start does allow using multiple CPU cores for the task in parallel but in practice, the intercommunication between the cores causes so much extra work that avoiding the use of cat at the start of pipeline is still faster – at least for locally accessible files. In case the file was on remote network drive with slow connection, using the cat here could improve performance. That said, the od can only handle about 4 MB/s so it will be the bottleneck in most cases. Jun 29, 2022 at 7:49
1

This code produces a "pure" hex dump string and it runs faster than the all the other examples given. It has been tested on 1GB files filled with binary zeros, and all linefeeds. It is not data content dependent and reads 1MB records instead of lines.

perl -pe 'BEGIN{$/=\1e6} $_=unpack "H*"'

Dozens of timing tests show that for 1GB files, these other methods below are slower. All tests were run writing output to a file which was then verified by checksum. Three 1GB input files were tested: all bytes, all binary zeros, and all LFs.

hexdump -ve '1/1 "%.2x"'                    #  ~10x  slower
od -v -t x1 -An | tr -d "\n "               #  ~15x  slower
xxd -p | tr -d \\n                          #   ~3x  slower
perl -e 'local \$/; print unpack "H*", <>'  # ~1.5x  slower
- this also slurps the whole file into memory

To reverse the process:

perl -pe 'BEGIN{$/=\1e6} $_=pack "H*",$_'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.