9

I would like to know what the benefit of using multiple IDAT-Chunks inside a PNG Image is.

The PNG documentation says

There may be multiple IDAT chunks; if so, they shall appear consecutively with no other intervening chunks. The compressed datastream is then the concatenation of the contents of the data fields of all the IDAT chunks.

I can't imagine it's because of the maximum size (2^32 bytes) of the data-block inside the chunk.

1
  • I imagine the most logical reason is "so that a compressor does not need to load and expand the entire image in a single go."
    – Jongware
    Apr 8, 2015 at 11:44

2 Answers 2

7

Recall that all PNG chunks (including IDAT chunks) have a prefix with the chunk length. To put all the compressed stream in a single huge IDAT chunk would cause these two inconveniences:

  • On the encoder side: the compressor doesn't know the total compressed data size until it has finished the compression. Then, it would need to buffer the full compressed data in memory before writing the chunk prefix.

  • On the decoder side: it depends on how chunk decoding is implemented; if it buffers each chunk in memory (allocating the space given by the chunk length prefix) and, after filling it and checking the CRC, it passes the content to the uncompressor, then, again, having a single huge IDAT chunk would be a memory hog.

Considering this, I believe that use rather small IDAT chunks (say, 16KB or 64KB) should be recommended practice. The overhead (12 bytes per chunk, less than 1/5000 if len=64KB) is negligible.

1
  • I'm not sure it represents any benefit to the decoder at all. What's it supposed to do if the file comes as a single chunk that is too large to load? Throw its hands up and declare the file "unsupported"? How does the encoder know what the limit is for every decoder that could try to read the file? I think it's only a benefit to the encoder and there's no reason to artificially limit the size to speculatively solve a problem that you can't anticipate.
    – sh1
    Jan 25, 2017 at 22:59
3

It appears that when reading a PNG file, libpng limits the chunks of data it buffers to 8192 bytes even if the IDAT chunk size in the file is larger. This puts an upper limit on the allocation size needed for libpng to read and decompress IDAT chunks. However, a checksum error still cannot be detected until the entire IDAT chunk has been read and this could take much longer with large IDAT chunks.

Assuming you're not concerned with early detection of CRC errors (if they do occur they'll still be detected but later on) then small IDAT chunks don't offer any benefit to the reader. Indeed, small IDAT chunks imply more separate calls to zlib and more preamble/postamble costs within zlib, so it's generally less efficient in processing time as well as space on disk.

For the writer, it's convenient to write finite-length IDAT chunks because you can determine before the write how long the chunk will be. If you want to write a single IDAT chunk then you must either complete the compression before beginning to write anything (requiring a lot of temporary storage), or you must seek within your output to update the IDAT chunk length once you know how long it is.

If you're compressing the image and streaming the result concurrently this might be impossible. If you're writing the image to disk then this is probably not a big deal.

In short, small chunks are for the compressing-on-the-fly, streaming-output use case. In most other situations you're better off with just a single chunk.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.