4

Using command-line ghostscript, is it possible to remove duplicate embedded objects (images) in the PDF and replace them with a single instance?

I have a 200+ pages PDF with a background image and some smaller logos on each page. The file is very large, because the very same background image and logo binaries are embedded in each individual page, instead of being embedded once and then referenced on each page. I am not the creator of the PDF so I can not solve the problem at it's source.

(I do not want to shrink or reduce the image quality, and I do not want delete them completely.)

2 Answers 2

7

As supplement to ghostscript, pdfsizeopt does a very good job in eliminating duplicate embedded objects (including background images) in the PDF and can be run in addition before or after a file is processed by ghostscript. A bit tricky to include in the workflow due it's dependencies however, and creates a lot of temporary files. Can be found at https://github.com/pts/pdfsizeopt (formerly https://code.google.com/p/pdfsizeopt/)

My 200+ pages document got from 150MB to 40MB just by removing duplicate images.

2

No, ghostscript (more specifically the pdfwrite device) won't replace image XObjects or inline images, it doesn't test them to see if tehy are identical.

It would be possible to do so, but it means checking every byte of each image, which can be very expensive on performance, so we don't do it at the moment. If you want to have a go at modifying the source I can give some suggestions on where to start.

FWIW many other objects are tested for duplicates, but not images, simply because of the time taken to read and hash large images.

6
  • 1
    I somehow doubt your assumption about "it means checking every byte of each image". Wouldn't it be possible to create hashes from the image streams, and then compare only these? (Unless you count the creation of a hash as "checking every byte", which somehow it also is...) Commented Dec 4, 2014 at 15:49
  • Yes, I do count creating a hash of an image as 'checking every byte', because, well, that's what you need to do. Currently pdfwrite creates an MD5 hash for a large number of different object types for exactly this purpose, but it doesn't do it for images because reading multiple megabytes of data for what is normally a fairly unusual feature is seens as not worth it. As I said, if someone really wants to do it I can provide pointers on where to create the hash, how to determine if an existing hash matches, and how to replace the new image with the reference to the old.
    – KenS
    Commented Dec 4, 2014 at 15:55
  • 1
    It would be really nice if somebody would pick up such a challenge and implement it with your help. There are a lot of "bad" PDFs out there which are too big for the reason described in the OP. Commented Dec 4, 2014 at 16:06
  • Maybe a 2-step approach could speed up things? First check for objects of the same length. In most cases, images will have different lengths. Only if another object matches in size, do a hash? (Changing the source & building gs myself is currently no option I have, but thanks for the offer.)
    – TeXter
    Commented Dec 4, 2014 at 16:06
  • You would have to then hash both images. Given that we have to decompress and read the whole of the image data anyway, its not that big an overhead (the time taken to process the image data through an MD5 hash algorithm), however I think a 2 stage approach would be quite a bit slower, simply because doing the hash as the image is processed is going to be quicker than doing it after the fact. Though it would probably be quicker for the common cases where the images don't match. More coding though.
    – KenS
    Commented Dec 4, 2014 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.