4

I read the first byte to differentiate file types but both PDF and DOCX has a "0x50" magic number. How do I handle this circumstance?

4
  • 1
    Why not use the file extension?
    – Oscar
    Aug 24, 2015 at 9:23
  • Agh stupid of me, I'm just making things complicated. Thank you so much for putting some sense in to me ! :) Aug 24, 2015 at 10:06
  • 5
    @Oscar because you can just rename a file something.pdf even if it isn't a PDF; that doesn't make it a PDF, that just makes it a file named with the extension ".pdf"; magic detection adds an additional sanity check. I work with a system which has to process different image types, and I can't send a JPEG through the PDF process or it would break, hence the magic detection.
    – JoshDM
    Jul 1, 2020 at 20:06
  • @JoshDM If in my system I expect pdf files to have pdf extensions and you send me a jpg with pdf extension, I just discard it as soon as it throws error. Of course, you can try with magic numbers and similar techniques, but I prefer personally to stay in the KISS principle.
    – Oscar
    Jul 2, 2020 at 10:45

3 Answers 3

7

PDF files don't have a "magic" byte they start with. If you read the PDF specification you'll see they have to start with "%PDF", but in practice many PDF files do not.

1) Just looking for a %PDF header to identify PDF files is highly unreliable, a valid PDF file is a file you can parse (that at least has a trailer, cross-reference table and so forth).

2) There was a suggestion once that PDF files contain binary data before the %PDF header to make sure they were treated as binary files. As a result PDF readers at one point started accepting a certain number of binary bytes (random bytes) before the %PDF header. Such files cannot be detected by a simple magic number or string of magic numbers.

5
  • 2
    [citation needed] for most of your unusual claims. Hugo's citations indicate that %PDF is pretty standard.
    – NH.
    Jan 29, 2018 at 22:27
  • 2
    See (for example) implementation note 13 in the PDF Reference second edition for PDF version 1.3 from Adobe Systems: "13. Acrobat viewers require only that the header appear somewhere within the first 1024 bytes of the file.". Yes, this is an old document, but it is still the basis of a large part of the PDF standards in use worldwide. Also, I've written the first versions of a major commercial PDF preflight software starting in 1997. Trust me, my unusual claim is correct. Jan 30, 2018 at 22:43
  • While this is an unusual claim, I have a stack of PDFs made with crawfordtech.com/products/enterprise-output-management/pro-pdf that have ~50 seemingly random bytes at the beginning. They are the same bytes in every file, though.
    – Jayen
    Jul 30, 2020 at 4:15
  • 3
    @Jayen in a current software putting anything before the %PDF or after the %%EOF can be considered a bug (unless the pdfs are not meant for distribution but merely for some special printer queue for example).
    – mkl
    Oct 15, 2020 at 6:23
  • I have several thousand PDF documents from many different pages as a test dataset for PyPDF2. Here are the results for the first 8 bytes: 7842x %PDF-1.4, 4937x %PDF-1.3, 4524x %PDF-1.2, 1664x %PDF-1.6, 2522x %PDF-1.5, 868x %PDF-1.1, 401x %PDF-1.7, 83x %PDF-1.0, 1x TM-107, 1x t55-56, 1x ISSRes, 1x \n%PDF-1, 1x 052165, 1x feb5.p, 1x `` Jun 19, 2022 at 19:05
2

It is weird because I see 0x25 for PDF files and 0x50 for DOCX files ... (source 1 source 2). But still, when you open those files in text mode, in ISO 8859-1 encoding, you are able to see a DOCX document start as "PK" while a PDF document would start with "%PDF".

Hope it helps ! Hugo.

1
  • 2
    That's because in ASCII, 0x25 = '%', and 0x50 = 'P' ... the DOCX files are actually PKZip files, you can rename them to .zip and extract them. Jun 28, 2022 at 1:06
-1

Seem PDF have multiple signatures. Some types of PDF ends with these 8 bytes

0A 0D 0A 30 0D 0A 0D 0A

2
  • 2
    In a current software putting anything before the %PDF or after the %%EOF can be considered a bug (unless the pdfs are not meant for distribution but merely for some special printer queue for example).
    – mkl
    Oct 15, 2020 at 6:24
  • These PDFs come as email attachment, we check them for consistency with format and turn them .... into PDF (probably bypassing this when it is already pdf) and store. They can be displayed then in the browser
    – T.S.
    Oct 15, 2020 at 13:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.