38

What I'm looking for is a counterpart to file -I (Darwin; -i on Linux).

For example, given:

$ file -I filename.pdf
filename.pdf: application/octet-stream; charset=binary

I would like to be able to do something like this:

$ [someCommand] filename.pdf application/pdf

The result would be that filename.pdf would then be typed as application/pdf.

The reason for the question is that sometimes web servers use the wrong MIME type, which results in programs refusing to open the file. (Most often text/plain, in my experience.)

I've been searching man, the web and this site for about two and a half hours. Tried everything from hex dumps to xattr to text editors.

Your help would very much be appreciated.

Chris

3
  • It's unclear whether you are working on OS X or Linux.
    – JWWalker
    Mar 12, 2015 at 23:26
  • I'm using OS X. I mentioned Linux because there was a spat elsewhere on this site about -I vs -i, respectively. Both therefore seemed relevant, though.
    – curieux
    Mar 13, 2015 at 2:13
  • In Linux the command in question is xdg-mime.
    – 0andriy
    Jan 10, 2022 at 14:05

3 Answers 3

34

The thing about MIME types is they're almost entirely fictional.

MIME and HTTP ask us to pretend that all of our files have a piece of metadata identifying the "content type". When we send files around the network, the "content type" metadata goes with them, so nobody ever misinterprets the content of a file.

The truth is this metadata doesn't exist. By the time MIME was invented, it was really too late to convince any OS vendors to adopt a new type system for files. Unix had settled on magic numbers, DOS had settled on 3-letter filename suffixes, and classic MacOS had its creator codes and type codes. (MacOS type codes were closest to the MIME model, since they actually were separate from both the filename and the content. But being only 4 letters long, MIME types wouldn't fit.)

Nobody stores MIME-compatible content types in their filesystem. When a MIME message composer or HTTP server wants to send a file, it decides the file type in the traditional way (filename suffix and/or magic number) and maps the result to a MIME type.

In contrast to the theory (where MIME eliminates file type guessing), MIME as implemented in practice has moved the "guess file type based on filename suffix and/or magic number" logic from the receiver of the file to the sender. As you have noticed, the sender doesn't usually do a better job than the receiver would have done if forced to figure it out for itself. Frequently in the case of a web server, the server's eagerness to slap a Content-type on a file makes things worse. There's no reason for a web server to know anything about the format of files it serves when it is only being used to distribute them and has no need to interpret their contents.

The file command guesses file type by reading the content and looking for magic numbers and strings. The -I option doesn't change that. It just chooses a different output format.

To change the Content-Type header that a web server sends for a specific file, you should be looking in your web server's configuration manual. There's nothing you can do to the file itself.

4
  • Classic Mac OS type and creator codes are not stored in the resource fork, but in fields of the disk catalog. (Not that it's important for this question.)
    – JWWalker
    Mar 12, 2015 at 20:38
  • Thank you for the explanation. It's funny because I suspected a few things you just confirmed. (Although, you added considerably more after that!) Two things if I may. First, OS X does store MIME types of downloaded files, but I don't see that it uses the information for any practical purpose. If you drop such a file into Text Wrangler, for example, the binary is surrounded by XML tags, one of which is MIME type. Changing it does not change file -I output or solve any problems. Second, so, when I d/l a pdf that opens as a text file and changing the .ext doesn't work, how do I fix it?
    – curieux
    Mar 13, 2015 at 2:29
  • This is a matter of cost. Server running the HTTP service is assumed to be in better position to perform the magic because it's closer to the file, it's assumed to be more powerful than client computers, and can cache that information for future use. Apr 25, 2018 at 18:50
  • 3
    It's worth noting that binary cloud storage (Amazon S3, Azure Blob Storage, etc.) nowadays DOES store separate metadata for each binary including Content-Type. Whatever the uploading client passed as mime type in the Content-Type header will be stored there and returned when reading the resource using GET or HEAD requests. Jan 14, 2019 at 19:09
4

It's a bit of a category mistake to talk about ‘the MIME type of a file’ – ‘files’ don't have MIME types; only octet streams have them (I'm not necessarily disagreeing with @wumpus-q-wumbley's description of MIME types as ‘fictional’, but this is another way of thinking about it).

MIME stands for Multipurpose Internet Mail Extensions, as originally described in in RFC 2045, and MIME types were originally intended to describe what a receiver is supposed to do with the bunch of bytes soon to follow down the wire, in the rest of the email message. They were very naturally repurposed in (for example) the HTTP protocol, to let a client understand how it is to interpret the bytes in the HTTP response which this MIME type forms the header of.

The fact that the file command can display a MIME type suggests the further extension of the idea, to act as the key which lets a windowing system look up the name of an application which should be used to open the file.

Thus, if ‘the MIME type of a file’ means anything, it means ‘the MIME type which a web server would prefix to this file if it were to be delivered in response to an HTTP request’ (or something like that). Thought of like that, it's clear that the MIME type is part of the web server's configuration, and not anything intrinsic to the file – a single file might be delivered with various MIME types depending on the URL which retrieves it, and details of the request and configuration. Thus an XHTML file might be delivered as text/html or application/xml or application/octet-stream depending on the details of the HTTP request, the directory the file's located in, or indeed the phase of the moon (the latter would be an unhelpful server configuration).

A web server might have a number of mechanisms for deciding on this MIME type, which might include a lookup table based on any file extension, a .htaccess file, or indeed the output of the file command.

So the answer to your question is: it depends.

  • If what you want to do is change how a web server delivers this file, then you need to look at either your web server documentation, or the contents of your system's /etc/mime.types file (if your system uses that and if the server is configured to fall back on that).
  • If what you want to do is to change the application which opens a given (type of) file, then your OS/window-manager documentation should help.
  • If you need to change the output of the file command specifically, for some other reason, then man file is your friend, and you'll probably need to grub around in the magic numbers file, reasonably carefully.
3
  • Thank you for your comment. I'm not running a web server, though. My issue was trying to open a file that someone else's server mangled by using the wrong MIME type. Occasionally, I get a PDF encoded as plain text, and nothing will open it as a PDF. Changing the extension doesn't help. I thought the problem was the MIME type, but evidently not. Any ideas?
    – curieux
    Mar 13, 2015 at 2:36
  • Ah, well in this specific case (OS X), it might be that the PDF has been given a wrong 'creator code', which is plausible if it was downloaded from a server which said 'this is a text/plain octet-stream, honest!' Try GetFileInfo <file> and look at the 'type' and 'creator' four-character codes. If those don't look like "\0\0\0\0", then this might indeed be the problem, and SetFile and its -c or -t options might help; specifically, setting them to '' might restore the default behaviour. Mar 13, 2015 at 13:34
  • @curieux History note (apologies if you know this already!): it used to be that MacOS cared a lot more about the type and creator codes, and used them, exclusively, to do the sort of lookup of helper applications which we're discussing here. That was a good plan, but since MacOS switched to unix and became OS X, it's become more natural to use techniques based on file extensions and file heuristics, so that the type/creator codes have mostly become forgotten, except for (what may be) this sort of case, where they leap up and bite you on the bum. Mar 13, 2015 at 13:44
4

If you have a pdf, and the $file --mime-type command answer octet-stream and not application/pdf, you have a corruption in your file.

The pdf readers will read it, and ignore the problem, but if you upload this file to a web application, the application will recognize the mime-type as a octet-sream. Sometimes it is a problem, mainly if you validate the mime-type (I sometimes have this problem in my application).

To get a fast solution, use a ghost script like this:

gs -o new.pdf -sDEVICE=pdfwrite -dPDFSETTINGS=/prepress old.pdf
1
  • Unrecoverable error: undefined in getOperand stack: --nostringval-- 528_Bertolon_1789_Ob-elektricheskoi-materii-tela.pdf
    – progonkpa
    Nov 29 at 11:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.