15

Bottom-line:
Do I need to be concerned about setting post_max_filesize >> memory_limit?

Details:
This answer suggests that uploaded files do not need to fit within php’s memory_limit. The php docs suggest that the entire post should fit within php’s memory limit.

I find the docs surprising and I’m hoping someone can elaborate. For example take the following php configs:

; config A
memory_limit = 50M
upload_max_filesize = 100M
post_max_filesize = 1000M
max_file_uploads = 10    

and

; config B
memory_limit = 50M
upload_max_filesize = 10M
post_max_filesize = 1000M
max_file_uploads = 100    

With these configurations I’d expect to be able to:

  • upload 10x100mb files to server A,
  • and 100x10mb files to server B.
I would also expect that:
  • Working with any one of the 10 files uploaded to server A is a problem (100Ms of file in a 50M bag…).
  • Working with any 1 of the 100 files uploaded to server B is okay (10 < 50).
While experimenting with less round but equivalently related numbers, I’ve found these expectations hold true.

This experience would lead me to say that "generally the memory_limit should be larger than the upload_max_filesize"; instead, the php docs say:

generally speaking, memory_limit should be larger than post_max_size.

Why and what happens if it isn't?

When my php code is executed I see no evidence that all of the posted files are in memory. It seems to me that all I’ve got is a $_FILES array of paths to files found exclusively on disk. Is php holding the whole post in memory at some point prior to my ability to introspect the environment? Do I need to be concerned about setting post_max_filesize >> memory_limit?

Aside:
Violating the manual's rule does not result in a grossly broken server (w/ php5.3 apache2.2 debian 6).

1
  • I would think that the suggestion stems from the fact that if you try to do any processing on all the files (store to variables for instance) then you will exceed PHP's memory limit. Not sure, though, so I'm not putting this as a real answer. Feb 24, 2011 at 15:47

3 Answers 3

14

PHP will accept uploaded files that are individually smaller than upload_max_filesize and together take less than post_max_size bytes. The PHP documentation is wrong with regard to memory_limit which does not need to hold the file contents posted.

The following configuration works with both Apache2 module and CGI, and accepts files smaller than 1G.

 upload_max_filesize = 1G
 post_max_size = 1G
 memory_limit = 32M
0
2

Do I need to be concerned about setting post_max_filesize >> memory_limit?

Only if you plan on reading an entire file into memory and the file that you read in is larger than the space you have allocated to PHP (i.e. memory_limit), in which case you'll run out of memory.

3
  • Uploaded files are not in the POST system there in the _FILES system and php does not give you the entire file contents of them when there are uploaded you have to fread them
    – Barkermn01
    Feb 24, 2011 at 16:01
  • Thus "Only if you plan on reading an entire file into memory"
    – thetaiko
    Feb 24, 2011 at 16:03
  • This seems to roughly support my suggestion that file size and not post size should be relevant. Are you willing to commit to the docs being wrong and post size and memory limit are really only related by use case and the size of the largest posted file one reads from /tmp?
    – Finn
    Feb 24, 2011 at 16:51
-3

My own personal experience is that you HAVE to have a memory_limit higher than post_max_size and upload_max_size.

The post_max_size refers to the entirety of the POSTed data. This includes any form fields that may have been included with the file itself. The upload_max_size is the largest allowable size a file can be within that upload.

For instance. with a post_max_size of 10mb and a upload_max_size of 1mb, you could upload 9 files, each 1mb in size, Why 9 files? because part of the POST data is the file metadata - filename, mimetype, file size, etc... This all takes up some space, so your 9 files will actually take up 9.01megabytes or so. The 0.99 leftover is too small for another file, so you can't upload that 10th, even though it fits within the upload_max_size limit.

As for memory_limit, not only do you have to have enough "room" for the files that were uploaded, you have to remember that this limit applies to the script as a whole. A memory_limit of 10mb would allow for only a 9megabyte file to be uploaded, because PHP itself and all the associated code and libraries will suck up (say) 1 megabyte already.

Even though the files aren't held in memory - they get dumped out to temp files as soon as possible, they are passed in to PHP from Apache via STDIN. PHP has to read the files from that stream and copy them out to the temporary files you use in the ['tmp_name'] section of the $_FILES array.

For whatever reason, PHP seems to be basically doing "file_get_contents()" and slurping the files up in bulk, rather than doing a streaming-type copy. Hence requiring a memory_limit that exceeds the largest allowed file size.

1
  • Granted my setup was a bit of spherical chicken in a vacuum situation but this was intentionality so as to avoid the tangential discussion of field size, upload time etc. Can you provide more detail on php's behavior and explain how I have a working system that apparently contradicts your assertion?
    – Finn
    Feb 24, 2011 at 16:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.