8

I'm looking for the most efficient way to write the contents of the PHP input stream to disk, without using much of the memory that is granted to the PHP script. For example, if the max file size that can be uploaded is 1 GB but PHP only has 32 MB of memory.

define('MAX_FILE_LEN', 1073741824); // 1 GB in bytes
$hSource = fopen('php://input', 'r');
$hDest = fopen(UPLOADS_DIR.'/'.$MyTempName.'.tmp', 'w');
fwrite($hDest, fread($hSource, MAX_FILE_LEN));
fclose($hDest);
fclose($hSource);

Does fread inside an fwrite like the above code shows mean that the entire file will be loaded into memory?

For doing the opposite (writing a file to the output stream), PHP offers a function called fpassthru which I believe does not hold the contents of the file in the PHP script's memory.

I'm looking for something similar but in reverse (writing from input stream to file). Thank you for any assistance you can give.

1 Answer 1

9

Yep - fread used in that way would read up to 1 GB into a string first, and then write that back out via fwrite. PHP just isn't smart enough to create a memory-efficient pipe for you.

I would try something akin to the following:

$hSource = fopen('php://input', 'r');
$hDest = fopen(UPLOADS_DIR . '/' . $MyTempName . '.tmp', 'w');
while (!feof($hSource)) {
    /*  
     *  I'm going to read in 1K chunks. You could make this 
     *  larger, but as a rule of thumb I'd keep it to 1/4 of 
     *  your php memory_limit.
     */
    $chunk = fread($hSource, 1024);
    fwrite($hDest, $chunk);
}
fclose($hSource);
fclose($hDest);

If you wanted to be really picky, you could also unset($chunk); within the loop after fwrite to absolutely ensure that PHP frees up the memory - but that shouldn't be necessary, as the next loop will overwrite whatever memory is being used by $chunk at that time.

9
  • Thanks pauld, I like your idea of using unset too.
    – Lakey
    Commented Aug 30, 2013 at 2:39
  • 2
    @pauld Nice! For best performance, the unset should be used more like while (...) {...} unset($chunk); re-allocating the memory every iteration wouldn't have much of an impact, but freeing that memory after, is a bit more important here. Commented Sep 24, 2014 at 18:56
  • 1
    @marijnz0r, following a good practice isn't premature optimization. Any code which operates on memory like this should ALWAYS free the memory. Premature optimization would be to analyze the performance of this code, and some variations, selecting an implementation with the lowest time to completion on large data-sets. "Free the memory" isn't early optimization. Commented Aug 24, 2017 at 20:54
  • 1
    @TonyChiboucas picture an API that receives 30 million requests per hour, then question why you would free 1024 from memory :)
    – zanderwar
    Commented May 14, 2022 at 2:22
  • 1
    @zanderwar, that's exactly the point I was making ^_^ Commented May 25, 2022 at 23:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.