70

How can I have PHP 5.2 (running as apache mod_php) send a complete HTTP response to the client, and then keep executing operations for one more minute?

The long story:

I have a PHP script that has to execute a few long database requests and send e-mail, which takes 45 to 60 seconds to run. This script is called by an application that I have no control over. I need the application to report any error messages received from the PHP script (mostly invalid parameter errors).

The application has a timeout delay shorter than 45 seconds (I do not know the exact value) and therefore registers every execution of the PHP script as an error. Therefore, I need PHP to send the complete HTTP response to the client as fast as possible (ideally, as soon as the input parameters have been validated), and then run the database and e-mail processing.

I'm running mod_php, so pcntl_fork is not available. I could work my way around this by saving the data to be processed to the database and run the actual process from cron, but I'm looking for a shorter solution.

6
  • 5
    Sorry, but this looks like a total misuse of the PHP language. Sep 30, 2010 at 17:07
  • 3
    Not as much as misuse of the PHP language as misuse of a webserver process. If no HTTP / web is involved anymore no webserver should be busy with it.
    – Wrikken
    Sep 30, 2010 at 17:10
  • 82
    System abuse or not, sometimes we must do things we don't like due to requirements outside of our control. Doesn't make the question invalid, just makes the situation unfortunate. Sep 30, 2010 at 18:26
  • 15
    I don't see how this is abuse, at all. If it is, someone should tell Amazon to shut down amazon.com, since most of the work involved in packing and shipping an order takes place after the purchase web request is completed. Either that, or set a two week timeout on amazon.com purchase requests and only deliver the response to the browser once the order has been delivered to the customer.
    – tex
    Apr 29, 2011 at 14:34
  • 27
    let's try to keep personal opinions to ourselves. answer the question or go elsewhere, please. Dec 8, 2011 at 18:48

13 Answers 13

52

I had this snippet in my "special scripts" toolbox, but it got lost (clouds were not common back then), so I was searching for it and came up with this question, surprised to see that it's missing, I searched more and came back here to post it:

<?php
 ob_end_clean();
 header("Connection: close");
 ignore_user_abort(); // optional
 ob_start();
 echo ('Text the user will see');
 $size = ob_get_length();
 header("Content-Length: $size");
 ob_end_flush(); // Strange behaviour, will not work
 flush();            // Unless both are called !
 session_write_close(); // Added a line suggested in the comment
 // Do processing here 
 sleep(30);
 echo('Text user will never see');
?>

I actually use it in few places. And it totally makes sense there: a banklink is returning the request of a successful payment and I have to call a lot of services and process a lot of data when that happens. That sometimes takes more than 10 seconds, yet the banklink has fixed timeout period. So I acknowledge the banklink and show him the way out, and do my stuff when he is already gone.

7
  • 2
    I advice to add session_write_close(); after flush(); if you are using sessions, otherwise you will not be able to use your site (in the same browser tab) until your (background) processing finish. Nov 21, 2013 at 13:10
  • 3
    it doesnt work on php5 and chrome browser on linux, chrome waits 30 seconds before terminating the connection Mar 24, 2014 at 17:55
  • 2
    The ignore_user_abort(); // optional would have no effect, without passing a value (Boolean) that function returns the current setting.
    – ficuscr
    Jun 3, 2014 at 18:51
  • 2
    I tested this solution on my shared hosting, and "Text user will never see" was shown after 30 seconds waiting.
    – NLemay
    May 8, 2015 at 14:37
  • 4
    There should be ignore_user_abort(true); instead of ignore_user_abort(); Jun 17, 2015 at 11:46
29

Have the script that handles the initial request create an entry in a processing queue, and then immediately return. Then, create a separate process (via cron maybe) that regularly runs whatever jobs are pending in the queue.

2
  • 2
    This the solution I originally had in mind. On the other hand, setting up a processing queue for the sole purpose of working around a timeout in a third party application makes me feel a bit uneasy. Sep 30, 2010 at 22:32
  • 2
    This solution suffers from lack of parallelism... or one will need to start a pool of worker-proccesses to serve the queue. I ended up posting and then disconnecting http requests to self-localhost (in a manner described by SomeGuy here) to utilize a pool of existing httpd workers as background processors. Sep 30, 2014 at 19:36
9

What you need is this kind of setup

alt text

3
  • Umm, but, based on this diagram, status message gets sent back to client only when cron executes - 5-10 minutes maximum. Anyway, nice diagram! Mar 11, 2011 at 10:45
  • status messages could be requested at any time :) the point was that there are two separate and independent processes going on here. But otherwise, thanks! Mar 11, 2011 at 14:44
  • 2
    +1 Wow, great diagram! But instead of the user requesting the status continuously, I think that websockets are better.
    – Oriol
    Mar 9, 2013 at 3:17
9

One can to use "http fork" to oneself or any other script. I mean something like this:

// parent sript, called by user request from browser

// create socket for calling child script
$socketToChild = fsockopen("localhost", 80);

// HTTP-packet building; header first
$msgToChild = "POST /sript.php?&param=value&<more params> HTTP/1.0\n";
$msgToChild .= "Host: localhost\n";
$postData = "Any data for child as POST-query";
$msgToChild .= "Content-Length: ".strlen($postData)."\n\n";

// header done, glue with data
$msgToChild .= $postData;

// send packet no oneself www-server - new process will be created to handle our query
fwrite($socketToChild, $msgToChild);

// wait and read answer from child
$data = fread($socketToChild, $dataSize);

// close connection to child
fclose($socketToChild);
...

Now the child script:

// parse HTTP-query somewhere and somehow before this point

// "disable partial output" or 
// "enable buffering" to give out all at once later
ob_start();

// "say hello" to client (parent script in this case) disconnection
// before child ends - we need not care about it
ignore_user_abort(1);

// we will work forever
set_time_limit(0);

// we need to say something to parent to stop its waiting
// it could be something useful like client ID or just "OK"
...
echo $reply;

// push buffer to parent
ob_flush();

// parent gets our answer and disconnects
// but we can work "in background" :)
...

The main idea is:

  • parent script called by user request;
  • parent calls child script (same as parent or another) on the same server (or any other server) and gives request data to them;
  • parent says ok to user and ends;
  • child works.

If you need to interact with child - you can use DB as "communication medium": parent may read child status and write commands, child may read commands and write status. If you need that for several child scripts - you should keep child id on the user side to discriminate them and send that id to parent each time you want to check status of respective child.

I've found that here - http://linuxportal.ru/forums/index.php/t/22951/

2
  • 1
    This approach (slightly modified) is the only working solution I found to create background task from apache's mod_php without overhead of starting separate OS process - this will occupy and use one of already existing httpd workers instead Sep 30, 2014 at 19:27
  • In the parent script's fread($socketToChild, $dataSize), where does $dataSize come from? Do you need to know exactly how much data to expect out of the socket (including size of headers)? I must be missing something. Jun 4, 2016 at 7:36
5

What about calling a script on the file server to execute as if it had been triggered at the command line? You can do this with PHP's exec.

2
  • +1, something like Gearman is already set up for it (but other / ones own solutions are of course equally valid).
    – Wrikken
    Sep 30, 2010 at 17:11
  • 1
    exec() is often a problem in shared/hosted spaces. Plus a huge security risk.
    – ErnestV
    Jan 31, 2015 at 11:54
5

You can use the PHP function register-shutdown-function that will execute something after the script has completed its dialog with the browser.

See also ignore_user_abort - but you shouldn't need this function if you use the register_shutdown_function. On the same page, set_time_limit(0) will prevent your script to time out.

1
4

Using a queue, exec or cron would be an overkill to this simple task. There is no reason not to stay within the same script. This combination worked great for me:

        ignore_user_abort(true);
        $response = "some response"; 
        header("Connection: close");
        header("Content-Length: " . mb_strlen($response));
        echo $response;
        flush(); // releasing the browser from waiting
        // continue the script with the slow processing here...

read more in: How to continue process after responding to ajax request in PHP?

1
  • 2
    You may need to disable additional buffering which occurs in Apache: <?php apache_setenv('no-gzip', 1); ini_set('zlib.output_compression', 0); ini_set('implicit_flush', 1);?> Mar 26, 2012 at 16:08
4

It is possible to use cURL for that, with a very short timeout. This would be your main file:

<?php>
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, "http://example.com/processor.php");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_TIMEOUT_MS, 10);      //just some very short timeout
    curl_exec($ch);
    curl_close($ch);
?>

And this your processor file:

<?php
    ignore_user_abort(true);                       //very important!
    for($x = 0; $x < 10; $x++)                     //do some very time-consuming task
        sleep(10);
?>

As you can see, the upper script will timeout after a short time (10 milliseconds in this case). It is possible that CURLOPT_TIMEOUT_MS will not work like this, in this case, it would be equivalent to curl_setopt($ch, CURLOPT_TIMEOUT, 1).

So when the processor file has been accessed, it will do its tasks no matter that the user (i.e. the calling file) aborts the connection.

Of course you can also pass GET or POST parameters between the pages.

3
  • 2
    I have been looking for a solution to this problem for quite a while now and this one works! Thanks a lot. The other solutions might work in specific scenarios, except if you have have limited control over your webserver only and can't fork new processes; a configuration I commonly find on commercial webservers. This solution still works! One important addition. For UNIX systems you need to add curl_setopt($ch, CURLOPT_NOSIGNAL, 1); for timeouts < 1 sec to work. Check here for the explanation. Feb 3, 2017 at 9:12
  • finally, genuine!
    – Reloecc
    May 5, 2020 at 9:54
  • My processor script didn't execute, so I increased CURLOPT_TIMEOUT_MS to 100. It works like a charm, and enables me to quickly respond when receiving events from our payment provider.
    – Jette
    Mar 2, 2022 at 7:43
3

You can create an http request between server and server. (not browser is needed). The secret to create a background http request is setting a very small timeout, so the response is ignored.

This is a working function that I have used for that pupose:

MAY 31 PHP asynchronous background request Another way to create an asynchronous request in PHP (simulating background mode).

 /**
  * Another way to make asyncronous (o como se escriba asincrono!) request with php
  * Con esto se puede simpular un fork en PHP.. nada que envidarle a javita ni C++
  * Esta vez usando fsockopen
  * @author PHPepe
  * @param  unknown_type $url
  * @param  unknown_type $params
  */
 function phpepe_async($url, $params = array()) {
  $post_params = array(); 
     foreach ($params as $key => &$val) {
       if (is_array($val)) $val = implode(',', $val);
         $post_params[] = $key.'='.urlencode($val);
     }
     $post_string = implode('&', $post_params);

     $parts=parse_url($url);

     $fp = fsockopen($parts['host'],
         isset($parts['port'])?$parts['port']:80,
         $errno, $errstr, 30);

     $out = "POST ".$parts['path']." HTTP/1.1\r\n";
     $out.= "Host: ".$parts['host']."\r\n";
     $out.= "Content-Type: application/x-www-form-urlencoded\r\n";
     $out.= "Content-Length: ".strlen($post_string)."\r\n";
     $out.= "Connection: Close\r\n\r\n";
     if (isset($post_string)) $out.= $post_string;

     fwrite($fp, $out);
     fclose($fp);
 }

 // Usage:
 phpepe_async("http://192.168.1.110/pepe/feng_scripts/phprequest/fork2.php");

For more info you can take a look at http://www.phpepe.com/2011/05/php-asynchronous-background-request.html

0

You can split these functions into three scripts. 1. Initiate process and call second one via exec or command, this is also possible to run via http call. 2. second one will run database processing and at the end will start last one 3. last one will email

0

Bah, I misunderstood your requirements. Looks like they're actually:

  • Script receives input from an external source you do not control
  • Script processes and validates the input, and lets the external app know if they're good or not and terminates the session.
  • Script kicks off a long-running proccess.

In this case, then yes, using an outside job queue and/or cron would work. After the input is validated, insert the job details into the queue, and exit. Another script can then run, pick up the job details from the queue, and kick off the longer process. Alex Howansky has the right idea.

Sorry, I admit I skimmed a bit the first time around.

0

I would recommend spawning a new async request at the end, rather than continuing the process with the user.

You can spawn the other request using the answer here: Asynchronous PHP calls?

0

In your Apache php.ini config file, make sure that output buffering is disabled:

output_buffering = off

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.