1

I am working on this for hours and its driving me nuts!

so i think its a really tough one hence the detailed question.

The problem is:

  • I have a server that accepts file post uploads by php script.

  • The data will be distributed to several discs.

  • I want to save the file to the right disc during upload (tmp file).

Because:

  • The file needs to be moved later, and i don't want it to take long (copy from the systems tmp directory to target disk)

  • The system drive is an ssd that won't be very happy when so much data is written and deleted on it.

This server config changes the init value. during an upload an can see the temp file and see how its filesize increases:

Alias /thundercloud /home/thundercloud
<Directory /home/thundercloud>
    php_admin_value upload_tmp_dir /home/thundercloud/locations/42/tmp
</Directory>

My sripts are in /home/thundercloud and there are several symlinks in /home/thundercloud/locations (named numerically 1-n) that all point to the according mountpoints.

Now what i need is to get the number 42 (which is disk no. 42 - no there are only 14 disks in the server, dis his just for test) to be dynamic.

I have a lot of control over the request so I basically don't care whether its determined by hostname, get variable, subdirectory etc.

So my first attempt was this:

RewriteLock /var/lock/apache2/rewrite.lock
Alias /thundercloud /home/thundercloud
RewriteMap getuplaodlocation prg:/home/thundercloud/uploadloc.php
RewriteEngine On
<Directory /home/thundercloud>
    AllowOverride All
    php_admin_value upload_tmp_dir ${getuplaodlocation:%{THE_REQUEST}}
</Directory>

the corresponding rewrite map was:

#!/usr/bin/php
<?
set_time_limit(0); # forever program!
$keyboard = fopen("php://stdin","r");
while (1) {
    $line = trim(fgets($keyboard)); // dont care for now
    echo '/home/thundercloud/locations/42/tmp'.PHP_EOL;
}

I jut hardcoded 42. if it worked i would have parsed the correct falue from THE_REQUEST . It did not work. the script was chmod 777 and everything.

My next attempt was:

Alias /thundercloud /home/thundercloud
<Directory /home/thundercloud>
    SetEnvIfNoCase Host lionel\.2x\.to upload=/home/thundercloud/locations/42/tmp
    php_admin_value upload_tmp_dir upload
</Directory>

no luck.

so my final attempt was:

Alias /thundercloud /home/thundercloud
<Directory /home/thundercloud>
    php_admin_value upload_tmp_dir /home/thundercloud/locations/%{REMOTE_HOST}/tmp
</Directory>

no luck either. i printed ini_get_all() and got the following output:

["upload_tmp_dir"]=>
  array(3) {
    ["global_value"]=>
    string(47) "/home/thundercloud/locations/%{REMOTE_HOST}/tmp"

so obviously this "constant" isn't available like this in this scope

now i ran out of ideas

i must admin my apporaches are really trial and error.

anyony can point me in the right direction or tell me its not possible and justify that?

ps: it obviously doesnt work at php runtime ;)

edit:

Ok after some more hours and a useful comment from Marc B I came up with this "solution":

<VirtualHost *:80>
    ServerName 42.thundercloud.lionel.2x.to
    DocumentRoot /home/thundercloud
    php_admin_value upload_tmp_dir /home/thundercloud/locations/42/tmp
</VirtualHost>
<VirtualHost *:80>
    ServerName 43.thundercloud.lionel.2x.to
    DocumentRoot /home/thundercloud
    php_admin_value upload_tmp_dir /home/thundercloud/locations/43/tmp
</VirtualHost>

Just brute force an own VirtualHost container for every disk... its ugly and I hoped to avoid that step and have a generic webserver config, but it seems like the only way to go - atleast it works.

I still wonder whether theres a nicer way and i think its very interesting to go so deep into webserver configurations. Apache is incredibly flexible and its getting really interesting when you push it to the limits (thats at least what i think).

Thats why I will leave this question open and put a bounty on it.

edit:

After some more research I think there might be a solution applying the techniques here: http://httpd.apache.org/docs/2.0/vhosts/mass.html

something like:

<VirtualHost *> 
Use CanonicalName off
VirtualDocumentRoot /home/thundercloud/locations/%0/tmp/
php_admin_value open_basedir VIRTUAL_DOCUMENT_ROOT
<VirtualHost>

This is just an idea, not tested yet. still trying to figure it out. The paths are not correct in this case but there might be a workaround with symlinks. Would be nice if some1 could point me in the right direction.

10
  • Any reason you want different upload dirs? PHP already assigns a random name to the temp file, so two or more uploads are unlikely to stomp on each other. As for the Apache variables, those only apply in certain areas which specifically look for/expect them. php_value/php_admin_value is not one of them.
    – Marc B
    Apr 18, 2011 at 20:35
  • in this case different directories are different physical hard disks. i stated the reasons in the question (no redundant copying of files). in fact uploads are not unlikely but impossible to collied by their name (assuming you dont have a shared network mount as temp directory for several servers or something). Apr 18, 2011 at 20:47
  • 1
    Gotcha. Only thing I can think of is setting a vhost for each disk and use a rewrite rule to re-route the uploads as they come in. Then you can hardcode the tmp path in each vhost without resorting to variables. Unless you're adding disks at a furious rate or their names change frequently, it might just be enough to work.
    – Marc B
    Apr 18, 2011 at 20:51
  • yes i am thinking about that too. problem is by my current design the locations are not 1-n on on every server but each disk in the whole cloud has its own id, therefore they can grow pretty big and i would have to configure the vhosts for every server and it would need a lot of maintainance. i hoped to have one generic config for all servers but the appraoch makes sense, gotta think that through a bit more. maby i can change my database layout to dientify disks differently. Apr 18, 2011 at 20:58
  • 2
    For mass virtualHosting, have a look at mod_macro you will be able to write all your VH with simple commands like "Use MyVHOSt 42.somwhere.com dir42"
    – regilero
    Apr 29, 2011 at 15:29

2 Answers 2

3
+50

There's no way to use variable substitution in php_admin_value settings. This is just because apache modules can only parse their own config directives. mod_rewrite supports using environment variables because doing so is built into mod_rewrite itself.

Because you can't use variables, you'll have to set up all of the possible values using something like <Location>'s. For example:

RewriteEngine on
RewriteRule /upload(\d+)$       /upload.php

php_admin_value upload_tmp_dir /tmp

<Location /upload1>
    php_admin_value upload_tmp_dir /tmp/1
</Location>
<Location /upload2>
    php_admin_value upload_tmp_dir /tmp/2
</Location>
....
<Location /upload100>
    php_admin_value upload_tmp_dir /tmp/100
</Location>

Then you just post to /upload{$id}. This is obviously not as nicely dynamic as it would be if substitution were possible. There are 2 obvious alternatives to make it neater:

1) Have a separate build script which generates a file to put in /etc/httpd/conf.d with the relevant lines for each server

2) Create a universal config file which simply has e.g. the 100 location rules. On each server, you'd then just need to set up the symlinks you wanted and have the PHP which generates the form decide which volume to point the upload at - you'd end up with a bunch of /uploadX urls that wouldn't work on each server, but nothing would post there anyway.

The alternative is to use mod_macro, which basically lets you do the same as above but with a smaller config file, generating the expanded config automatically, but still essentially just generating a big config file with lots of vhost/location blocks. To do it as nicely as you original asked, you've need to modify mod_php.

5
  • good idea to use location instead of vhosts! +1 this way i can use the same domain which is convinient! the idea of the b uild script is good too, i think i am gonna do that! is there any specific reason why it is not possible dynamically? or is it just something i have to believe. maby it would be possible usign an apache module? (i would definately prefer your solution over this but just out of curiosity!) May 3, 2011 at 17:06
  • 1
    it's just because apache modules can only parse their own config directives. mod_rewrite supports using environment variables because it's built into mod_rewrite. the alternative is to use mod_macro, which basically lets you do the same as above but with a smaller config file, generating the expanded config automatically, but still essentially just generating a big config file with lots of vhost/location blocks. to do it as nicely as you original asked, you've need to modify mod_php.
    – Cal
    May 3, 2011 at 17:34
  • thanks! +1 would you move up that comment into your answer so its complete? May 5, 2011 at 9:37
  • i finally tweaked your solution a bit. instead of location i am using locatin match to apply a regex. i partionize my ranges according to servers. i allocated max 100 disks per server and use the last two digits to identify the disk. ex. 1001 is disk 01 on server 10. the drives are mounted accordingly. this way i am good with 100 static location rules on eveyr server. May 28, 2011 at 15:31
  • theres also a discussion about the core issue here: old.nabble.com/LocationMatch-Regex-within--td13807271.html May 28, 2011 at 19:43
-1

If i understood the question correctly, you can always use ini_set

ini_set ("upload_tmp_dir", "/home/thundercloud/locations/42/tmp"); 
2
  • 8
    this does not work, because you have to set the temp dir before the request gets handed off the php. by the time execution passes to php, the uploaded files have already been stored in the temp dir
    – Cal
    May 4, 2011 at 1:02
  • +1 thats the correct reason but its also explicitly stated int he manpages which configurations are compatible with ini set May 5, 2011 at 9:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.