2

I have a text file that is around 1.7MB that has a list of words that I need for my application. I need to have it loaded into an array when the page loads and then be available to other PHP files that AJAX requests are made to. Right now I am loading it into an array on page load and then storing it in a session so that the AJAX file has it available when requests are made to it. Is this the best way to do this?

2
  • Why not to just load it into an array on page load? Why put it into session? Do you really need all these words on each page load? Aug 19, 2010 at 17:47
  • What I was wondering was if I load the file into an array in the PHP file that the AJAX request goes to, if it would load the file into the array on every request.
    – roflwaffle
    Aug 19, 2010 at 23:35

3 Answers 3

4

Really depends on what you are doing with the data. If the AJAX only needs a certain couple of words from that huge list it would probably be best stored in a database.

Do remember that even if the list is stored in a session it will imply that you script loads up that array on every single call. Seams like a huge burden for something that maybe is not really needed.

You don't give us much detail, though.

3
  • and you could gain time performing sql queries instead of loopin the entire 1.7MB of words Aug 19, 2010 at 17:51
  • Just for some more clarification, the array would have close to 200,000 elements (words). The AJAX request is basically checking if the given word is in the array.
    – roflwaffle
    Aug 19, 2010 at 19:40
  • @Brandon don't hesitate in putting it into a DB. You'll get a boost in performance (speed search) using decreased resources (less ram).
    – Frankie
    Aug 19, 2010 at 22:25
1

If the entire array must be loaded into PHP, I'd write a script that made it into a PHP script that contains an array definition of the list of words, so you'd have a .php file that contains:

$myArray = array('blah', 'blah', 'blah'); //etc

Then, I'd include that (require_once)... most PHP script engines will be able to optimize that pretty easily... not much point in storing it in a session, would just use up more disk space/DB space/memory for storage.

0

The file command is pretty quick, so I'm sure there's going to be much to gain by caching anything.

Do you use the data on every single page load? If not, don't go with sessions as you'll have the overhead of loading it even when you don't need it.

Does the data depend upon the user? If not, don't go with session as you'll be duplicating the data all over the place for no reason.

If you are doing more complex parsing, then I'd suggest caching the file using

file_put_contents($filename, '<?php return '.var_export($array, true).';');

Then, all you need to do is $data = include $filename to fetch the data...

You could even make it smart enough to detect changes:

function getData() {
    static $data = array();
    if (empty($data)) {
        $cache = '/path/to/cache/file';
        $full = '/path/to/original/file';
        if (!file_exists($cache) || filemtime($cache) < filemtime($full)) {
            $data = parseDataFileIntoArray($full);
            $code = '<?php return '.var_export($data, true).';';
            file_put_contents($cache, $code);
        } else {
            $data = include $cache;
        }
    }
    return $data;
}
1
  • Care to explain the downvote? Assuming that the entire array must be in PHP land (not using a DB), this is a fairly decent design. It avoids the double memory usage issue on each page load that anything serialized would have. It's usable on all servers (since it doesn't require a PECL extension to work). And it will take full advantage of an OPCode cache if you have one installed (With no need to change the function)...
    – ircmaxell
    Aug 19, 2010 at 18:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.