0

I have a 10MB file that I want to write onto a MYSQL using PHP.

Here it is (trimmed down):

if(isset($_FILES['EDIT_DATABASE']))
{        
$TARGET = 'TEXTFILE/geo.txt';
move_uploaded_file($_FILES['EDIT_DATABASE']['tmp_name'], $TARGET);

$FILE = fopen($TARGET,'r');

while($LINE = fgets($FILE)) 
{           
INSERT INTO MYSQL
}}

It's constantly stops after a little bit of time running. And therefore I cannot add all the data at once.

My max_execution_time is 30, but the script runs for about 10-20 min before running out. max_input_time is -1 (unlimited). I've checked other PHP MySQL settings and everything seems to be in order. What am I missing?

What can be causing the problem. I get absolutely NO ERRORs when I run it. It just stops in 10-20 min without finishing. Any ideas?

1 Answer 1

0

Just a thought, why don't you just read 50 lines of the text file in at a time into the database, instead of all at once?

5
  • I was actually thinking about doing that, read about 10000. But I am unsure of how to read 10000 then restart and read next 10000. Personally I would like to fix the problem instead of going around it.
    – Maciek
    Feb 24, 2014 at 23:39
  • The reason you might want to consider reading in chucks, is there might be data loss when trying to send the data to the database. Sending in chunks will ensure integrity in the data when received. This isn't really a work-around, its more of a ideal solution then sending in 10k+ lines of data to the database. You would have to keep track of how many lines you have read, and your position in the text file. Push the data into the database and start back at your position counter. Feb 24, 2014 at 23:43
  • Did not work. I found a way to get rid of the array and work with each line individually "packets of 1" but it still timed out. :(
    – Maciek
    Feb 25, 2014 at 4:16
  • keep the pointer variables stored in a session, then after it reaches the max line, refresh. Feb 25, 2014 at 15:30
  • I did that with a $_GET from url. Refreshed every 1000 entries, but at 5000, it stopped. I guess it is the browser that is stopping the code.
    – Maciek
    Feb 25, 2014 at 17:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.