0

there is a large csv file need to be import to mysql database
there are to ways bellow

1 , upload this csv file using a php then

<?php
$row = 1;
$handle = fopen("test.csv","r");
while ($data = fgetcsv($handle, 1000, ",")) {
    // record rows to to database
}
fclose($handle);
?> 

this way is easy ,but waiting for a long time

2, upload this csv file using a php
then use ajax to record each row of the csv and output the ajax feedback

<script type="text/javascript">
    var row =0;
    $.ajax({
        url:"csv_to_db.php",    //record only one row
        type:"POST",
        data:{
            row : row;
        },
        success:function(){
            row++;
        }
    });
</script>

this way can see the active feedback ajax info ,and the browser will not die when the method 1 wait too long time
but this way will excute fgetcsv function in "scv_to_db.php" every time, seems waiting the cpu and memery

5
  • 1
    How big is your CSV? I'm currently importing ~30MB csv's with ~25K lines and it works fine (around 10 seconds). May 21, 2013 at 9:33
  • there should be thousands row in the csv , I don't know yet. you import it in only a php only for one time and only using 10 seconds? May 21, 2013 at 9:36
  • 1
    Upload it and save it in a file, then run a CRON that gets all the CSV files in the folder and imports it. Once imported, delete original file or if you want to keep it move it to another folder.
    – We0
    May 21, 2013 at 9:37
  • 1
    As I said, I'm importing 25K lines on a shitty server and it works fine. Your 2nd. option sucks, because you're calling fopen() after every ajax call. You could cache the fopen() result and you should inset rows in a bulk (not 1 by 1) May 21, 2013 at 9:40
  • thank you , i think i should use method 1 such as what you did May 21, 2013 at 9:44

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.