Questions tagged [load-data-infile]

A mySQL function for loading external data directly into the database.

load-data-infile
Filter by
Sorted by
Tagged with
375 votes
23 answers
1.3m views

How do I import CSV file into a MySQL table?

I have an unnormalized events-diary CSV from a client that I'm trying to load into a MySQL table so that I can refactor into a sane format. I created a table called 'CSVImport' that has one field for ...
Iain Samuel McLean Elder's user avatar
193 votes
10 answers
209k views

MySQL load NULL values from CSV data

I have a file that can contain from 3 to 4 columns of numerical values which are separated by comma. Empty fields are defined with the exception when they are at the end of the row: 1,2,3,4,5 1,2,3,,...
Spiros's user avatar
  • 2,251
138 votes
15 answers
574k views

Import data in MySQL from a CSV file using LOAD DATA INFILE

I am importing some data of 20,000 rows from a CSV file into MySQL. Columns in the CSV file are in a different order than MySQL tables' columns. How can I automatically assign columns corresponding to ...
MANJEET's user avatar
  • 1,773
130 votes
12 answers
135k views

access denied for load data infile in MySQL

I use MySQL queries all the time in PHP, but when I try LOAD DATA INFILE I get the following error #1045 - Access denied for user 'user'@'localhost' (using password: YES) Does anyone know what ...
Brian's user avatar
  • 27.1k
101 votes
15 answers
276k views

How to import a CSV file into a MySQL table

How can I import a CSV file into a MySQL table? I would like for the first row of data be used as the column names. I read How do I import CSV file into a MySQL table?, but the only answer was to use ...
lcm's user avatar
  • 1,767
71 votes
8 answers
234k views

Importing a csv into mysql via command line

I'm trying to import a very large .csv file (~4gb) into mysql. I was considering using phpmyadmin, but then you have a max upload size of 2mb. Someone told me that I have to use the command line. I ...
de1337ed's user avatar
  • 3,253
70 votes
3 answers
85k views

LOAD DATA LOCAL, How do I skip the first line?

I'm trying to Load a CSV file into my MySQL database, But I would like to skip the first line. I fact It contains the name of my columns and no interesting data. Here is the query I'm using: LOAD ...
Roch's user avatar
  • 21.9k
70 votes
6 answers
93k views

mysql load data infile can't get stat of file Errcode: 2

I have looked all over and found no solution, any help on this would be great. Query: LOAD DATA INFILE '/Users/name/Desktop/loadIntoDb/loadIntoDB.csv' INTO TABLE `tba`.`tbl_name` FIELDS TERMINATED ...
Phill Pafford's user avatar
68 votes
4 answers
88k views

How to skip columns in CSV file when importing into MySQL table using LOAD DATA INFILE?

I've got a CSV file with 11 columns and I have a MySQL table with 9 columns. The CSV file looks like: col1, col2, col3, col4, col5, col6, col7, col8, col9, col10, col11 and the MySQL table looks ...
Camsoft's user avatar
  • 11.9k
55 votes
3 answers
53k views

MySQL LOAD DATA INFILE with ON DUPLICATE KEY UPDATE

For loading huge amounts of data into MySQL, LOAD DATA INFILE is by far the fastest option. Unfortunately, while this can be used in a way INSERT IGNORE or REPLACE works, ON DUPLICATE KEY UPDATE is ...
Jan's user avatar
  • 3,094
46 votes
11 answers
46k views

mysqldump table without dumping the primary key

I have one table spread across two servers running MySql 4. I need to merge these into one server for our test environment. These tables literally have millions of records each, and the reason they ...
Zee Spencer's user avatar
  • 3,488
36 votes
3 answers
40k views

MySQL load data infile - acceleration?

sometimes, I have to re-import data for a project, thus reading about 3.6 million rows into a MySQL table (currently InnoDB, but I am actually not really limited to this engine). "Load data infile..." ...
DBa's user avatar
  • 1,293
35 votes
6 answers
21k views

Switching from MySQL to PostgreSQL - tips, tricks and gotchas?

I am contemplating a switch from MySQL to PostgreSQL. What are your tips, tricks and gotchas for working with PostgreSQL? What should a MySQLer look out for? See also: How different is PostgreSQL ...
Toby Hede's user avatar
  • 36.9k
35 votes
3 answers
57k views

MYSQL LOAD DATA INFILE ignore duplicate rows (autoincrement as primary key)

I ran into some trouble using LOAD DATA INFILE command as i wanted to ignore the lines that was already in the data base..say if i have a table with data as follows, id |name |age -----------------...
Hasitha Shan's user avatar
  • 2,950
30 votes
9 answers
79k views

Easy way to export a SQL table without access to the server or phpMyADMIN

I need a way to easily export and then import data in a MySQL table from a remote server to my home server. I don't have direct access to the server, and no utilities such as phpMyAdmin are installed. ...
Jrgns's user avatar
  • 25k
20 votes
3 answers
59k views

Import CSV to MySQL

I have created a database and a table. I have also created all the fields I will be needing. I have created 46 fields including one that is my ID for the row. The CSV doesn't contain the ID field, nor ...
404error's user avatar
  • 575
20 votes
2 answers
45k views

Mysql permission errors with 'load data'

I am running into a permission error when trying to load data from a flat file database dump into a new table. I know that the schema of the file and my table is the same and I tried tweaking the ...
sutee's user avatar
  • 12.7k
20 votes
2 answers
17k views

mysql load data infile where clause

I need to update existing rows in table with load data infile based on some condition, is this possible? load data infile 'E:/xxx.csv' into table tld_tod @aaa, @xxx_date, @ccc fields terminated by ','...
user avatar
19 votes
2 answers
32k views

Trying to do LOAD DATA INFILE with REPLACE and AUTO_INCREMENT

I am trying to load a file onto a MySQL database, having the primary key auto_incremented and I would like the data to be updated if i find any duplicate rows. However, the REPLACE keywords only works ...
jonasMcFerreira's user avatar
18 votes
2 answers
31k views

MySQL LOAD DATA LOCAL INFILE example in python?

I am looking for a syntax definition, example, sample code, wiki, etc. for executing a LOAD DATA LOCAL INFILE command from python. I believe I can use mysqlimport as well if that is available, so ...
David Perron's user avatar
17 votes
2 answers
45k views

How to change string date to MySQL date format at time of import of CSV using MySQL's LOAD DATA LOCAL INFILE

I'm using MySQL's LOAD DATA LOCAL INFILE SQL statement to load data from a CSV file into an existing database table. Here is an example SQL statement: LOAD DATA LOCAL INFILE 'file.csv' INTO TABLE ...
Camsoft's user avatar
  • 11.9k
16 votes
2 answers
78k views

PHP - Import CSV file to mysql database Using LOAD DATA INFILE

I have a .csv file data like that Date,Name,Call Type,Number,Duration,Address,PostalCode,City,State,Country,Latitude,Longitude "Sep-18-2013 01:53:45 PM","Unknown","outgoing call",'123456',"0 Secs","...
chimbu's user avatar
  • 806
16 votes
5 answers
25k views

mysql LOAD DATA INFILE with auto-increment primary key

I am trying to load a data file into mysql table using "LOAD DATA LOCAL INFILE 'filename' INTO TABLE 'tablename'". The problem is the source data file contains data of every fields but the primary ...
biomed's user avatar
  • 647
15 votes
1 answer
34k views

LOAD DATA from CSV file where doublequote was used as the escape character

I have a bunch of CSV data that I need to load into a MySQL database. Well, CSV-ish, perhaps. (edit: actually, it looks like the stuff described in RFC 4180) Each row is a list of comma-separated ...
rampion's user avatar
  • 88.2k
15 votes
2 answers
11k views

Is it not possible to show warnings when using mysqlimport?

This article: http://www.linuxask.com/questions/how-to-show-the-warnings-during-mysqlimport says it is not possible to show warnings when using mysqlimport: When you use mysqlimport to import data ...
jerrygarciuh's user avatar
  • 21.5k
15 votes
5 answers
34k views

MYSQL: Display Skipped records after LOAD DATA INFILE?

In MySQL I've used LOAD DATA LOCAL INFILE which works fine. At the end I get a message like: Records: 460377 Deleted: 0 Skipped: 145280 Warnings: 0 How can I view the line number of the records ...
Nate Murray's user avatar
  • 3,931
14 votes
2 answers
23k views

MySQL infile ignore header row

Trying to get MySQL to ignore a specific row when importing through an INFILE command. Essentially, it's the "header" row in the CSV file. LOAD DATA LOCAL INFILE 'C:\myfile.txt' REPLACE INTO TABLE ...
Plummer's user avatar
  • 6,580
13 votes
7 answers
20k views

Program for working with large CSV Files [closed]

Are there any good programs for dealing with reading large CSV files? Some of the datafiles I deal with are in the 1 GB range. They have too many lines for Excel to even deal with. Using Access can ...
Kibbee's user avatar
  • 65.9k
12 votes
8 answers
9k views

What is the fastest way to load an XML file into MySQL using C#?

Question What is the fastest way to dump a large (> 1GB) XML file into a MySQL database? Data The data in question is the StackOverflow Creative Commons Data Dump. Purpose This will be used in an ...
AgentConundrum's user avatar
12 votes
1 answer
12k views

MySQL LOAD DATA INFILE - Loading a file with no primary key

I was trying to load a data file into mysql table using "LOAD DATA LOCAL INFILE 'filename' INTO TABLE 'tablename'". The problem is the source data file contains data of every fields but the primary ...
jack's user avatar
  • 17.6k
11 votes
5 answers
22k views

Error 1148 MySQL The used command is not allowed with this MySQL version

I am using MySQL LOAD DATA LOCAL INFILE command and I get this error: PDOException: SQLSTATE[42000]: Syntax error or access violation: 1148 The used command is not allowed with this MySQL version: ...
Chris Muench's user avatar
  • 17.9k
11 votes
1 answer
26k views

How to use LOAD DATA INFILE statement when file is another location?

I want to use LOAD DATA INFILE statement to import data in my table. If the file is available on the same system, it works nicely and imports successfully, but when the file is located on another ...
Sameek Mishra's user avatar
10 votes
3 answers
12k views

How to see progress of .csv upload in MySQL

I have a very large .csv file, and I'm loading it into mysql with the LOAD DATA INFILE command. Because it takes so long, I'd like to see how far along the upload has progressed. I've tried 2 ...
Tucker's user avatar
  • 7,147
10 votes
3 answers
22k views

LOAD DATA LOCAL INFILE gives the error The used command is not allowed with this MySQL version

I have a PHP script that calls MySQL's LOAD DATA INFILE to load data from CSV files. However, on production server, I ended up with the following error: Access denied for user ... (using password: ...
Salman A's user avatar
  • 268k
9 votes
7 answers
25k views

MySQL Convert latin1 data to UTF8

I imported some data using LOAD DATA INFILE into a MySQL Database. The table itself and the columns are using the UTF8 character set, but the default character set of the database is latin 1. ...
Kibbee's user avatar
  • 65.9k
8 votes
6 answers
1k views

What is the best way to achieve speedy inserts of large amounts of data in MySQL?

I have written a program in C to parse large XML files and then create files with insert statements. Some other process would ingest the files into a MySQL database. This data will serve as a ...
Chris Kloberdanz's user avatar
8 votes
2 answers
17k views

MySQL Load Data Infile auto incremented ID value

I want to insert an ID column to my table and get data of this table from a text file. For example my text file is like that: 12 1212 4989 121 121 23 123 110 789 99 234 ...
JoshuaJeanThree's user avatar
8 votes
4 answers
21k views

MySql bulk load command line tool

Does MySql have a bulk load command line tool like bcp for SQLServer and sqlldr for Oracle? I know there's a SQL command LOAD INFILE or similar but I sometimes need to bulk load a file that is on a ...
Mike Q's user avatar
  • 23k
8 votes
2 answers
22k views

infile .csv file in sas with header

I have a csv file which named dataset1.csv and it contains header with 3 variables att1 (character), att2 and att3 (numeric data). I tried following code filename test 'C:\Users\1502911\Desktop\...
useR's user avatar
  • 3,072
8 votes
1 answer
13k views

Importing bulk CSV data in UTF-8 into MySQL

I'm trying to import about 10K rows of UTF-8 encoded data into a new MySQL table. I can do so successfully with LOAD DATA INFILE via MySQL Workbench but it the UTF-8 characters get mangled. I've ...
Tom's user avatar
  • 30.4k
8 votes
5 answers
4k views

Upload 95 GB CSV file into MySQL MyISAM table via Load data infile: CSV engine an alternative?

I'm trying to upload a 95 GB CSV file into a MySQL database (MySQL 5.1.36) via the following command: CREATE TABLE MOD13Q1 ( rid INT UNSIGNED NOT NULL AUTO_INCREMENT, gid MEDIUMINT(6) UNSIGNED NOT ...
Janvb's user avatar
  • 1,290
7 votes
5 answers
11k views

How to speed up a data loading into InnoDB (LOAD DATA INFILE)?

I want to speed up a data loading. I use MySQL 5.5, InnoDB and have 1M rows of data (65Mb file). It takes 5 minutes. What mysql settings and commands affect the speed of LOAD DATA INFILE for InnoDB? ...
Jean Louis's user avatar
  • 1,545
7 votes
5 answers
13k views

LOAD DATA LOCAL INFILE fails - from php, to mysql (on Amazon rds)

We're moving our database from being on the webserver to a separate server (from an Amazon EC2 webserver to an RDS instance.) We have a LOAD DATA INFILE that worked before that is going to need the ...
Redzarf's user avatar
  • 2,598
7 votes
4 answers
19k views

What file and directory permissions are required for MySQL LOAD DATA INFILE?

I have a script that is trying to load some data into MySQL with LOAD DATA INFILE. For some reason, it works if the file is in the /tmp directory, but not if the file is in another directory with ...
Tim Martin's user avatar
  • 3,667
7 votes
2 answers
13k views

LOAD DATA LOCAL INFILE forbidden after php / mariadb update

I know this has been asked before but I could not find any newer posts that could be of help. LOAD DATA LOCAL INFILE has been working perfectly up until today. I have not changed anything other than ...
Asle's user avatar
  • 787
7 votes
9 answers
55k views

MySQL LOAD DATA Error (Errcode: 2 - "No such file or directory")

I am trying to load data into a table of my MySQL database, and getting this error. LOAD DATA LOCAL INFILE 'C:\Users\Myself\Desktop\Blah Blah\LOAD DATA\week.txt' INTO TABLE week; Reference: this ...
Solace's user avatar
  • 8,832
7 votes
1 answer
6k views

Load data infile with blob field

Given I have to dump a lot amount of inserts in a short period managed to dump all records to a file and then loading them by the load data infile sentence of mysql. This was working fine but now I ...
Pabloks's user avatar
  • 1,484
7 votes
2 answers
4k views

Is it possible to use a LOAD DATA INFILE type command to UPDATE rows in the db?

Pseudo table: | primary_key | first_name | last_name | date_of_birth | | 1 | John Smith | | 07/04/1982 | At the moment first_name contains a users full name for many rows. ...
cw84's user avatar
  • 2,150
7 votes
2 answers
6k views

MySQL load data infile loading exactly half the records

I am loading a CSV file using MySQL's LOAD DATA INFILE but only half the rows load. I have tested different files and exactly half of the rows will load every time. How can I get all the rows to load?...
Grasshopper's user avatar
7 votes
3 answers
38k views

SQL Loader : Load into 2 Tables from 1 controlfile and 1 CSV

Table T1 Structure:col1 number,col2 number Table T2 Structure: col1 number,col2 number,col3 number csv file: row1:1,2,3,4,5,6 row2:1,2,3,4,5,6 I want data in Table T1 as: col1 col2 ===== ==== ...
user avatar

1
2 3 4 5
13