Anyone know a quick easy way to migrate a SQLite3 database to MySQL?
Join Stack Overflow to learn, share knowledge, and build your career.
|
Here is a list of converters (not updated since 2011): An alternative method that would work nicely but is rarely mentioned is: use an ORM class that abstracts specific database differences away for you. e.g. you get these in PHP (RedBean), Python (Django's ORM layer, Storm, SqlAlchemy), Ruby on Rails (ActiveRecord), Cocoa (CoreData) i.e. you could do this:
|
||||
|
|
|
Everyone seems to starts off with a few greps and perl expressions and you sorta kinda get something that works for your particular dataset but you have no idea if it's imported the data correctly or not. I'm seriously surprised nobody's built a solid library that can convert between the two. Here a list of ALL the differences in SQL syntax that I know about between the two file formats: The lines starting with:
are not used in MySQL
Here is a very basic hacked up perl script which works for my dataset and checks for many more of these conditions that other perl scripts I found on the web. Nu guarentees that it will work for your data but feel free to modify and post back here.
|
|||||||||||||||||||||
|
|
Here is a python script, built off of Shalmanese's answer and some help from Alex martelli over at Translating Perl to Python I'm making it community wiki, so please feel free to edit, and refactor as long as it doesn't break the functionality (thankfully we can just roll back) - It's pretty ugly but works use like so (assuming the script is called
Which you can then import into mysql note - you need to add foreign key constrains manually since sqlite doesn't actually support them here is the script:
|
|||||||||||||||||||||
|
|
It's messy because dump files are database vendor specific. If you're using Rails, a great plugin exists for this. Read: http://blog.heroku.com/archives/2007/11/23/yamldb_for_databaseindependent_data_dumps/ UpdateCurrently maintained fork: https://github.com/ludicast/yaml_db |
||||
|
|
|
Surprised no one's mentioned this by now, but there's actually a tool explicitly for this. It's in perl, SQL:Translator: http://sqlfairy.sourceforge.net/ Converts between most any form of tabular data (Different SQL formats, Excel spreadsheet), and even makes diagrams of your SQL schema. |
|||||
|
|
||||
|
|
Get a SQL dump
Import dump to MySQLFor small imports:
or
This will prompt you for a password. Please note: If you want to enter your password directly, you have to do it WITHOUT space, directly after
For larger dumps:mysqlimport or other import tools like BigDump. BigDump gives you a progress bar:
|
|||||
|
|
I recently had to migrate from MySQL to JavaDB for a project that our team is working on. I found a Java library written by Apache called DdlUtils that made this pretty easy. It provides an API that lets you do the following:
The tools that we ended up with weren't completely automated, but they worked pretty well. Even if your application is not in Java, it shouldn't be too difficult to whip up a few small tools to do a one-time migration. I think I was able to pull of our migration with less than 150 lines of code. |
|||
|
|
|
Probably the quick easiest way is using the sqlite .dump command, in this case create a dump of the sample database.
You can then (in theory) import this into the mysql database, in this case the test database on the database server 127.0.0.1, using user root.
I say in theory as there are a few differences between grammars. In sqlite transactions begin
MySQL uses just
There are other similar problems (varchars and double quotes spring back to mind) but nothing find and replace couldn't fix. Perhaps you should ask why you are migrating, if performance/ database size is the issue perhaps look at reoginising the schema, if the system is moving to a more powerful product this might be the ideal time to plan for the future of your data. |
|||||||||
|
|
I've just gone through this process, and there's a lot of very good help and information in this Q/A, but I found I had to pull together various elements (plus some from other Q/As) to get a working solution in order to successfully migrate. However, even after combining the existing answers, I found that the Python script did not fully work for me as it did not work where there were multiple boolean occurrences in an INSERT. See here why that was the case. So, I thought I'd post up my merged answer here. Credit goes to those that have contributed elsewhere, of course. But I wanted to give something back, and save others time that follow. I'll post the script below. But firstly, here's the instructions for a conversion... I ran the script on OS X 10.7.5 Lion. Python worked out of the box. To generate the MySQL input file from your existing SQLite3 database, run the script on your own files as follows,
I then copied the resulting dumped_sql.sql file over to a Linux box running Ubuntu 10.04.4 LTS where my MySQL database was to reside. Another issue I had when importing the MySQL file was that some unicode UTF-8 characters (specifically single quotes) were not being imported correctly, so I had to add a switch to the command to specify UTF-8. The resulting command to input the data into a spanking new empty MySQL database is as follows:
Let it cook, and that should be it! Don't forget to scrutinise your data, before and after. So, as the OP requested, it's quick and easy, when you know how! :-) As an aside, one thing I wasn't sure about before I looked into this migration, was whether created_at and updated_at field values would be preserved - the good news for me is that they are, so I could migrate my existing production data. Good luck! UPDATE Since making this switch, I've noticed a problem that I hadn't noticed before. In my Rails application, my text fields are defined as 'string', and this carries through to the database schema. The process outlined here results in these being defined as VARCHAR(255) in the MySQL database. This places a 255 character limit on these field sizes - and anything beyond this was silently truncated during the import. To support text length greater than 255, the MySQL schema would need to use 'TEXT' rather than VARCHAR(255), I believe. The process defined here does not include this conversion. Here's the merged and revised Python script that worked for my data:
|
|||||
|
|
The python script worked after a few modifications as follows:
...
...
... |
|||||
|
|
If you are using Python/Django it's pretty easy: create two databases in settings.py (like here https://docs.djangoproject.com/en/1.11/topics/db/multi-db/) then just do like this:
|
|||
|
|
|
I use data loader for migrating almost any data, it helps me to convert MSSQL to MYSQL, MS access to MSSQL, mysql, csv loader, foxpro and MSSQL to MS access, MYSQl, CSV, foxpro etc. In my view this is a best Data Migration Tool Download Free : http://www.dbload.com |
|||
|
|
|
Based on Jims's solution: Quick easy way to migrate SQLite3 to MySQL?
This works for me. I use sed just to throw the first line, which is not mysql-like, but you might as well modify dump.py script to throw this line away. |
|||||||||||||
|
|
MySQL Workbench (GPL license) migrates from SQLite very easily via the Database Migration wizard. Installs on Windows, Ubuntu, RHEL, Fedora and OS X. |
|||
|
|
|
Ha... I wish I had found this first! My response was to this post... script to convert mysql dump sql file into format that can be imported into sqlite3 db Combining the two would be exactly what I needed: When the sqlite3 database is going to be used with ruby you may want to change:
to:
alas, this only half works because even though you are inserting 1's and 0's into a field marked boolean, sqlite3 stores them as 1's and 0's so you have to go through and do something like:
but it was helpful to have the sql file to look at to find all the booleans. |
||||
|
|
|
fallino correctly identified the location of the error in the script. I have the solution. The problem is the following lines:
The replacement pattern (2nd parameter) in the re.sub calls is a "regular" string, so instead of \1 expanding to the first regexp match, it expands to a literal 0x01. Likewise, \2 expands to 0x02. For example, a line containing:
,'t','f',
would be replaced with:
<0x01>10<0x02> The fix is to either change the replacement strings by adding an 'r' prefix, or by escaping the \1 and \2 in the existing string. Since easy manipulation of regexp strings is what raw strings are for, here's the fix using those:
|
|||||
|
|
this software out of the box - works for me. try it and let other know. https://dbconvert.com/sqlite/mysql/ In addition: I had to make one small change: somehow the auto_increment of one field (a field found from error message) was not enabled. So in phpmyadmin i check property A_I of this field and it works completely. Hope it helps. Dunn. |
|||||
|
|
There is no need to any script,command,etc... you have to only export your sqlite database as a I used it and it worked amazing... |
|||
|
This script is ok except for this case that of course, I've met : INSERT INTO "requestcomparison_stopword" VALUES(149,'f'); INSERT INTO "requestcomparison_stopword" VALUES(420,'t'); The script should give this output : INSERT INTO requestcomparison_stopword VALUES(149,'f'); INSERT INTO requestcomparison_stopword VALUES(420,'t'); But gives instead that output : INSERT INTO requestcomparison_stopword VALUES(1490; INSERT INTO requestcomparison_stopword VALUES(4201; with some strange non-ascii characters around the last 0 and 1. This didn't show up anymore when I commented the following lines of the code (43-46) but others problems appeared:
This is just a special case, when we want to add a value being 'f' or 't' but I'm not really comfortable with regular expressions, I just wanted to spot this case to be corrected by someone. Anyway thanks a lot for that handy script !!! |
||||
|
|
|
I wrote this simple script in Python3. It can be used as an included class or standalone script invoked via a terminal shell. By default it imports all integers as NOTE: It requires MySQL Connector/Python 2.0.4 or higher Here's a link to the source on GitHub if you find the code below hard to read: https://gist.github.com/techouse/4deb94eee58a02d104c6
|
|||||||||
|
|
This simple solution worked for me:
|
|||
|
|
|
I have taken the Python script from https://stackoverflow.com/a/32243979/746459 (above) and fixed it to cope with our own sqlite schemas. There were a few issues to deal with. You may find it in source control here: https://bitbucket.org/mjogltd/sqlite3mysql Also available is the same thing wrapped as a Docker image, here: https://hub.docker.com/r/mjog/sqlite3mysql/ - it is fully usable even under a Windows desktop. |
||||
|
|
|
I have carefully checked all the answers in this post, as well as the answers in another related post Translating Perl to Python. Yet none could fully solve my problem. My scenario is I need to migrate a database of Trac from sqlite to MySQL, and the database contains a lot of tech-based wiki content. Therefore inside the Eventually I have written my own tool for this purpose: https://github.com/motherapp/sqlite_sql_parser The usage is relatively simple:
Two files would be generated: One could do further manual modifications on the DB schema file using any text editor, without worrying about changing the content. Hope it could helps others in future. |
||||
|
|
watch out for CREATE statements |
||||
|
|
protected by vaultah Jan 5 '16 at 8:29
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?
