14

I need to migrate a database from Postgres 7 to SQL Server 2008. I am familiar with the SSIS Import and Export wizard but I am stumped about how to define the data source or define the data provider.

What is the best way to migrate Postgres to SQL Server, and how do I define data sources/drivers for postgres?

4
  • You'd choose SQL Server Native Client from the DataSources dropdown. I'm guessing Postgres needs the odbc provider but I don't know how to implement it Sep 14, 2010 at 16:58
  • How are you connecting to Postgres currently? You don't have an ODBC data source for postgres set up yet? Yeah, you need that.
    – JohnB
    Sep 14, 2010 at 17:01
  • Question 1: Why on earth would anybody want to do this ? Doing that is not only economic suicide, it's technical suicide as well. Oct 21, 2015 at 17:15
  • you can use my answer here: stackoverflow.com/questions/6563846/… Dec 6, 2021 at 5:49

4 Answers 4

13

I was having problems using the Import Wizard in SQL Server 2008 R2 to import tables from PostgreSQL. I had the PostgreSQL ODBC driver installed, so for the Data Source in the Import Wizard I chose ".Net Framework Data Provider for Odbc" and supplied the DSN name for my PostgreSQL database. The wizard found the tables okay, but when I went to perform the import I got the error

Column information for the source and destination data could not be retrieved.

“Billing” -> [dbo].[Billing]:

– Cannot find column -1.

I found the solution in the Microsoft blog post here. Apparently the problem is that various ODBC drivers use different attribute names when reporting column metadata. To get the import to work I had to edit the "ProviderDescriptors.xml" file, which was located at

C:\Program Files\Microsoft SQL Server\100\DTS\ProviderDescriptors\ProviderDescriptors.xml

In the ...

<dtm:ProviderDescriptor SourceType="System.Data.Odbc.OdbcConnection">

... element I had to change the attributes from ...

<dtm:ColumnSchemaAttributes
    NameColumnName = "COLUMN_NAME"
    OrdinalPositionColumnName="ORDINAL_POSITION"
    DataTypeColumnName = "TYPE_NAME"
    MaximumLengthColumnName = "COLUMN_SIZE"
    NumericPrecisionColumnName = "COLUMN_SIZE"
    NumericScaleColumnName = "DECIMAL_DIGITS"
    NullableColumnName="NULLABLE"
    NumberOfColumnRestrictions="4"
/>

... to ...

<dtm:ColumnSchemaAttributes
    NameColumnName = "COLUMN_NAME"
    OrdinalPositionColumnName="ORDINAL_POSITION"
    DataTypeColumnName = "TYPE_NAME"
    MaximumLengthColumnName = "LENGTH"
    NumericPrecisionColumnName = "PRECISION"
    NumericScaleColumnName = "SCALE"
    NullableColumnName="NULLABLE"
    NumberOfColumnRestrictions="4"
/>

That is, I had to tweak the MaximumLengthColumnName, NumericPrecisionColumnName, and NumericScaleColumnName attribute values to "LENGTH", "PRECISION", and "SCALE", respectively.

Once that change was made the import from PostgreSQL to SQL Server ran successfully.

5
  • 1
    Thanks for the insight!! I never did get it to work! Nov 19, 2016 at 13:30
  • I had the same issue. The ProviderDescriptors.xml resides in both under Program Files and Program Files(x86). Typically SQL Management Studio is 32bit, and runs the DTS wizard as 32bit. So change the file in the (x86) tree. Only got it to work under 32bit Import/export program. 64bit Import/Export never did work properly even after changing the file in the other location.
    – Derek Wade
    Nov 6, 2018 at 13:19
  • 4
    Update: 64bit does also work, but don't make a "copy" of the file in the same directory. all files (even if not named *.xml) are loaded and can override your changes.
    – Derek Wade
    Nov 6, 2018 at 16:16
  • This worked for me, thanks a lot! And I did make copies @DerekWade, but it didn't seem to collide. Maybe they were processed in the right order.
    – GerardV
    May 15, 2019 at 19:03
  • Removing a backup copy (which in my case was ProviderDescriptors.xml~) helped. Thus the order is important.
    – Vertigo
    Oct 26, 2020 at 8:14
11

I wish you the best of luck in trying to import from PostgreSQL into SQL Server using SQL Server Import and Export Wizard. However, I have read numerous message board threads with people having trouble getting it to work. For example:

Here is the most helpful thread that I have found on the topic:


To help someone who might be trying to achieve similar goal as mine. Instead of selecting the “PostgreSQL OLE DB Provider” in the data source drop down menu of SQL Server Import and Export Wizard, select “.Net Framework Data Provider for Odbc”

Then you have to make a DSN and provide a ConnectionString. Following ConnectionString worked for me

Driver={PostgreSQL};Server=localhost;Port=5432;Database=TestMasterMap;Uid=postgres;Pwd=;

To make a DSN you have to go into Administrative Toolsà Data Sources (ODBC) and create a user DSN. Once this is done you can supply the DSN name in the DSN text box of SQL Server Import and Export Wizard.


One commenter claimed that it worked, but that he got "Out of memory while reading tuples" errors on big tables. So for tables with more than 3 million rows, he had to break the import up into 3 million row chunks.

Also, there's a link to the native .NET provider for PostgreSQL in that thread.

Personally, if this is something that I only had to do once, and if I understood the schema and the data fairly well, I would try:

  1. export the data from PostgreSQL as flat files
  2. create the schema in SQL Server (without PKs or constraints)
  3. use the SSIS Import/Export Wizard to import the flat files
  4. then create PKs and necessary constraints

It might take you less time to do the above than messing with SSIS Import/Export Wizard and PostgreSQL for days (but it would be nice if those tools worked!)

2
  • I just tried to import my tables as CSV files to SQL Server 2005 and noticed that the CSV import is buggy as hell. I just couldn't make it work due to various issues. PostgreSQL imported the same files without any problems just like that.
    – juzzlin
    Mar 20, 2014 at 17:13
  • I like the idea of exporting the data to CSV. Working with SSIS directly didn't work well for me, specially because the data I wanted to export was organized in views, which the odbc driver (or SSIS itself) doesn't seem to pick up.
    – Andre
    Sep 20, 2016 at 20:59
1

As I finished commenting the answer above, I thought of trying SQL WorkbenchJ; it has a datapump feature that worked pretty well for me. I managed to export data from my PostgreSQL database to an SQL server instance.

Those who'd like to run this in batch mode (via shell), here's how to do it: Google Groups Thread. The WbCopy command mentioned on the discussion isn't really documented anywhere I could find, but you can generate one through the datapump interface and then change whatever you need.

0

To give a little more practical example of how you can achieve what's described in marked answer; you can export from PostgresQL to flat files then use bcp Utility to import in SQL Server. e.g. in a .bat file, for a single table (and you need to have the table already created in the destination SQL DB):

@echo off

set DbName=YOUR_POSTGRES_DB_NAME
set csvpath=C:\PATH_TO_CSV\CSV_NAME.csv
set username=YOUR_POSTGRES_DB_USERNAME

:: Export to CSV, note we're using a ~ delimiter to avoid issues with commas in fields
psql -U %username% -d %DbName% -c "COPY (select * from SOURCE_TABLE_NAME) TO STDOUT (FORMAT CSV, HEADER TRUE, DELIMITER '~', ENCODING 'UTF8');" > %csvpath%

:: Import CSV to SQL Server
set logpath=C:\bcplog.txt
set errorlogpath=C:\bcperrors.txt
set sqlserver=YOUR_SQL_SERVER
set sqldb=YOUR_DB_NAME

:: page code 65001 = UTF-8
bcp DESTINATION_TABLE_NAME IN %csvpath% -t~ -F1 -c -C65001 -S %sqlserver% -d %sqldb% -T -o %logpath% -e %errorlogpath%

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.