0

I need to create a data flow for an existing MS SSDT project that inports a flat CSV file into an existing database table. So far so good.

However I would like to reject all entries where the column "code" match values already stored in the db. Even better, if possible, in the case that the column "code" maches an entry in the database, I would like to update the column "description". The important thing is that under no circumstances should duplicate code entries be created.

Thanks

1 Answer 1

0

Ok so seeing as I figured it out, I thought someone else might find it useful:

The short answer is that a lookup is needed between the data source and destinations. This "lookup" will filter between matches that need updating and new values that need to go straight into a new table row (see image). SSDT with lookup

Values that match the database and need updating to the description need to be fed into an "OLE DB command".

Within the "lookup" component we need to do the following:

  1. Go to the general tab and select Redirect rows to no match output
  2. Go to the connection tab and insert the following SQL:

    SELECT id, code FROM tableName
    
  3. Go into the "Columns" tab and check the "id" column on the "Available lookup Columns" table. Also chech the "code" column and drag it to its corresponding "Available Inputs Columns" counterpart to map them to eachother so that the look up can compare them. -- At this point if you get an error caused by the mapping, try to replace the code in setep 2 with:

    SELECT id, CAST(code AS nvarchar(50)) AS code FROM tableName
    
  4. In the Error Output, ensure that id under "Lookup Match Output" has a description of "Copy Column"

Now we need to configure the "OLE DB command" component:

  1. Go to the "Connection Managers" tab and ensure the component is connected to the desired DB
  2. Go to "Component Properties" and add the following code to the "SQLCommand" property:

    UPDATE tableName SET description = ? WHERE id = ?
    

    Note the "?". It is supposed to be there to indicate a parameter must be added to the "Column Mappings" tab, do not replace them.

  3. Finally go into the "Column Mappings" tab and map Param_0 (the first ?) to the "description" column and "Param_1" to the "id" column. No action is needed on the "code" or any other column the db table may contain.

Now give yourself a big pat on the back for having completed a task, that in SQL would normally be one line of code, in about 10 time-consuming steps ;-)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.