0

I'm new to VS and SSIS. I'm trying to consolidate data that I have in visual studio using the Script Component Transformation Editor that currently looks like this:

date type seconds

2016/07/07 personal 400

2016/07/07 business 300

2016/07/07 business 600

and transform it to look like this:

date type avgSeconds totalRows

2016/07/07 personal 400 1

2016/07/07 business 450 2

Basically, counting and taking the average for type and date. I tried both VB.net and C# options and am open to either (new to both). Does anyone have any ideas on how i can do this?

I'm using VS 2012. I'm thinking i need to create a temp or buffer table to keep count of everything as I go through the Input0_ProcessInputRow and then write it to the output at the end, I just can't figure out how to do this. Any help is much appreciated!!

5
  • Is there any reason you want to use a script over an execute sql task? This looks like a simple query using AVG and Group By.
    – Joe C
    Jul 12, 2016 at 16:41
  • yeah, I'm grabbing data from two separate places and joining it together.
    – tjayne13
    Jul 12, 2016 at 17:09
  • How about loading that joined data to a staging table then executing an aggregate query against it? I am thinking of ways to avoid the coding and potential memory issues of using a procedural script.
    – Joe C
    Jul 12, 2016 at 17:15
  • Can I do that all inside Visual Studio? and if so, how? Sorry for all the questions, I'm really new to this program
    – tjayne13
    Jul 12, 2016 at 17:22
  • I posted an answer based on your comments.
    – Joe C
    Jul 12, 2016 at 17:32

1 Answer 1

0

Based on your comments this might work.

Set up a data flow task that will do a merge join to combine the data https://www.simple-talk.com/sql/ssis/ssis-basics-using-the-merge-join-transformation/

Send the data to a staging table. This can be created automatically by the oledb destination if you click 'New' next to table drop down. If you plan to rerun this package then you will need to add an execute sql task before the data flow task to delete from or truncate this staging table.

Create an execute sql task with your aggregation query. Depending on your situation an insert could work. Something like:

Insert ProductionTable
Select date, type, AVG(Seconds) avgSeconds, Vount(*) totalRows
   From StagingTable
   Group By date, type

You can also use the above query, minus the insert production table, as a source in a data flow task in case you need to apply more transforms after the aggregation.

2
  • okay great thanks, Im going to try this. I'm guessing ill need to create an oledb database to put the staging table in?
    – tjayne13
    Jul 12, 2016 at 17:46
  • Hopefully you can create a table in the destination database. Otherwise you can look into using a temp table. stackoverflow.com/questions/5631010/…
    – Joe C
    Jul 12, 2016 at 17:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.