0

I am trying to import a text file to a table that has additional fields as well that I need to populate at the same time. The fields are trans_type_code (varchar), raw_record_status (varchar), import_file_id (int), and import_file_line (int). The first three fields don't change per imported record but the fourth one obviously does. I'm executing the stored procedure to import the data from VBS, passing only the import_file_id.

I first considered specifying the destination fields for the imported text using BULK INSERT and a FORMATFILE, but I couldn't figure out how to populate the other fields at the same time. I also didn't know how to get the line number from the import file.

I've created a process using bulk insert to a temp table and then select from the temp table to insert into import_raw_records to populate the first four fields listed above, but not all of the import records have the same number of fields. If I create the temp table with the maximum number of possible fields for the imported data, the data in the text file ends up filling up every field without wrapping to the next record based on the ROWTERMINATOR = \n that I'm using. Records with only four fields imported will fill up 20 fields in the temp table in a single record rather than creating five separate records in the temp table.

I confirmed that the CRLFs in my import file are indeed 0D0A. This is the code that is importing the data: set @sqlcmd = ' BULK INSERT #temp_import_records FROM ''' + @import_file + ''' WITH ( ROWTERMINATOR = ''\n'' ) '

What is the best way to deal with this? Should I be approaching this differently? I'm not exactly a newbie, but I still have an awful lot to learn about SQL. Thanks for reading this and I hope you can help.

Thanks, John

2
  • It sounds like your real problem is that your source file doesn't have good delimiters. I would post that as a question with some sample data.
    – David
    Nov 20, 2013 at 20:07
  • @David, thanks for helping me narrow this down. I'll do that too, but is this sound methodology for what I'm trying to do, or is there a better way? Nov 20, 2013 at 21:23

1 Answer 1

0

Yes, I would load the data in to a temp table that has enough columns to fit the widest record. Then you can use SQL to move data from there in to the destination tables.

Note that "BULK INSERT" does not handle escape characters so if you save one cell that contains an embed line feed, Excel will handle it fine, but "BULK INSERT" will treat it as a record delimiter and insert 2 rows.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.