In need of your opinions. Currently developing an application in VB.NET.
I have a text file which contains more than one thousand rows. Each rows contains the data needed to be inserted into the database. A sample of a row is as follows:
0001--------SCOMNET--------0.100---0.105
At first glance, one might figured that each column was separated with a tab but actually each column was separated by blank spaces (I used '-' to denote as blank spaces because somehow could not get SO text editor to show the blank spaces).
The first column is defined by
Substring(0, 12) which is the data [0001--------]
second column
Substring(12, 12) in which the data is [SCOMNET-----]
third column is
Substring(24, 8) in which the data is [---0.100]
and last column is
Substring(32, 8) in which the data is [---0.105]
My initial though is to extract the lines for the text file and stored in as a list of strings, then while looping, do the separation of the each string item in the list with the SubString() function and insert it one by one until the end of the list. But this will no doubt takes time.
In my scenario, how can I take advantage of the SqlBulkCopy? Or if there is any other ways to approach this for a faster insert? Say;
- open file
- start loop
- read line
- separate each column in the line with substring
- save in a DataTable
- end loop
- BCP(DataTable)