Questions tagged [sqlbulkcopy]

Lets you efficiently bulk load a SQL Server table with data from another source.

Filter by
Sorted by
Tagged with
1 vote
1 answer
72 views

DataTable and big SqlDecimal [closed]

I just found out that when you use DataTable and SqlBulkCopy in an attempt to load a SQL Server table, you can't use big decimal values. DataTable only supports the .NET decimal type. .NET decimal is ...
Alsin's user avatar
  • 1,556
2 votes
3 answers
71 views

Error when importing data from Excel to SQL Server with CLR

SQL DBA here, not a C# or a programmer guy. Below is some code I found online and tried to adapt to my needs: public partial class StoredProcedures { [Microsoft.SqlServer.Server.SqlProcedure] ...
Stackoverflowuser's user avatar
1 vote
1 answer
38 views

implement bulk copy in different environment

I have a problem about bulk copy command in PostgreSQL. For instance i have a sample command of copying file CSV to table like below CREATE TEMPORARY TABLE AlterTable ( a1 varchar(50), a2 ...
proram013's user avatar
1 vote
1 answer
262 views

504 Gateway Timeout: STRANGE!!! while server accepting the file into DB the response is not making it to the Client

I have service that uploads and parses line by line into DataTable and records it via SQL bulk copy into the DB. Locally this service works fine with overall implementation in 20 secs and in LAN dev ...
Nara Omur's user avatar
0 votes
2 answers
60 views

WriteToServerAsync Not working as expected when Upgrade to .net 6

My current project was in .net 5 which I have upgraded to .net 6 version. Earlier the code was working fine but after upgrade this code seems to be not working. public async Task AddBulkAsync(...
Roshmi Augustine's user avatar
0 votes
0 answers
154 views

SqlBulkCopy WriteToServer using a DataTable just does nothing

I'm trying to use SqlBulkCopy for inserting bigger amounts of data fast. For that I populate a DataTable with the data I want to insert (up to about 100,000 rows) and call the WriteToServer method. ...
bomberman2910's user avatar
0 votes
0 answers
46 views

Get primary keys after inserting data into SQL Server database using SqlBulkCopy [duplicate]

I'm using SqlBulkCopy to insert records into a SQL Server database. I have a requirement to save the data into 3 tables based on the primary key which I get after inserting into 1st table. As the ...
Sri Ram Kumar's user avatar
0 votes
0 answers
110 views

Why would SqlBulkCopy be extremely slow sometimes?

I'm trying to diagnose a performance issue with SqlBulkCopy. I have a point in my code where most of the time, one row is being inserted, but sometimes hundreds of rows can be inserted, most of the ...
Eduardo Wada's user avatar
  • 2,616
0 votes
0 answers
68 views

Need to improve speed of processing

"I have a script of insert and update in a folder which is executed by window service. In the window service, there is a function that will check and get the batch of 10 files and executes and ...
Manjay's user avatar
  • 19
0 votes
1 answer
447 views

BULK INSERT into Managed Instance from a file in Azure Files?

How can I use BULK INSERT to load files that are in Azure Files, from a Managed Instance? Everything I can find talks about Azure Blob storage, but I've got an old Perl script that we don't want to ...
mbourgon's user avatar
  • 1,306
0 votes
1 answer
267 views

Powershell, SQL Server, SQLBULKCOPY: Syntax to specify codepage or encoding

I'm using a Powershell script found at ((https://blog.netnerds.net/2015/01/powershell-high-performance-techniques-for-importing-csv-to-sql-server/) to load data from quotes delimited, comma separated ....
Bruce Foxton's user avatar
0 votes
1 answer
200 views

SqlBulkCopy failing when trying to handle more than 1 row

I'm trying to perform a bulk upload operation to a SQL Server database. I'm working in .NET 7, and I'm using a SqlBulkCopy instance to upload my data as a DataTable. I've had all sorts of issues ...
Simon Painter's user avatar
1 vote
1 answer
186 views

How to improve SqlBulkCopy performance with Azure SQL

When I run a program on local DB to insert 100,000 records. It take around 2 seconds to import. After switch to Azure SQL the time increase with high percentage of DTU. This is my code using (var ...
Weanich Sanchol's user avatar
1 vote
1 answer
496 views

Copy contents of the Excel file from a stream asynchronously using SqlBulkCopy

I have a project where I need to copy the contents of the .xlsx file I received in Web API Controller (in the form of the Stream from MultipartReader) to SQL Server Database. I'm using SqlBulkCopy for ...
JohnDiGriz's user avatar
1 vote
1 answer
223 views

Powershell, SQL Server, SQLBULKCOPY: Importing NULL Values in .csv

I'm using a Powershell script found at ((https://blog.netnerds.net/2015/01/powershell-high-performance-techniques-for-importing-csv-to-sql-server/) to load data from quotes delimited, comma separated ....
Bruce Foxton's user avatar
2 votes
1 answer
277 views

C# Using foreach loop to bulkcopy data to SQL Server

I have a C# console app which reads million source data from a CSV file and inserts them into SQL Server in batches. I group the data by 1000 count and use foreach to loop the groups. Each loop ...
howard.huang's user avatar
1 vote
0 answers
68 views

SQL Bulk Copy by mapping columns dynamically does not work but works manually

I am trying to do a columnMappings for a bulk insert but i cant get the Columnmapping to work in a for loop but when i do it manually it works just fine. I get this error when trying a dynamic ...
E S's user avatar
  • 105
0 votes
1 answer
134 views

Cannot remove columns of DataTable before using SqlBulkCopy

I want to remove columns of DataTable that have name start with "##" then bulk to database using C# and SQL Server. This is my code: private bool WriteData(DataTable _dt) { DataTable dt =...
buithienquyet's user avatar
0 votes
1 answer
371 views

SqlBulkCopy.WriteToServerAsync() does not write to target SQL Server table, bulkCopy.WriteToServer() does

Just as the title states. I am trying to load a ~8.45GB csv file with ~330 columns (~7.5 million rows) into a SQL Server instance, but I'm doing the parsing internally as it the file has some quirks ...
LSM07's user avatar
  • 787
0 votes
0 answers
203 views

SqlBulkCopy from Oracle to SQL Server with DataReader is slow (100k to 20M)

I have been working on sending a table from Oracle to SQL Server with SqlBulkCopy. I am doing it by reading with a datareader from oracle and then streaming into SQL Server. I have tried with ...
Baldie47's user avatar
  • 1,168
2 votes
0 answers
81 views

Timeout exception and 'Cannot access destination table #name' on bulk.WriteToServer(readerTabel) with small timeout /SQL wait time/block

I have a problem with SqlBulkCopy that sometimes(pretty often) i get a timeout error and an sql error 'Cannot access destination table #temp'. My timeout on bulkcopy is 5 seconds. After further ...
Sabin Alexandru's user avatar
0 votes
0 answers
100 views

c# Save JSON response to Database table

Ok I tried this for a couple of days now. I'm trying to read JSON response and insert/update it to DB table (already exists) based on a condition from the response. I have created a Console app. What ...
StackUseR's user avatar
  • 908
0 votes
0 answers
65 views

{"Failed to convert parameter value from a Object[] to a Decimal."} in SqlBulkCopy using SqlParameter [duplicate]

I need to insert bulk data into a SQL Server database. The data is fetched in SqlParameter. foreach (DataColumn c in data.Columns) { SqlDbType dbType = GetSqlDbTypeFromDotnetType(c.DataType); ...
Sakul Bhatia's user avatar
1 vote
1 answer
124 views

How to get acceptable performance saving data tables to database using table adapter Update (update in batches)?

I am having some performance issues updating a large datatable to SQL server using the table adapter functionality. I am doing exactly what this examples is doing: https://learn.microsoft.com/en-us/...
Nicholas DiPiazza's user avatar
2 votes
1 answer
392 views

How to get newly inserted Ids from identity primary key column when using SqlBulkCopy

I'm using C# and .NET Core 6. I want to bulk insert about 100 rows at once into database and get back their Ids from BigInt identity column. I have tried lot of different variants, but still do not ...
Vitaliy's user avatar
  • 100
0 votes
1 answer
108 views

SSIS C# SQLBulkCopy .csv file Error: Failed to convert parameter value from a String to a Boolean. String not recognized as a valid Boolean

I am trying to create a generic module that will load .csv files into SQL tables. The SQL tables are already created and their names, and the name of the file, will be passed as parameters. This what ...
MChalut's user avatar
  • 57
0 votes
1 answer
85 views

Load of generic .csv files into SQL Table using BulkCopy

I am trying to create a generic module that will load .csv files into SQL tables. The SQL tables are already created and their names, and the name of the file, will be passed as parameters. This ...
MChalut's user avatar
  • 57
0 votes
1 answer
259 views

Using Powershell to copy large Oracle table to SQL Server - memory issue

We are trying to copy data from a large Oracle table (about 50M rows) to SQL Server, using Powershell and SQLBulkCopy. The issue with this particular Oracle table is that it contains a CLOB field and ...
Alex V's user avatar
  • 315
0 votes
0 answers
176 views

SqlBulkCopy Slow inserting with async (from Oracle to SQL Server)

I'm trying to do an insert into SQL Server from Oracle using SqlBulkCopy, but it is too slow. Here are some details: SQL Server 2014 with recovery model "Simple" (clean database for this). ...
Baldie47's user avatar
  • 1,168
1 vote
1 answer
1k views

SqlBulkCopy in C# for copy from Oracle view to SQL Server table too slow

I'm trying to use SqlBulkCopy to copy from Oracle to SQL Server, I have a view in oracle with 12 million records so I'm looking for a fast way to copy. I've done some analysis and I would be able to ...
Baldie47's user avatar
  • 1,168
0 votes
1 answer
2k views

How to use SqlBulkCopy.WriteToServer to Bulk Insert rows into SQL Server , ONLY if Not Exists

We are using SqlBulkCopy.WriteToServer to bulk insert to SQL Server and works very well. However, it fails when a record already exists. What we need is to "ignore" those rows that already ...
Khodr's user avatar
  • 121
2 votes
0 answers
74 views

Is it possible to execute multiple "COPY FROM" commands from a file?

Via pgAdmin, I created a backup file with the following format: COPY alpha (columns) FROM stdin; data TAB data TAB ... data TAB data TAB ... data TAB data TAB ... \. COPY beta (columns) FROM stdin; ...
water_enjoyer's user avatar
0 votes
0 answers
129 views

I am trying to save a DataTable to SQL Server using SqlBulkCopy() but I get an exception

Here is my code to save the DataTable to SQL Server: SqlBulkCopy bcp = new SqlBulkCopy(con); bcp.DestinationTableName = "Hawalas"; ListtoDataTableConverter converter = new ...
Alikhan_Salehi's user avatar
1 vote
1 answer
1k views

Importing CSV to SQL Server using bulkcopy

I'm trying to import my CSV files into SQL Server. I found this code and it works perfectly and very fast: # Database variables $sqlserver = "servername" $database = "datebasename" ...
Faenno's user avatar
  • 27
1 vote
1 answer
188 views

Bulk insert from CSV - country codes are not coming to SQL table

I have a simple CSV file including 60 rows of data like below: "Name","City","Region","Primary Phone","Postal Code" "MyCompany Inc.","...
Eray Balkanli's user avatar
0 votes
0 answers
103 views

WriteToServer does not throw exception if destination table does not exist in database

I'm using SqlBulkCopy to bulk insert data in C#, but it's not throwing an exception if the destination table is not present in the database. Here is my code: SqlConnection connection = new ...
SANDEEP's user avatar
  • 1,062
0 votes
0 answers
153 views

SQL bulk copy C#

I have an ASP.NET application to insert an excel file as a bulk I have an Excel file containing 50,000 records. I am using SQLBulkCopy to insert it. I can insert it successfully from the development ...
Net Flex's user avatar
0 votes
1 answer
578 views

Error 2200: Copy Activity - Partitioned Parquet Column Not Found

Things I've tried: I have checked the pipeline's JSON and none of the columns have a space in them. I added in the "physicalType: "UTF8"" to the JSON as it was missing beforehand. ...
Mariah Akinbi's user avatar
0 votes
1 answer
298 views

How can I make AllowDBNull work for GUID?

I am using DataColumn to create a new column for a DataTable. I am using AllowDBNull to specify that a column can have NULL values. This works fine except for the case when I have a column with ...
oyeesh's user avatar
  • 551
0 votes
0 answers
193 views

SqlBulkCopy when column types are unknown

I am working on a C# app that imports csv data to SQL server. I am using CsvHelper to achieve this. The use case here is that the app will be used by anyone who wishes to import data from CSV to SQL ...
oyeesh's user avatar
  • 551
0 votes
0 answers
359 views

Export Data from Sql Server to To Excel Using Sqlbulkcopy

is any one here have the idea to export/copy data from sql server to excel. I use OPENROWSET and its working but the client wants to get the data using bulk copy is there a way to get the data using ...
Regime Evangelista Lesmoras's user avatar
0 votes
3 answers
900 views

SqlBulkCopy WriteToServer with an IDataReader instead of DataTable and Programmatically Adjusted Field Values

We have a working code in C# that utilizes SqlBulkCopy to insert records into a table from a stored procedure source. At a high-level: Reads data from a stored procedure that puts the records into a ...
robertviper08's user avatar
1 vote
1 answer
292 views

SQL Insert statements in C# on Azure DB - Running very slow

I'm working on an importer in our web application. With the code I currently have, when you are connecting via local SQL server, it runs fine and within reason. I'm also creating a .sql script that ...
tvb2754's user avatar
  • 127
1 vote
0 answers
1k views

Using SQL Bulkcopy based on Db contexts connection

In my .NET Core project (.NET 5) I'm using EF Core 6 and for SQL auth, it's using Azure AD Identity authentication. Here is my startup where DB context is initialized: services.AddDbContext<...
Sandeep Thomas's user avatar
-1 votes
1 answer
98 views

Prevent SqlBulkCopy to change XML value

I am using SqlBulkCopy to transfer data from one database to another. My problem is that my XML column will be changed during the bulk copy. I think that the SqlBulkCopy class is parsing/computing the ...
Robert Wolf's user avatar
2 votes
3 answers
820 views

Bulk Insert with NTS geometries in .NET Core

As you probably know, inserting data into a table the "Entity Framework"-way is incredibly slow when you try to insert a lot of rows. One other way is to use SqlBulkCopy which does a great ...
Johan's user avatar
  • 1,310
0 votes
2 answers
433 views

Uploading 100,000 rows into a SQL Server database

What's the fastest way to upload bulk (over 100,000 records) from an ASP.NET MVC application? I tried using the following methods: BulkCopy but I keep getting this error: Message: Error writing to ...
Oghenemaro Okolosio's user avatar
2 votes
1 answer
217 views

SqlBulkCopy throws "Operation is not valid due to the current state of the object"

I'm attempting to create a custom DataReader for an IAsyncEnumerable<T> collection in order to load the records into a database table via SqlBulkCopy. I'm following along the lines of the code ...
JamesEggers's user avatar
  • 12.9k
5 votes
1 answer
11k views

What is the difference between polybase and bulk insert, copy methods in azure data factory and when to use them?

Can some one please elaborate on when to use Polybase versus bulk insert in azure datafactory, what are the differences between these two copy methods?
Indra 's user avatar
  • 51
2 votes
1 answer
312 views

BCP multiple files in/out using Azure Active Directory Interactive

I am trying to export/import tables using bcp, I am able to do it one by one as the documentation says using Azure Active Directory Interactive (I require Multifactor authentication to access the DB), ...
Cesar Mejia's user avatar

1
2 3 4 5
20