0

I have a SQL Server database that will contain many tables that all connect, each with a primary key. I have a Dictionary that keeps track of the the primary keys fields are for each table. My task is to extract data every day from attribute-centric XML files and insert them into a master database. Each XML file has the same schema. I'm doing this by using an XMLReader and importing the data into a DataSet.

I can't use an AutoNumber for the keys. Let's say yesterday's XML file produced a DataTable similar to the following, and it was imported into a database

-------------------------------------
| Key | Column1 | Column2 | Column3 |
|-----------------------------------|
|  0  | dsfsfsd | sdfsrer | sdfsfsf |
|-----------------------------------|
|  1  | dertert | qweqweq | xczxsdf |
|-----------------------------------|
|  2  | prwersd | xzcsdfw | qwefkgs |
-------------------------------------

If today's XML file produces the following DataTable

-------------------------------------
| Key | Column1 | Column2 | Column3 |
|-----------------------------------|
|  0  | sesdfsd | hjghjgh | edrgffb |
|-----------------------------------|
|  1  | wrwerwr | zxcxfsd | pijghjh |
|-----------------------------------|
|  2  | vcbcvbv | vbnvnbn | bnvfgnf |
-------------------------------------

Then when I go to import the new data into the database using SqlBulkCopy, then there will be duplicate keys. My solution to this is to use DateTime.Now.Ticks to generate unique keys. Theoretically, this should always create a unique key.

However, for some reason DateTime.Now.Ticks is not unique. For example, 5 records in a row might all have the key 635387859864435908, and the next 7 records might have the key 635387859864592164, even though I am generating that value at different times. I want to say that the cause of the problem is that my script is calling DateTime.Now.Ticks several times before it updates the time.

Can anyone else think of a better way to generate keys?

10
  • Is there a reason you cannot just use GUIDs?
    – Chris Haas
    Jun 19, 2014 at 19:25
  • 1
    DateTime.Now.Ticks was not intended be unique. The amount of work a processor can get done in a tick is actually quite a bit, as you found out. Jun 19, 2014 at 19:26
  • @ChrisHaas I haven't heard of GUIDs. I'm looking into them now to see if this will solve the problem
    – TFischer
    Jun 19, 2014 at 19:27
  • 1
    Check out Eric Lippert's three part article on GUIDs for more information.
    – Chris Haas
    Jun 19, 2014 at 19:29
  • 2
    Just a warning, depending on how many records you have there can be a perf issue if you use a GUID for a primary key AND have it as a clustered index (which is the default). I usually pick a different column or group of columns to cluster on when I've got a GUID PK.
    – Chris Haas
    Jun 19, 2014 at 19:38

3 Answers 3

4

It's possible that the value of DateTime.Now is cached for a small amount of time for performance reasons. We do something similar to this and there are 2 possible options that we use:

  1. Keep a list of numbers that you've used on the server you're on and increment if you can determine the number has already been used
  2. Convert the field to a string and append a GUID or some other random identifier on the end of it. A GUID can be created with System.Guid.NewGuid().ToString();

Obviously neither of these plans are going to make the risk of collision zero, but they can help in reducing it.

2

If you have huge amount of data and you need to have a unique key for each row just use GUID

0

You could do something like the following to get a unique id (SQL Fiddle):

SELECT
CONCAT(YEAR(GETDATE()), DATEDIFF(DAY, STR(YEAR(GETDATE()), 4) + '0101',
GETDATE() ) + 1,  ROW_NUMBER() OVER(ORDER BY id DESC)) UniqueID
FROM supportContacts s

This would work if you only run the query once per day. If you ran it more than once per day you would need to grab the seconds or something else (SQL Fiddle):

SELECT CONCAT(CurrYear, CurrJulian, CurrSeconds, Row) AS UniqueID
FROM
(
  SELECT
  YEAR(GETDATE()) AS CurrYear, 
  DATEDIFF(DAY, STR(YEAR(GETDATE()), 4) + '0101', GETDATE() ) + 1 AS CurrJulian,  
  ROW_NUMBER() OVER(ORDER BY id DESC) AS Row,
  datediff(second, left(convert(varchar(20), getdate(), 126), 10), getdate()) AS CurrSeconds
  from supportContacts s
) AS m

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.