1

I have an enormous set of data that was loaded into a SQL database incorrectly (it's a Vertica DB, if that matters).

Each row has an id and a timestamp. The data was loaded sequentially (sequential ids), however the timestamps were converted incorrectly, resulting in all times being loaded as "AM".

Here's a simplified example of what's in the DB:

id    |      time
001   | 2013-01-01 00:00:01 // Jan 1st
002   | 2013-01-01 01:20:00
...   | ...
500   | 2013-01-01 11:59:59
501   | 2013-01-01 00:00:01 // should be 12:00:01 (PM)
502   | 2013-01-01 00:10:00 // should be 12:10:00 (PM)
...   | ...
750   | 2013-01-01 11:59:59 // should be 23:59:59 (PM)
751   | 2013-01-02 00:00:00 // next day (the 2nd)

I need a query that will find me ID ranges for rows that need 12 hours added to their time. For example, for the example data above, the row returned should be 501, 750. This way I could add 12 hours where the id >= 501, and <= 750.

Basically, I need some sort of a look-back + look-forward mechanism. Programmatically, it would mean iterating over each row and checking whether the day of the date is the same as the next row, and checking whether the time is earlier than the previous row. However, I'm sure there's some better way to do it in SQL...

6
  • What is your RDBMS? Need to know function names to extract date and time from your (apparent) datetime column.
    – PM 77-1
    Apr 5, 2013 at 22:49
  • 1
    It's Vertica, but I just need some SQL pseudocode for the logic. I can handle the specifics of my RDBMS in implementation. Now that I think about it, it's two joins on itsself, perhaps? One with where b.id = a.id - 1, and another where b.id = a.id + 1? Hmmm... Apr 5, 2013 at 22:54
  • I was thinking of a single SELF JOIN.
    – PM 77-1
    Apr 5, 2013 at 22:56
  • "look-back" and "look-forward" in standard SQL is done with window functions. lag() and lead(). Don't know if Vertica supports that.
    – user330315
    Apr 5, 2013 at 23:02
  • Aha, lag() and lead()! Thanks @a_horse_with_no_name... I'll try fiddling with those. Apr 5, 2013 at 23:18

1 Answer 1

0

This is not exactly what you're looking for, but it will give all row pairs that are out of sync for their given day.

SELECT t1.id As 'ID1', t1.timestamp As 'Stamp1', t2.id As 'ID2', t2.timestamp As 'Stamp2' 
FROM Table1 t1
JOIN Table1 t2 ON Date(t1.timestamp) = Date(t2.timestamp) AND 
                  Time(t2.timestamp) <= Time(t1.timestamp) AND
                  t1.id <= t2.id
WHERE t1.id <> t2.id OR Time(t2.timestamp) <> Time(t1.timestamp)
ORDER BY t1.id;

Here's SQL Fiddle

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.