0

In Mysql, I have a table with two columns (id, uuid). Then I inserted 30 million values into it. (ps: the uuid can repeated)

Now, I want to find the repeat value in the table by using Mysql grammar, but the sql spent too much time.

I want to search all columns, but it takes much time, so I tried querying first million rows, the it spent 8 seconds.

Then I tried with 10 million rows, it spend 5mins, then with 20 million rows, the server seem died.

select count(uuid) as cnt
from uuid_test
where id between 1
and 1000000
group by uuid having cnt > 1;

Anyone can help me to optimized the sql, thanks

6
  • 1
    Your table has 3000-watt columns?
    – Ian Kemp
    Jan 30, 2019 at 6:55
  • 1
    Create in index on uuid column. It will take a long time to build but the select/group by queries will then become faster.
    – Salman A
    Jan 30, 2019 at 6:56
  • Seeing the asker is from China, I presume W = 10000 Jan 30, 2019 at 7:02
  • 1
    @YaobinThen I see: 10,000 = 万 = wàn = W. TIL! resources.allsetlearning.com/chinese/grammar/….E4.B8.87.28w.C3.A0n.29
    – Ian Kemp
    Jan 30, 2019 at 7:12
  • @YaobinThen thanks, my english is very poor
    – Reggie
    Jan 30, 2019 at 7:29

2 Answers 2

1

Try this query,

SELECT uuid, count(*) cnt FROM uuid_test GROUP BY 1 HAVING cnt>1;

Hope it helps.

1
  • thanks,it works,it`s only needed 10s. It seems that the Where and Between conditions cause the query to be too slow.
    – Reggie
    Jan 30, 2019 at 8:40
0

Often the fastest way to find duplicates uses a correlated subquery rather than aggregation:

select ut2.*
from uuid_test ut2
where exists (select 1
              from uuid_test ut2
              where ut2.uuid = ut.uuid and
                    ut2.id <> ut.id
             );

This can take advantage of an index on uuid_test(uuid, id).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.