508

What is the difference between the EXISTS and IN clause in SQL?

When should we use EXISTS, and when should we use IN?

22 Answers 22

254

The exists keyword can be used in that way, but really it's intended as a way to avoid counting:

--this statement needs to check the entire table
select count(*) from [table] where ...

--this statement is true as soon as one match is found
exists ( select * from [table] where ... )

This is most useful where you have if conditional statements, as exists can be a lot quicker than count.

The in is best used where you have a static list to pass:

 select * from [table]
 where [field] in (1, 2, 3)

When you have a table in an in statement it makes more sense to use a join, but mostly it shouldn't matter. The query optimiser should return the same plan either way. In some implementations (mostly older, such as Microsoft SQL Server 2000) in queries will always get a nested join plan, while join queries will use nested, merge or hash as appropriate. More modern implementations are smarter and can adjust the plan even when in is used.

7
  • 4
    Could you elaborate on "When you have a table in an in statement it makes more sense to use a join, but it doesn't really matter. The query optimiser will return the same plan either way."? Not the query optimiser part, the part where you can use a JOIN as a replacement for IN.
    – farthVader
    Jan 27, 2015 at 3:52
  • select * from [table] where [field] in (select [field] from [table2]) returns the same results (and query plan) as select * from [table] join [table2] on [table2].[field] = [table].[field].
    – user4570983
    Aug 5, 2015 at 18:45
  • 1
    @Sander it doesn't: the first query returns all the columns from table, while the second returns everything from table and table2. In some (mostly older) SQL databases the in query will get implemented as a nested join, while the join query can be nested, merged, hashed, etc - whatever's quickest.
    – Keith
    Aug 5, 2015 at 19:43
  • 2
    Okay, I should have specified columns in the select clause, but you should update your answer because it clearly states that the queries "will return the same plan either way".
    – user4570983
    Aug 5, 2015 at 20:07
  • exists can be used within a case statement, so they can be handy that way also i.e. select case when exists (select 1 from emp where salary > 1000) then 1 else 0 end as sal_over_1000 Aug 25, 2016 at 4:17
157

EXISTS will tell you whether a query returned any results. e.g.:

SELECT * 
FROM Orders o 
WHERE EXISTS (
    SELECT * 
    FROM Products p 
    WHERE p.ProductNumber = o.ProductNumber)

IN is used to compare one value to several, and can use literal values, like this:

SELECT * 
FROM Orders 
WHERE ProductNumber IN (1, 10, 100)

You can also use query results with the IN clause, like this:

SELECT * 
FROM Orders 
WHERE ProductNumber IN (
    SELECT ProductNumber 
    FROM Products 
    WHERE ProductInventoryQuantity > 0)
2
  • 61
    @user2054927 Last query will correctly return no rows if the subquery returns no rows - nothing dangerous about that! Jun 4, 2016 at 9:36
  • I am confused. The last query assumes ProductNumber is in Orders table? If it is not it will give an error that column does not exists.
    – abd
    Oct 30, 2023 at 8:37
89

Based on rule optimizer:

  • EXISTS is much faster than IN, when the sub-query results is very large.
  • IN is faster than EXISTS, when the sub-query results is very small.

Based on cost optimizer:

  • There is no difference.
9
  • 30
    Proof of your argument? I don't think IN would be faster than EXISTS ever! May 6, 2014 at 6:50
  • 28
    @Nawaz How about the proof why IN is always slower than EXISTS?
    – ceving
    Jul 10, 2014 at 13:55
  • 2
    Badly implemented query optimizer? I've seem something like this (though not exactly this situation) happen in a certain RDBMs...
    – Haroldo_OK
    Aug 19, 2015 at 17:31
  • 8
    Why was this upvoted so many times? There's absolutely no reason why this assumption-based statement should be generally true.
    – Lukas Eder
    Jan 31, 2018 at 21:37
  • 4
    @LukasEder People like straight-forward answers, even if it comes at the detriment of correctness. Compare "Tea is healthy." vs "Researchers have claimed that drinking tea can reduce the chances of developing type 2 diabetes by up to 25 per cent if three or four cups are consumed a day. However, drinking more than four cups of tea a day can increase women’s risk of developing rheumatoid arthritis, research has found."
    – NoName
    Jan 12, 2020 at 4:23
47

I'm assuming you know what they do, and thus are used differently, so I'm going to understand your question as: When would it be a good idea to rewrite the SQL to use IN instead of EXISTS, or vice versa.

Is that a fair assumption?


Edit: The reason I'm asking is that in many cases you can rewrite an SQL based on IN to use an EXISTS instead, and vice versa, and for some database engines, the query optimizer will treat the two differently.

For instance:

SELECT *
FROM Customers
WHERE EXISTS (
    SELECT *
    FROM Orders
    WHERE Orders.CustomerID = Customers.ID
)

can be rewritten to:

SELECT *
FROM Customers
WHERE ID IN (
    SELECT CustomerID
    FROM Orders
)

or with a join:

SELECT Customers.*
FROM Customers
    INNER JOIN Orders ON Customers.ID = Orders.CustomerID

So my question still stands, is the original poster wondering about what IN and EXISTS does, and thus how to use it, or does he ask wether rewriting an SQL using IN to use EXISTS instead, or vice versa, will be a good idea?

5
  • 16
    I don't know about the OP, but I would like the answer to this question! When should I use EXISTS instead of IN with a subquery that returns IDs?
    – Roy Tinker
    Jul 13, 2010 at 17:45
  • 11
    in the JOIN, you will need a DISTINCT
    – Jaider
    Mar 26, 2014 at 18:51
  • 7
    great demonstration, but pretty much leave the question unanswered Mar 14, 2017 at 17:00
  • @RoyTinker the answer is an opinionated mix between "use X when it makes the query easier to understand than Y for the maintaining developer" and "use X when doing it makes it acceptably faster/less resource intensive than Y, which is causing a performance issue". Engineering is a compromise
    – Caius Jard
    Jul 14, 2021 at 8:48
  • @CaiusJard Right, agreed. The system I was building in 2010 was an in-house JSON <=> SQL ORM, so performance was the primary concern over how "readable" the generated queries would be.
    – Roy Tinker
    Jul 15, 2021 at 17:57
32
  1. EXISTS is much faster than IN when the subquery results is very large.
    IN is faster than EXISTS when the subquery results is very small.

    CREATE TABLE t1 (id INT, title VARCHAR(20), someIntCol INT)
    GO
    CREATE TABLE t2 (id INT, t1Id INT, someData VARCHAR(20))
    GO
    
    INSERT INTO t1
    SELECT 1, 'title 1', 5 UNION ALL
    SELECT 2, 'title 2', 5 UNION ALL
    SELECT 3, 'title 3', 5 UNION ALL
    SELECT 4, 'title 4', 5 UNION ALL
    SELECT null, 'title 5', 5 UNION ALL
    SELECT null, 'title 6', 5
    
    INSERT INTO t2
    SELECT 1, 1, 'data 1' UNION ALL
    SELECT 2, 1, 'data 2' UNION ALL
    SELECT 3, 2, 'data 3' UNION ALL
    SELECT 4, 3, 'data 4' UNION ALL
    SELECT 5, 3, 'data 5' UNION ALL
    SELECT 6, 3, 'data 6' UNION ALL
    SELECT 7, 4, 'data 7' UNION ALL
    SELECT 8, null, 'data 8' UNION ALL
    SELECT 9, 6, 'data 9' UNION ALL
    SELECT 10, 6, 'data 10' UNION ALL
    SELECT 11, 8, 'data 11'
    
  2. Query 1

    SELECT
    FROM    t1 
    WHERE   not  EXISTS (SELECT * FROM t2 WHERE t1.id = t2.t1id)
    

    Query 2

    SELECT t1.* 
    FROM   t1 
    WHERE  t1.id not in (SELECT  t2.t1id FROM t2 )
    

    If in t1 your id has null value then Query 1 will find them, but Query 2 cant find null parameters.

    I mean IN can't compare anything with null, so it has no result for null, but EXISTS can compare everything with null.

3
  • 1
    This answer is reasonable synopsis of Tom Kite's sentiment (asktom.oracle.com/pls/asktom/…) Sep 6, 2013 at 14:58
  • I think this answer is based on intuition, which is fair enough. But it cannot be universally true. For example, it is almost certainly not true of Ingres, which would parse both the equivalent SQL queries to be the same QUEL query, which lacks SQL's - ahem - 'richness' when it comes to writing the same thing multiple ways.
    – onedaywhen
    Aug 23, 2016 at 7:56
  • These 2 queries are logically equivalent if and only if t2.id is defined as "NOT NULL". To grantee the equivalency with no dependency in the table definition the 2nd query should be "SELECT t1.* FROM t1 WHERE t1.id not in (SELECT t2.id FROM t2 where t2.id is not null)" Oct 9, 2016 at 9:50
21

IN supports only equality relations (or inequality when preceded by NOT).
It is a synonym to =any / =some, e.g

select    * 
from      t1 
where     x in (select x from t2)
;

EXISTS supports variant types of relations, that cannot be expressed using IN, e.g. -

select    * 
from      t1 
where     exists (select    null 
                  from      t2 
                  where     t2.x=t1.x 
                        and t2.y>t1.y 
                        and t2.z like '℅' || t1.z || '℅'
                  )
;

And on a different note -

The allegedly performance and technical differences between EXISTS and IN may result from specific vendor's implementations/limitations/bugs, but many times they are nothing but myths created due to lack of understanding of the databases internals.

The tables' definition, statistics' accuracy, database configuration and optimizer's version have all impact on the execution plan and therefore on the performance metrics.

1
  • 2
    Upvote for your comment on performance: without focusing on a specific DBMS, we should assume that it is up to the optimizer to work out what works best.
    – Manngo
    Nov 10, 2017 at 6:27
20

If you are using the IN operator, the SQL engine will scan all records fetched from the inner query. On the other hand if we are using EXISTS, the SQL engine will stop the scanning process as soon as it found a match.

0
10

The Exists keyword evaluates true or false, but IN keyword compare all value in the corresponding sub query column. Another one Select 1 can be use with Exists command. Example:

SELECT * FROM Temp1 where exists(select 1 from Temp2 where conditions...)

But IN is less efficient so Exists faster.

5

I think,

  • EXISTS is when you need to match the results of query with another subquery. Query#1 results need to be retrieved where SubQuery results match. Kind of a Join.. E.g. select customers table#1 who have placed orders table#2 too

  • IN is to retrieve if the value of a specific column lies IN a list (1,2,3,4,5) E.g. Select customers who lie in the following zipcodes i.e. zip_code values lies in (....) list.

When to use one over the other... when you feel it reads appropriately (Communicates intent better).

3

As per my knowledge when a subquery returns a NULL value then the whole statement becomes NULL. In that cases we are using the EXITS keyword. If we want to compare particular values in subqueries then we are using the IN keyword.

3

Which one is faster depends on the number of queries fetched by the inner query:

  • When your inner query fetching thousand of rows then EXIST would be better choice
  • When your inner query fetching few rows, then IN will be faster

EXIST evaluate on true or false but IN compare multiple value. When you don't know the record is exist or not, your should choose EXIST

3

Difference lies here:

select * 
from abcTable
where exists (select null)

Above query will return all the records while below one would return empty.

select *
from abcTable
where abcTable_ID in (select null)

Give it a try and observe the output.

1
  • 2
    Hmmm... Error: [SQL0104] Token ) was not valid. In both cases. Are you assuming a particular RDBMS? Aug 10, 2017 at 15:42
3

The reason is that the EXISTS operator works based on the “at least found” principle. It returns true and stops scanning table once at least one matching row found.

On the other hands, when the IN operator is combined with a subquery, MySQL must process the subquery first, and then uses the result of the subquery to process the whole query.

The general rule of thumb is that if the subquery contains a large volume of data, the EXISTS operator provides a better performance.

However, the query that uses the IN operator will perform faster if the result set returned from the subquery is very small.

1

My understand is both should be the same as long as we are not dealing with NULL values.

The same reason why the query does not return the value for = NULL vs is NULL. http://sqlinthewild.co.za/index.php/2010/02/18/not-exists-vs-not-in/

As for as boolean vs comparator argument goes, to generate a boolean both values needs to be compared and that is how any if condition works.So i fail to understand how IN and EXISTS behave differently .

1

In certain circumstances, it is better to use IN rather than EXISTS. In general, if the selective predicate is in the subquery, then use IN. If the selective predicate is in the parent query, then use EXISTS.

https://docs.oracle.com/cd/B19306_01/server.102/b14211/sql_1016.htm#i28403

1
  • 2
    It should be noted that even at the time you posted this answer in 2017, you were referring to an oracle product that was released 12 years prior and was already well past its end of life
    – Caius Jard
    Jul 14, 2021 at 8:40
0

If a subquery returns more than one value, you might need to execute the outer query- if the values within the column specified in the condition match any value in the result set of the subquery. To perform this task, you need to use the in keyword.

You can use a subquery to check if a set of records exists. For this, you need to use the exists clause with a subquery. The exists keyword always return true or false value.

0

I believe this has a straightforward answer. Why don't you check it from the people who developed that function in their systems?

If you are a MS SQL developer, here is the answer directly from Microsoft.

IN:

Determines whether a specified value matches any value in a subquery or a list.

EXISTS:

Specifies a subquery to test for the existence of rows.

0

I found that using EXISTS keyword is often really slow (that is very true in Microsoft Access). I instead use the join operator in this manner : should-i-use-the-keyword-exists-in-sql

0

If you can use where in instead of where exists, then where in is probably faster.

Using where in or where exists will go through all results of your parent result. The difference here is that the where exists will cause a lot of dependet sub-queries. If you can prevent dependet sub-queries, then where in will be the better choice.

Example

Assume we have 10,000 companies, each has 10 users (thus our users table has 100,000 entries). Now assume you want to find a user by his name or his company name.

The following query using were exists has an execution of 141ms:

select * from `users` 
where `first_name` ='gates' 
or exists 
(
  select * from `companies` 
  where `users`.`company_id` = `companies`.`id`
  and `name` = 'gates'
)

This happens, because for each user a dependent sub query is executed: enter image description here

However, if we avoid the exists query and write it using:

select * from `users` 
where `first_name` ='gates' 
or users.company_id in  
(
    select id from `companies` 
    where  `name` = 'gates'
)

Then depended sub queries are avoided and the query would run in 0,012 ms

enter image description here

0

I did a little exercise on a query that I have recently been using. I originally created it with INNER JOINS, but I wanted to see how it looked/worked with EXISTS. I converted it. I will include both version here for comparison.

SELECT DISTINCT Category, Name, Description 
FROM [CodeSets] 
WHERE Category NOT IN (
    SELECT def.Category
      FROM [Fields] f
      INNER JOIN [DataEntryFields] def ON f.DataEntryFieldId = def.Id
      INNER JOIN Section s ON f.SectionId = s.Id
      INNER JOIN Template t ON s.Template_Id = t.Id
    WHERE t.AgencyId = (SELECT Id FROM Agencies WHERE Name = 'Some Agency')
        AND def.Category NOT IN ('OFFLIST', 'AGENCYLIST', 'RELTO_UNIT', 'HOSPITALS', 'EMS', 'TOWCOMPANY', 'UIC', 'RPTAGENCY', 'REP')
        AND (t.Name like '% OH %') 
        AND (def.Category IS NOT NULL AND def.Category <> '')
)
ORDER BY 1

Here are the statistics: enter image description here

Here is the converted version:

SELECT DISTINCT cs.Category, Name, Description 
FROM [CodeSets] cs
WHERE NOT Exists (
    SELECT * FROM [Fields] f
      WHERE EXISTS (SELECT * FROM [DataEntryFields] def 
                       WHERE def.Id  = f.DataEntryFieldId
                           AND def.Category NOT IN ('OFFLIST', 'AGENCYLIST', 'RELTO_UNIT', 'HOSPITALS', 'EMS', 'TOWCOMPANY', 'UIC', 'RPTAGENCY', 'REP')
                           AND (def.Category IS NOT NULL AND def.Category <> '')
                           AND def.Category = cs.Category
                           AND EXISTS (SELECT * FROM Section s 
                                          WHERE f.SectionId = s.Id
                                          AND EXISTS (SELECT * FROM Template t 
                                                        WHERE s.Template_Id = t.Id
                                                           AND EXISTS (SELECT * FROM Agencies 
                                                                          WHERE Name = 'Some Agency' and t.AgencyId = Id)
                                                                             AND (t.Name like '% OH %')
                                                    )
                                        )
                    )
)
ORDER BY 1

The results, at least to me, were unimpressive. enter image description here

If I were more technically knowledgeable about how SQL works, I could give you an answer, but take this example as you may and make your own conclusion.

The INNER JOIN and IN () is easier to read, however.

-1

EXISTS Is Faster in Performance than IN. If Most of the filter criteria is in subquery then better to use IN and If most of the filter criteria is in main query then better to use EXISTS.

1
  • 1
    That claim is really not backed by any evidence, is it?
    – Lukas Eder
    Jan 31, 2018 at 21:40
-2

If you are using the IN operator, the SQL engine will scan all records fetched from the inner query. On the other hand if we are using EXISTS, the SQL engine will stop the scanning process as soon as it found a match.

2
  • @ziggy explain? This is pretty much what the accepted answer also says. In MUST check every single record, exists can stop as soon as it finds just one. Jul 15, 2014 at 14:54
  • Nope, not correct. IN and EXISTS can be equivalent and transformed into each other.
    – Lukas Eder
    Jan 31, 2018 at 21:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.