6

I have just discovered BigQuery’s QUALIFY operator and have been reading about it at https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#qualify_clause

That documentation though does not explain why I should use QUALIFY instead of a normal WHERE predicate. If we take the example provided in the documentation:

SELECT item,
  RANK() OVER (PARTITION BY category ORDER BY purchases DESC) as rank
FROM Produce
WHERE Produce.category = 'vegetable'
  QUALIFY rank <= 3

That query could also be written as

SELECT
  item,
  RANK() OVER (PARTITION BY category ORDER BY purchases DESC) as rank
FROM Produce
WHERE Produce.category = 'vegetable'
AND rank <= 3

and it would produce the same result. So what is the advantage of using QUALIFY?

2
  • Ah, just read the documentation again, perhaps the reason is that using QUALIFY means the analytical function does not need to be in the SELECT clause. Thus it is syntactic sugar, which is no bad thing. Are there any other reasons to use QUALIFY? Could it offer a performance gain perhaps?
    – jamiet
    Nov 28, 2021 at 22:33
  • 1
    Jiho's answer is good, but just to be clear: your second query there won't work, because WHERE only accepts columns that exist in the FROM tables, not ones defined in the SELECT clause. Your second query will give an error saying something like "Unrecognized name: rank". So you can only write a query like that using QUALIFY (or with a subquery).
    – Acrofales
    Feb 23 at 15:48

1 Answer 1

8

One usage of the QUALIFY clause is to filter the results by the analytic function (sometimes with WINDOW FUNCTION). As you mentioned in the comments this could be seen as Syntactic sugar since the results of the analytic function can be stored in an additional subquery and can be filtered with WHERE clasue.

USAGE 01: Find user_id's last login info

w/ QUALIFY
SELECT user_id, ip, country_code, os, ...,
FROM login_logs
WHERE TRUE
QUALIFY ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY log_datetime DESC) = 1
;
wo/ QUALIFY
#standardSQL
WITH
login_log AS (
    SELECT
        user_id, ip, country_code, os, ...,
        ROW_NUMBER() OVER (
            PARTITION BY user_id ORDER BY log_datetime DESC
        ) AS row_num
    FROM user_login_info_table
)
SELECT user_id, ip, country_code, os, ...
FROM login_log
WHERE row_num = 1
;
Performances

I just had tested two different approaches with my data and found out that the slot time of the two queries are almost similar. However, the QUALIFY clause has a slight advantage in shuffled byte usage since it doesn't require keeping the results of row_num columns.

Query 01

Query 02

USAGE 02: Find if OS had changed when users login

SELECT
    log_datetime, user_id, os,
    LAG(os, 1, NULL) OVER user_id_os_list as previous_os,
FROM login_logs
WHERE TRUE
QUALIFY (previous_os != os)
WINDOW user_id_os_list AS (
    PARTITION BY user_id ORDER BY log_datetime
)
;
1
  • 1
    I believe LAG(os, 1, NULL) can be replaced with LAG(os) because default value for offset is 1 and default_expression is optional.
    – Roar S.
    Apr 13 at 11:13

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.