9

Is there anyway to join two tables on multiple potential conditions?

I'm currently migrating some code from Postgres to Bigquery where I joined on multiple potential values like:

SELECT
 *
FROM
 (
 SELECT
   offer_table.offer_id
   ,customer_table.customer_name
   ,customer_table.visit_count
   ,ROW_NUMBER() OVER (PARTITION BY offer_table.offer_id ORDER BY customer_table.visit_count DESC) AS customer_visit_rank
 FROM
   offer_table
   LEFT JOIN customer_table ON
    (
    offer_table.customer_id = customer_table.customer_id
    OR offer_table.email = customer_table.email
    OR offer_table.phone = customer_table.phone
    )
 ) dummy
WHERE
  customer_visit_rank = 1

I needed to this because my offer and customer data had inconsistent usage of our id, email, and phone fields but all were valid potential matches. If multiple fields worked (ex: id and email matched), there would be duplicate rows and I'd filter them out based on the row_number column after ranking using the ORDER BY section.

However when I try to join on multiple conditions in BigQuery, I get this error message:

LEFT OUTER JOIN cannot be used without a condition that is an equality of fields from both sides of the join.

Has anyone figured out a solution to join on multiple values instead of doing the above?

6
  • 1
    Can you run three seperate queries for each one of the three JOIN ON conditions and then combine the results using UNION OR UNION ALL? Favour UNION ALL, because it will just append found rows and will not have to sort to remove duplicates, like UNION would have to. So basically SELECT ... ON customer_id ... UNION ALL SELECT ... ON email ... UNION ALL SELECT ... ON phone. If this is right, can i make it an answer, please?
    – flutter
    May 8, 2017 at 22:58
  • 2
    Use standard SQL. May 8, 2017 at 23:01
  • @flutter Wouldn't you have duplicate rows per offer_id by doing so? I'm trying to get a unique row per offer_id matched on the customer_table row that has the right data (highest visit_count in this example).
    – Frank Lee
    May 8, 2017 at 23:10
  • Just use standard SQL like Gordon suggested. cloud.google.com/bigquery/docs/reference/standard-sql May 9, 2017 at 1:03
  • 1
    In the linked Standard SQL syntax i can see UNION DISTINCT. Did you try going with that?
    – flutter
    May 9, 2017 at 1:21

1 Answer 1

7

You can write separate queries, then use COALESCE:

SELECT
  *
FROM
  (
    SELECT
      offer_table.offer_id
      ,COALESCE(c1.customer_name,c2.customer_name,c3.customer_name)
      ,COALESCE(c1.visit_count,c2.visit_count,c3.visit_count)
      ,ROW_NUMBER() OVER (PARTITION BY offer_table.offer_id ORDER BY customer_table.visit_count DESC) AS customer_visit_rank
    FROM
      offer_table
    LEFT JOIN customer_table c1
      ON offer_table.customer_id = customer_table.customer_id
    LEFT JOIN customer_table c2
      ON offer_table.email = customer_table.email
    LEFT JOIN customer_table c3
      ON offer_table.phone = customer_table.phone
   )
 ) AS dummy
WHERE
  customer_visit_rank = 1
1
  • I tried that approach on some other sample use case and this can easily lead to Resources exceeded during query execution: The query could not be executed in the allotted memory. Peak usage: 103% of limit. Top memory consumer(s): aggregate functions and GROUP BY clauses: 100%
    – fpopic
    Jan 23, 2020 at 23:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.