0

I have large tables in two different schemas inside the same Greenplum Postgresql database which I want to join using dplyr. I cannot provide a reproducible example because it involves proprietary data, but I can provide code (with names suitably changed).

In SQL this would be

SELECT column_name(s)
FROM schema1.table1
INNER JOIN schema2.table2
ON schema1.table1.column_name=schema2.table2.column_name;

Using dplyr, this is

my_db <- src_postgres(host = "s.net", user = "id",password = "xxx",dbname="db1",  options="-c search_path=schema1") 
my_db_src <- src_postgres(host = "s", user = "id", password = "xxx", dbname="db1",  options="-c search_path=schema2") 

tbl1 <- tbl(my_db, "table1")
tbl1 <- tbl(my_db_src, "table2")
cc_compare <- inner_join(tbl1 ,tbl2,by="customerid",copy=TRUE)

I'd like to join them in dplyr without using copy == TRUE, which takes a long time. Can dplyr accomplish this and if so how?

3
  • no reproducible data? and how large is the data?
    – user227710
    Jul 5, 2015 at 17:48
  • 3
    data.table join would be fast. Some reproducible example would be helpful.
    – akrun
    Jul 5, 2015 at 17:51
  • just made an edit to clarify. no reproducible data. @akrun data.table doesn't seem to do SQL queries, just operations on local data already in memory, right?
    – ganong
    Jul 6, 2015 at 16:07

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.