1

I have a TextMessage model, which has many histories

class TextMessage < ApplicationRecord

  has_many :histories, class_name: :CustomerServiceHistory, as: :item

  scope :latest_messages, -> {
      includes(histories: :action, phone: :customer)
      .where("customer_service_actions.name != 'close' OR customer_service_actions.name IS NULL")
      .where("text_messages.created_at = (SELECT MAX(text_messages.created_at) FROM text_messages WHERE text_messages.phone_id = phones.id)")
  }
end

A CustomerServiceHistory belongs to an item (which can be a text message or email). Users can either "read" or "close" an item. To do that, the CustomerServiceHistory belongs to a user and action (read or close).

class CustomerServiceHistory < ApplicationRecord
  belongs_to :action, class_name: :CustomerServiceAction,
                      foreign_key: :customer_service_action_id
  belongs_to :item, polymorphic: true
  belongs_to :user
end

I have an index page where I want to load all text messages except those that have been closed. This is where the latest_messages from TextMessage comes in.

.where("customer_service_actions.name != 'close' OR customer_service_actions.name IS NULL")

The where("customer_service_actions.name != 'close'... will load the text messages that don't have a "close" action associated with them.

The ... OR customer_service_actions.name IS NULL will load the text messages that don't have any customer_service_actions yet, and are considered "unread" to users.

The problem is when a text message has been "read" and then "closed" by a user, that text message now has two history records on it.

The where clause stops working because it's able to filter out the relationship between this text message and it's 'close' action but not its associations with its 'read' actions.

Also, many users can read a text message. There could be 100 users who read that text message. I want the text message to not load when there is just one "close" action on this text message, regardless of how many "read" actions there are.

Is this possible to do with just SQL?

Here is my SQL output.

SQL (1.0ms)  
SELECT  DISTINCT "text_messages"."id", 
  customer_service_histories.customer_service_action_id AS alias_0, 
  text_messages.created_at AS alias_1 
FROM "text_messages" 
LEFT OUTER JOIN "customer_service_histories" 
  ON "customer_service_histories"."item_id" = "text_messages"."id" 
  AND "customer_service_histories"."item_type" = $1 
LEFT OUTER JOIN "customer_service_actions" 
  ON "customer_service_actions"."id" = "customer_service_histories"."customer_service_action_id" 
LEFT OUTER JOIN "phones" 
  ON "phones"."id" = "text_messages"."phone_id" 
LEFT OUTER JOIN "customers" 
  ON "customers"."id" = "phones"."customer_id" 
  AND "customers"."company_id" = $2 
WHERE (
  customer_service_actions.name != 'close' 
  OR customer_service_actions.name IS NULL
) 
AND (
  text_messages.created_at = (
    SELECT MAX(text_messages.created_at) 
    FROM text_messages 
    WHERE text_messages.phone_id = phones.id
  )
) 
ORDER BY 
  customer_service_histories.customer_service_action_id DESC, 
  text_messages.created_at 
DESC LIMIT $3 OFFSET $4  
[["item_type", "TextMessage"], ["company_id", 1], ["LIMIT", 10], ["OFFSET", 0]]

2 Answers 2

1

Maybe using EXCEPT?

(SELECT * 
FROM "text_messages"
LEFT OUTER JOIN "customer_service_actions" 
ON "customer_service_actions"."id" = "customer_service_histories"."customer_service_action_id")
EXCEPT
(SELECT * 
FROM "text_messages"
LEFT OUTER JOIN "customer_service_actions" 
ON "customer_service_actions"."id" = "customer_service_histories"."customer_service_action_id"
WHERE "customer_service_actions"."name" LIKE 'close')

Edit: Apparently Rails ActiveRecord does not support EXCEPT queries. You could subtract queries in Rails tho.

q1 = TextMessage.all 
q2 = TextMessage.includes(:histories).where(customer_service_actions:{name: 'close'}) 
result = q1 - q2 

that may work

2
  • Although this might work, I couldn't find a way to implement this in rails to generate that query
    – Cruz Nunez
    Aug 30, 2019 at 15:31
  • Apparently Rails ActiveRecord does not support EXCEPT queries. You could subtract queries in Rails tho. q1 = TextMessage.all q2 = TextMessage.includes(:histories).where(customer_service_actions:{name: 'close'}) result = q1 - q2 that may work
    – RHFS
    Sep 3, 2019 at 18:34
0

I've got something working but it's unsatisfactory.

class TextMessage
  def self.search(query)
    return latest_messages.active unless query.present?

    # more code
  end

  scope :latest_messages, -> {
    where("text_messages.created_at = (SELECT MAX(text_messages.created_at) FROM text_messages WHERE text_messages.phone_id = phones.id)")
  }

  scope :active, -> {
    where(
      <<~SQL.squish
        text_messages.id NOT IN (
          SELECT text_messages.id
          FROM text_messages
          INNER JOIN customer_service_histories
            ON customer_service_histories.item_id = text_messages.id
            AND customer_service_histories.item_type = 'TextMessage'
          INNER JOIN customer_service_actions
            ON customer_service_actions.id = customer_service_histories.customer_service_action_id
          WHERE customer_service_actions.name = 'close'
        )
      SQL
    )
  }

This produces the SQL

SQL (1.9ms)  
SELECT  DISTINCT "text_messages"."id", 
  customer_service_histories.customer_service_action_id AS alias_0, 
  text_messages.created_at AS alias_1 
FROM "text_messages" 
INNER JOIN "phones" 
  ON "phones"."id" = "text_messages"."phone_id" 
INNER JOIN "customers" 
  ON "customers"."id" = "phones"."customer_id" 
  AND "customers"."company_id" = $1 
LEFT OUTER JOIN "customer_service_histories" 
  ON "customer_service_histories"."item_id" = "text_messages"."id" 
  AND "customer_service_histories"."item_type" = $2 
LEFT OUTER JOIN "customer_service_actions" 
  ON "customer_service_actions"."id" = "customer_service_histories"."customer_service_action_id" 
WHERE (
  text_messages.created_at = (
    SELECT MAX(text_messages.created_at) 
    FROM text_messages 
    WHERE text_messages.phone_id = phones.id
  )
) 
AND (
  text_messages.id NOT IN (
    SELECT text_messages.id
    FROM text_messages
    INNER JOIN customer_service_histories
      ON customer_service_histories.item_id = text_messages.id
      AND customer_service_histories.item_type = 'TextMessage'
    INNER JOIN customer_service_actions
      ON customer_service_actions.id = customer_service_histories.customer_service_action_id
    WHERE customer_service_actions.name = 'close'
  )
) 
ORDER BY 
  customer_service_histories.customer_service_action_id DESC, 
  text_messages.created_at DESC 
LIMIT $3 OFFSET $4  
[["company_id", 1], ["item_type", "TextMessage"], ["LIMIT", 10], ["OFFSET", 0]]

This is correct SQL but it uses SQL as strings. Ideally what I want is

  1. Load the correct data
  2. Use the correct SQL
  3. Use Rails syntax instead of SQL in strings
  4. Have to ability to chain these scopes together

Something like this

class TextMessage
  def self.search(query)
    return latest_messages.active unless query.present?

    # more code
  end


  scope :latest_messages, -> {
    where("text_messages.created_at = (SELECT MAX(text_messages.created_at) FROM text_messages WHERE text_messages.phone_id = phones.id)")
  }

  scope :active, -> {
    where.not(id: TextMessage.select(:id)
                             .joins(histories: :action)
                             .where(customer_service_actions: { name: 'close' })
             )
  }

  # more code
end

Using this Rails code loads the correct data but for some reason causes too much SQL

SQL (1.2ms)  
SELECT  DISTINCT "text_messages"."id", customer_service_histories.customer_service_action_id AS alias_0, text_messages.created_at AS alias_1 
FROM "text_messages" INNER JOIN "phones" ON "phones"."id" = "text_messages"."phone_id" 
INNER JOIN "customers" ON "customers"."id" = "phones"."customer_id" AND "customers"."company_id" = $1 
LEFT OUTER JOIN "customer_service_histories" ON "customer_service_histories"."item_id" = "text_messages"."id" AND "customer_service_histories"."item_type" = $2 
LEFT OUTER JOIN "customer_service_actions" ON "customer_service_actions"."id" = "customer_service_histories"."customer_service_action_id" 
WHERE (
  text_messages.created_at = (                         -- first condition
    SELECT MAX(text_messages.created_at) 
    FROM text_messages 
    WHERE text_messages.phone_id = phones.id
  )
) 
AND (
  text_messages.id NOT IN (
    SELECT "text_messages"."id" 
    FROM "text_messages" 
    INNER JOIN "customer_service_histories" ON "customer_service_histories"."item_id" = "text_messages"."id" AND "customer_service_histories"."item_type" = 'TextMessage' 
    INNER JOIN "customer_service_actions" ON "customer_service_actions"."id" = "customer_service_histories"."customer_service_action_id" 
    WHERE (
      text_messages.created_at = (                     -- repeated first condition
        SELECT MAX(text_messages.created_at) 
        FROM text_messages 
        WHERE text_messages.phone_id = phones.id
      )
    ) 
    AND "customer_service_actions"."name" = 'close'    -- second condition
  )
) 
ORDER BY 
  customer_service_histories.customer_service_action_id DESC, 
  text_messages.created_at DESC 
LIMIT $3 OFFSET $4  
[["company_id", 1], ["item_type", "TextMessage"], ["LIMIT", 10], ["OFFSET", 0]]

The created_at condition is repeated and then is paired with the actions.name condition. I tried many different combination of things to try to get it to work with more concise ruby syntax but I wan't satisfied with the SQL output.

I did find a way to use the ruby syntax and get the SQL I wanted, but I had to have both where() functions in the same scope.

class TextMessage
  def self.search(query)
    return latest_messages unless query.present?

    # more code
  end

  scope :latest_messages, -> {
    where("text_messages.created_at = (SELECT MAX(text_messages.created_at) FROM text_messages WHERE text_messages.phone_id = phones.id)")
    .where('text_messages.id NOT IN (?)', TextMessage.active_ids)
  }

  scope :active_ids, -> {
    TextMessage.select(:id).joins(histories: :action).where.not(
      customer_service_actions: { name: 'close' }
    )
  }

  # more code
end

I tried to put them in different scopes

  def self.search(query)
    return latest_messages.active unless query.present?

    # more code
  end

  scope :latest_messages, -> {
    where("text_messages.created_at = (SELECT MAX(text_messages.created_at) FROM text_messages WHERE text_messages.phone_id = phones.id)")
  }

  scope :active, -> {
    where('text_messages.id NOT IN (?)', TextMessage.active_ids)
  )

  scope :active_ids, -> {
    TextMessage.select(:id).joins(histories: :action).where.not(
      customer_service_actions: { name: 'close' }
    )
  }

But that caused more join clauses in the subquery

SQL (1.7ms)  
SELECT  DISTINCT "text_messages"."id", 
  customer_service_histories.customer_service_action_id AS alias_0, 
  text_messages.created_at AS alias_1 
FROM "text_messages" 
INNER JOIN "phones" 
  ON "phones"."id" = "text_messages"."phone_id" 
INNER JOIN "customers" 
  ON "customers"."id" = "phones"."customer_id" 
  AND "customers"."company_id" = $1 
LEFT OUTER JOIN "customer_service_histories" 
  ON "customer_service_histories"."item_id" = "text_messages"."id" 
  AND "customer_service_histories"."item_type" = $2 
LEFT OUTER JOIN "customer_service_actions" 
  ON "customer_service_actions"."id" = "customer_service_histories"."customer_service_action_id" 
WHERE (
  text_messages.created_at = (                      -- first condition
    SELECT MAX(text_messages.created_at) 
    FROM text_messages 
    WHERE text_messages.phone_id = phones.id
  )
) 
AND (
  "text_messages"."id" NOT IN (
    SELECT "text_messages"."id" 
    FROM "text_messages" 
    INNER JOIN "phones"                             -- unnecessary joins on phones 
      ON "phones"."id" = "text_messages"."phone_id" 
    INNER JOIN "customers"                          -- unnecessary joins on customers
      ON "customers"."id" = "phones"."customer_id" 
      AND "customers"."company_id" = $3 
    INNER JOIN "customer_service_histories" 
      ON "customer_service_histories"."item_id" = "text_messages"."id" 
      AND "customer_service_histories"."item_type" = $4 
    INNER JOIN "customer_service_actions" 
      ON "customer_service_actions"."id" = "customer_service_histories"."customer_service_action_id" 
    WHERE (
      text_messages.created_at = (                  -- repeated first condition
        SELECT MAX(text_messages.created_at) 
        FROM text_messages 
        WHERE text_messages.phone_id = phones.id
      )
    ) 
    AND "customer_service_actions"."name" = $5      -- second condition
  )
) 
ORDER BY 
  customer_service_histories.customer_service_action_id DESC, 
  text_messages.created_at 
DESC LIMIT $6 OFFSET $7  
[["company_id", 1], ["item_type", "TextMessage"], ["company_id", 1], ["item_type", "TextMessage"], ["name", "close"], ["LIMIT", 10], ["OFFSET", 0]]

Maybe there's something in rails that I didn't try, but I feel like I tried a lot of combinations.

String advantages

Anyways, I did benchmark testing by running the queries 1,000 times and found that the string queries were up to 25% faster than the ruby equivalents. In addition, they don't add any unnecessary joins or conditions, which is less work for the database server. I think I'm going to stick with the strings.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.