0

Currently I am trying to minimize the number of SQL queries in the rails application. And was using scope for the record filtering and Found that in my server log the scope trigger the SQL query although it show CACHE (0.00 ms) even though it is filtering the record using attributes of the record. (eg. filtering using obj.status)

I tried to use the class method in the model, again when ever I call

def pdf_files
  all.select(&:pdf?)
end

scopes,

scope :pdf_files, -> { all.select(&:pdf?) }

it triggers the sql query.

I tried overriding the ActiveRecord::Associations::CollectionProxy to include this method like,

class ActiveRecord::Associations::CollectionProxy
  def pdf_files
    to_a.select(&:pdf?)
  end
end

Worked, But this method can also be called from other model.

I tried using blocks to extend your associations with extra methods somethings like.

has_many :files, do
  def pdf_files
    to_a.select(&:pdf?)
  end
end

This works for me, it select the files without any sql query.

But the thing is. I need these methods only available for File model, as these methods are required for all other models associate with file model and I don't want to break the rails convention like, @email.files.pdf.

Also found that in rails console, when I type File::ProxyCollection it give me ActiveRecord::Associations::CollectionProxy. But don't know how to add method to it.

I want to be able to call @email.files.pdf?, @email.files.doc?, to filter the results without executing any query at all even if it is calling CACHE SQL Query.

Model definitions

class TestFile < ApplicationRecord
  belongs_to :fileable, polymorphic: true, optional: true
  scope :document_files, -> { all.select(&:document?) }
  scope :ebook_files, -> { all.select(&:ebook?) }

  enum file_type: [:document, :ebook, :paper, :article, :picture]
end

class TestEmail < ApplicationRecord
  has_many :test_files, as: :fileable
end

class TestPost < ApplicationRecord
  has_many :test_files, as: :fileable
end

In console

email = TestEmail.includes(:test_files).first

document_files = email.test_files.document_files

The query executed for document_files scope is enter image description here

But if I do something like,

class TestFile < ApplicationRecord
  belongs_to :fileable, polymorphic: true, optional: true

  enum file_type: [:document, :ebook, :paper, :article, :picture]
end

class TestEmail < ApplicationRecord
  has_many :test_files, as: :fileable do
    def document_files
      to_a.select(&:document?)
    end
  end
end

class TestPost < ApplicationRecord
  has_many :test_files, as: :fileable do
    def document_files
      to_a.select(&:document?)
    end
  end
end

the result are loaded without any query enter image description here

2
  • You're trying to do something very strange, to be honest. Could you provide the model definitions (with the association) and an example of a query for which you want to get rid of the unnecessary queries? Apr 12, 2019 at 13:19
  • @KonstantinStrukov I have edited my question to include the model definitions and screen shot showing the sql query state for each case. Is this what you are looking for? Apr 12, 2019 at 17:05

1 Answer 1

1

So you have a polymorphic association and you want to fetch the associated data with minimum possible queries, right?

As for me, the task itself ("minimize the number of queries") makes sense only with reasonable limitations. It's always a good thing to get rid of N+1, but it's not necessarily good idea to try to do all the stuff in 1 query or something like this - complicated query can be slower than several consequent trivial ones. Also, complex tricky queries tend to be bug-prone.

Now let's go back to your code. First of all, this idea

scope :document_files, -> { all.select(&:document?) }

looks quite bad to me because of two reasons:

  1. everything is being loaded in memory and in case of a large table it hurts. Sometimes it is a necessary trade-off, but not in this case (you don't need any heavy processing etc - just filtering the data)

  2. this "scope" is not a regular one - invoking it gives you Array instead of AR relation, so it cannot be chained as proper AR scopes do etc. It just a misleading code that smells. Just try smth. like TestFile.document_files.where(<some_extra_conditions>) and you will get NoMethodError - probably not the result one could expect from the simple and straightforward code...

Using association extensions is also useless in your case - it makes your code look more cryptic but doesn't really give you any benefits. You're wrong assuming it works in some special way - in fact, it does exactly the same job as if you call to_a.select... on the association proxy explicitly.

I'd suggest something simpler and more idiomatic:

class TestFile < ApplicationRecord
  belongs_to :fileable, polymorphic: true, optional: true
  scope :document_files, -> { where(file_type: :document) }
  scope :ebook_files, -> { where(file_type: :ebook) }

  enum file_type: [:document, :ebook, :paper, :article, :picture]
end

...

Then, using preloading where appropriate, you can get low number of queries (no N+1).

Want to optimize this even more? Well, there are plenty of ways to do this using more low-level API: for example, you could use connection#select_all with an arbitrarily complicated query and then instantiate the necessary records manually from ActiveRecord::Result with no more queries... But AR is quite opinionated ORM framework and if you're going to "fight" it, you will likely end up with messy and non-maintainable code quite fast.

If you really need something more flexible I'd suggest trying other options (Sequel, ROM)...

3
  • I have already tried that option too, for document_files scope, -> { where(file_type: :document) } also triggers a SQL query, which I want to avoid as we already have our required test_files already loaded and we just need to filter out the required files from that loaded records. Apr 13, 2019 at 3:26
  • And every time I do something like documents = @emal.test_files.document_files, every time it executes a new SQL query Apr 13, 2019 at 3:42
  • Just Thinking, what about using the helper methods for handling the non model logic Apr 16, 2019 at 1:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.