Uploaded image for project: 'Mongoid'
  1. Mongoid
  2. MONGOID-1965

Mongoid::Paranoia slow count() performance

    • Type: Icon: Task Task
    • Resolution: Done
    • 3.1.0
    • Affects Version/s: None
    • Component/s: None

      On a collection containing 3 million+ records, Paranoia pretty much trashes the normally-fast count() method. This is because, instead of asking Mongo for its count of the collection, we're actually doing a query counting all elements where deleted_at.ne => nil. An index on deleted_at (which ends up being worthless in most cases... our collection has 3 million+ records and maybe a dozen soft-deletions), it speeds it up some, but not nearly as much as would be desirable.

      Before index:

      Benchmark.measure { Product.count }
       =>   0.000000   0.000000   0.000000 (  218287.7)
      

      After index on deleted_at:

      Benchmark.measure { Product.count }
       =>   0.000000   0.000000   0.000000 (  3.594815)
      

      Still not great. So here's what I came up with for our system:

       def self.count
          collection.count - deleted.count
        end
      

      Benchmark on that method:

      Benchmark.measure { Product.count }
       =>   0.000000   0.000000   0.000000 (  0.000808)
      

      Huzzah!

      So my question is: what do people think about making this the standard behavior of the count method in the Mongoid::Paranoia module? Certainly, if you're doing mostly soft-deletes with few un-deleted records in your collection, this makes less sense, but that seems like the exception rather than the rule. Since Rails is about convention, wondering if this might be a good change? Or is there some fundamental reason I'm missing as to why this is a bad idea?
      (I'd be happy to submit a pull request.)

            Assignee:
            Unassigned Unassigned
            Reporter:
            mepatterson mepatterson
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: