Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-630

Documentation for countDocuments MUST mention estimatedDocumentCount

    XMLWordPrintable

    Details

    • Type: Task
    • Status: In Progress
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • # Replies:
      7
    • Last comment by Customer:
      false

      Description

      The estimatedDocumentCount helper is not easily discoverable by developers migrating from the count helper. For example, estimatedDocumentCount doesn't autocomplete in IDEs when searching for helpers related to count. This is a problem since applications that only need an estimated count can take a large performance hit when using countDocuments for that purpose.

      Drivers MUST add text to the documentation for countDocuments directing these users to estimatedDocumentCount. Suggested text:

      "For a fast count of the total documents in a collection see estimatedDocumentCount" with a link to the documentation for that helper.

      Original report below
      ----------------------------

      It was suggested I file a DRIVERS with some feedback based on a pair of observations converting applications from mgo to mongo-go-driver, although what follows is perhaps general to DRIVERS-501.

      In two independent instances, application developers where looking to upgrade to drivers which included the new Count APIs from DRIVERS-501. Developers matched up existing Count() with the new API and, probably via auto-complete on the driver API, intuitively selected CountDocuments() in the case of mongo-go-driver and presumed was equivalent to what the past driver (mgo) was doing.

      I'm guessing that the developer intuitively picking CountDocuments() is maybe viewed as a "success" from aim of DRIVERS-501. But, in both of these independent cases it had severe unexpected consequences. These were Count()'s with no query predicates. So, internally to mongod, we were getting the "fast path" constant time document count from collection metadata (and were fine with the inaccuracy). When switched to CountDocuments() and the agg, we were now getting a scan instead.

      In particular, one occurrence was against the oplog collection: the worst case. While other user collections on modern mongod versions would at least complete the agg with only an _id index scan, on the oplog it's a full document scan. I understand the scan's count is accurate now.. but IMO don't think is ever what a user would want/expect coming from constant time performance before.

      I'm unclear exactly how best to head off future cases. Maybe some API documentation on CountDocuments() that prominently calls out that it's not constant time like EstimatedCount()? Unclear if this would've helped or not. But the two occurrences of falling into the same trap makes me wonder if it could be a common mishap for users.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Days since reply:
                  33 weeks ago
                  Date of 1st Reply: