[SERVER-16444] Avoid copying data out of WT buffers during tables scans Created: 05/Dec/14  Updated: 01/Oct/15  Resolved: 25/Aug/15

Status: Closed
Project: Core Server
Component/s: Performance, Storage
Affects Version/s: None
Fix Version/s: 3.1.8

Type: Improvement Priority: Major - P3
Reporter: Mathias Stearn Assignee: Max Hirschhorn
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-18725 Use PlanExecutor::enqueue() in favor ... Closed
Related
related to SERVER-18912 Fatal assertion (duplicate key error)... Closed
related to SERVER-20197 find() on collection with large docum... Closed
related to SERVER-20512 Segfault in DeleteStage::work Closed
is related to SERVER-16150 Large in memory collection scan very ... Closed
Tested
Backwards Compatibility: Minor Change
Sprint: Quint Iteration 7, QuInt 8 08/28/15
Participants:
Linked BF Score: 0

 Comments   
Comment by Githook User [ 27/Aug/15 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-16444 Only make BSONObj underlying WSM owned if not mmapv1.

Resolves the performance regression with mmapv1 introduced by 764e0c4
where documents were being copied in PlanStages that perform
invalidations.

Partially reverted changes to DeleteStage and UpdateStage because the
document to return is not guaranteed to be owned after
makeObjOwnedIfNeeded() is called when running with mmapv1.
Branch: master
https://github.com/mongodb/mongo/commit/022cc024da1e4ff5664742befed0059c1ddfa255

Comment by Githook User [ 25/Aug/15 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-16444 Add doc comment to updateWithDamages() function.
Branch: master
https://github.com/mongodb/mongo/commit/559be68757f52d0a5f6e2c2622febea8acf80c35

Comment by Githook User [ 25/Aug/15 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-16444 Copy data in the query subsystem as needed.

A WorkingSetMember in the LOC_AND_OBJ state must be made owned when:

1. Its WorkingSetID is cached across multiple calls to work().

2. Multiple calls to next(), seekExact(), saveState(), etc. are
performed on the same WiredTiger cursor in a single work() call.

No longer necessary to always copy data out of WiredTiger buffers.
Branch: master
https://github.com/mongodb/mongo/commit/764e0c45471d5ca63c708f362be0e6d01ee72eb0

Comment by Githook User [ 25/Aug/15 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-16444 Change updateWithDamages() to return the updated data.

Enable the UpdateStage to perform an in-place update regardless of
whether the data underlying the WorkingSetMember is an unowned BSONObj
or an owned BSONObj.

Change updateWithDamagesSupported() to return true for the
inMemoryExperiment storage engine.
Branch: master
https://github.com/mongodb/mongo/commit/564f8089c0d4541215d1aa31dae331115e68b95f

Comment by Githook User [ 29/Jun/15 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-18912 Document and implement correct capped visibility rules

Fixes a bug introduced by SERVER-16444 changes.
Branch: master
https://github.com/mongodb/mongo/commit/40ddfd261f48ff20c649a6601ff19ed6b806332d

Comment by J Rassi [ 25/Jun/15 ]

Per discussion with Mathias, the work remaining on this ticket is:

  • Ensure that users of PlanExecutor (especially manual-yielding executors) don't rely on the integrity of unowned data between subsequent calls to getNext().
  • Ensure that PlanStages don't rely on the integrity of unowned data between subsequent calls to work().
  • Remove the calls to "data.makeOwned()" in WiredTigerRecordStore::Cursor::next() and WiredTigerRecordStore::Cursor::seekExact(), and briefly investigate performance improvement associated with doing so.

Tentatively assigning to david.storch for 3.1.6; let's finalize the scheduling of this during the planning meeting on Monday.

Comment by Githook User [ 09/Jun/15 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-16444 New API for navigating RecordStores
Branch: master
https://github.com/mongodb/mongo/commit/3b731debe162706cbbfabd9578bbb57ab5a7a7d8

Comment by Githook User [ 09/Jun/15 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-16444 Rename Record class to MmapV1RecordHeader

Better reflects it's purpose and avoids using the valuable name Record for an
MMAPv1-specific class.
Branch: master
https://github.com/mongodb/mongo/commit/f50d1d0b7df924926855badd3cd700653f75f0f8

Generated at Thu Feb 08 03:41:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.