[SERVER-65364] Update logic should pass info about which fields changed to index layer Created: 08/Apr/22  Updated: 29/Oct/23  Resolved: 22/Dec/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.3.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Mathias Stearn Assignee: Alberto Massari
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-70984 Decouple the concepts "in-place updat... Closed
Problem/Incident
Related
related to SERVER-74136 UpdateDriver can miss index update Closed
is related to SERVER-47140 Use diffing for full document replace... Backlog
Tested
Backwards Compatibility: Fully Compatible
Backport Requested:
v6.0
Sprint: QE 2022-09-19, QE 2022-10-03, QE 2022-10-17, QE 2022-10-31, QE 2022-11-14, QE 2022-11-28, QE 2022-12-12, QE 2022-12-26
Participants:
Linked BF Score: 114
Story Points: 5

 Description   

Right now when we do an update the index layer gets the pre- and post-image document of the update. Then it will extract keys from both and perform a symmetric set difference to see which keys need to be added or removed updated in the index. And it does this for each index. This is wasteful for two reasons:

  1. We should be able to completely skip over indexes of fields that didn't change.
  2. For indexes that cover multiple paths independently (eg wildcard and columnar, but not text), they can completely skip over fields that they know didn't change.

Since most updates don't update most fields, most of the index work we do on updates is wasteful work to detect which fields didn't change.



 Comments   
Comment by Ivan Fefer [ 17/Feb/23 ]

Requesting backport to fix BF-27632.

Comment by Githook User [ 21/Dec/22 ]

Author:

{'name': 'Alberto Massari', 'email': 'alberto.massari@mongodb.com', 'username': 'albymassari'}

Message: SERVER-65364 Allow fine-grained selection of indexes to update
Branch: master
https://github.com/mongodb/mongo/commit/5977e706431fd5705b59115ec0e0d2d7a2203246

Comment by Mathias Stearn [ 12/Jul/22 ]

Your plan only helps with issue 1. To address issue 2, it will need to propagate info about which fields are changing down to the indexing layer at the point that it generates the differing keys, at least passing through here

Comment by James Wahlin [ 08/Apr/22 ]

Sending to Query Execution as this work is done at execution time. Maybe worth debating whether this belongs to QE or SE.

Comment by Mathias Stearn [ 08/Apr/22 ]

This seems like something at the intersection of Query Opt, Query Exec, and Storage Exec. I thought Query Opt was the best owner, but feel free to reassign if I guessed wrongly.

Generated at Thu Feb 08 06:02:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.