[SERVER-65364] Update logic should pass info about which fields changed to index layer Created: 08/Apr/22 Updated: 29/Oct/23 Resolved: 22/Dec/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 6.3.0-rc0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Mathias Stearn | Assignee: | Alberto Massari |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||
| Backport Requested: |
v6.0
|
||||||||||||||||||||||||||||||||
| Sprint: | QE 2022-09-19, QE 2022-10-03, QE 2022-10-17, QE 2022-10-31, QE 2022-11-14, QE 2022-11-28, QE 2022-12-12, QE 2022-12-26 | ||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||
| Linked BF Score: | 114 | ||||||||||||||||||||||||||||||||
| Story Points: | 5 | ||||||||||||||||||||||||||||||||
| Description |
|
Right now when we do an update the index layer gets the pre- and post-image document of the update. Then it will extract keys from both and perform a symmetric set difference to see which keys need to be added or removed updated in the index. And it does this for each index. This is wasteful for two reasons:
Since most updates don't update most fields, most of the index work we do on updates is wasteful work to detect which fields didn't change. |
| Comments |
| Comment by Ivan Fefer [ 17/Feb/23 ] |
|
Requesting backport to fix BF-27632. |
| Comment by Githook User [ 21/Dec/22 ] |
|
Author: {'name': 'Alberto Massari', 'email': 'alberto.massari@mongodb.com', 'username': 'albymassari'}Message: |
| Comment by Mathias Stearn [ 12/Jul/22 ] |
|
Your plan only helps with issue 1. To address issue 2, it will need to propagate info about which fields are changing down to the indexing layer at the point that it generates the differing keys, at least passing through here |
| Comment by James Wahlin [ 08/Apr/22 ] |
|
Sending to Query Execution as this work is done at execution time. Maybe worth debating whether this belongs to QE or SE. |
| Comment by Mathias Stearn [ 08/Apr/22 ] |
|
This seems like something at the intersection of Query Opt, Query Exec, and Storage Exec. I thought Query Opt was the best owner, but feel free to reassign if I guessed wrongly. |