[SERVER-65365] Update logic should pass info about which fields changed to index layer Created: 08/Apr/22  Updated: 06/Dec/22  Resolved: 08/Apr/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Mathias Stearn Assignee: Backlog - Query Optimization
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Optimization
Participants:

 Description   

Right now when we do an update the index layer gets the pre- and post-image document of the update. Then it will extract keys from both and perform a symmetric set difference to see which keys need to be added or removed updated in the index. And it does this for each index. This is wasteful for two reasons:

1. We should be able to completely skip over indexes of fields that didn't change.
2. For indexes that cover multiple paths independently (eg wildcard and columnar, but not text), they can completely skip over fields that they know didn't change.

Since most updates don't update most fields, most of the index work we do on updates is wasteful work to detect which fields didn't change.



 Comments   
Comment by Mathias Stearn [ 08/Apr/22 ]

Sorry, my browser decided to submit this ticket twice.

Generated at Thu Feb 08 06:02:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.