[SERVER-7235] oplog documents should indicate which indexes were touched Created: 02/Oct/12 Updated: 23/Feb/15 Resolved: 23/Feb/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | 2.2.0 |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Eric Milkie | Assignee: | Unassigned |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Participants: | |||||||||
| Description |
|
If each oplog document had a hint of which indexes were touched, we could use this information on a secondary to intelligently prefetch index pages. Right now we always prefetch for all indexes, unless the administrator sets a manual flag to completely turn off the feature. The manual setting plus the all-or-nothing approach that the current code takes is suboptimal. On the primary, for an update operation, it knows if it had to touch indexes, so it could include this info when it logs the operation into its oplog. We could then make the --replIndexPrefetch command line and config file option a no-op. |
| Comments |
| Comment by Henrik Ingo (Inactive) [ 11/Aug/14 ] |
|
I see what you're trying to do, yet I hesitate a bit with the idea of polluting the oplog with data that is only relevant to an internal optimization. Actually, knowing which index pages have been touched could be interesting from a performance monitoring perspective. But this is completely unrelated to replication, so from a user experience point of view that should be a separate log. (...for example, performance monitoring data should be available also on a standalone server.) For an alternative path to prefetching, we could discuss the Faker code in Facebook's MySQL fork: We can discuss more details over phone, just wanted to make these general notes while hitting this in my triage batch. |
| Comment by Scott Hernandez (Inactive) [ 24/May/13 ] |
|
We need to be careful since it is possible that the replica doesn't have the same indexes. Might be good to just indicate which indexed fields were changed, and then get all affected indexes, but if the replica has different indexed fields (esp. not covered on the primary) this could be less than optimal. Keeping a list of the changed fields, in dot-notation, might solve this and provide a nice list of changes. |