[SERVER-7235] oplog documents should indicate which indexes were touched Created: 02/Oct/12  Updated: 23/Feb/15  Resolved: 23/Feb/15

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 2.2.0
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Eric Milkie Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-6890 prefetch all index pages for replicat... Closed
Participants:

 Description   

If each oplog document had a hint of which indexes were touched, we could use this information on a secondary to intelligently prefetch index pages.

Right now we always prefetch for all indexes, unless the administrator sets a manual flag to completely turn off the feature. The manual setting plus the all-or-nothing approach that the current code takes is suboptimal. On the primary, for an update operation, it knows if it had to touch indexes, so it could include this info when it logs the operation into its oplog.

We could then make the --replIndexPrefetch command line and config file option a no-op.



 Comments   
Comment by Henrik Ingo (Inactive) [ 11/Aug/14 ]

I see what you're trying to do, yet I hesitate a bit with the idea of polluting the oplog with data that is only relevant to an internal optimization.

Actually, knowing which index pages have been touched could be interesting from a performance monitoring perspective. But this is completely unrelated to replication, so from a user experience point of view that should be a separate log. (...for example, performance monitoring data should be available also on a standalone server.)

For an alternative path to prefetching, we could discuss the Faker code in Facebook's MySQL fork:
http://dom.as/2012/09/04/faker/
http://www.percona.com/doc/percona-server/5.5/management/innodb_fake_changes.html

We can discuss more details over phone, just wanted to make these general notes while hitting this in my triage batch.

Comment by Scott Hernandez (Inactive) [ 24/May/13 ]

We need to be careful since it is possible that the replica doesn't have the same indexes. Might be good to just indicate which indexed fields were changed, and then get all affected indexes, but if the replica has different indexed fields (esp. not covered on the primary) this could be less than optimal.

Keeping a list of the changed fields, in dot-notation, might solve this and provide a nice list of changes.

Generated at Thu Feb 08 03:13:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.