[SERVER-36458] Investigate microbenchmark regression with timestamp safe unique indexes Created: 06/Aug/18  Updated: 27/Aug/18  Resolved: 27/Aug/18

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Sulabh Mahajan Assignee: Brian Lane
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
is related to SERVER-34489 Enable new format Unique Index via FCV Closed
Sprint: Storage Engines 2018-08-13
Participants:
Linked BF Score: 0

 Description   

To measure performance impact due to introduction of timestamp safe unique indexes we added microbenchmarks targetting the unique indexes with SERVER-32755.

Initial run of the microbenchmarks show some regression. This ticket is to investigate that.



 Comments   
Comment by Brian Lane [ 27/Aug/18 ]

Based on the comments - looks good to close this one.  

Comment by Asya Kamsky [ 13/Aug/18 ]

sulabh.mahajan thanks for clarification. This seems reasonable to me given the work that was done. I don’t think we should prioritize this over other work we are currently doing.

Comment by Sulabh Mahajan [ 13/Aug/18 ]

asya,
The degradation will be for insert into all unique indexes on the primary. This does NOT apply to _id indexes, _id indexes are not impacted.
This degradation is only when inserting data into the unique indexes, querying unique indexes doesn't show a penalty.

Also degradation is somewhat less pronounced when replication is enabled. eg for insert on standalone vs 1-node replset:

Thread delta(%) - standalone delta(%) - replset
max -8.47% -2.81%
1 -9.17% -4.59%
2 -8.88% -6.28%
4 -8.42% -7.31%
8 -8.47% -2.81%
Comment by Asya Kamsky [ 10/Aug/18 ]

The degradation would be presumably same 10% for every unique index in the collection, is that right?

Just to make sure I'm understanding this correctly, this applies to _id index as well as any other unique index?

 

Comment by Alexander Gorrod [ 10/Aug/18 ]

asya and brian.lane: The change to allow unique indexes to behave correctly regards timestamp reads showed a performance regression on some microbenchmarks as described in this ticket. Can you help us to decide whether to put effort into general performance tuning in order to try to recover the performance difference?

Comment by Sulabh Mahajan [ 09/Aug/18 ]

Insert is slower because the new unique index inserts and removes a prefix key to make sure that nothing in parallel adds a duplicate entry. This is an extra bit of work being done with the new indexes. On disabling the responsible code I get back the performance. The relevant code is:

        // A prefix key is KeyString of index key. It is the component of the index entry that
        // should be unique.                                                     
        const KeyString prefixKey(keyStringVersion(), key, _ordering);           
        WiredTigerItem prefixKeyItem(prefixKey.getBuffer(), prefixKey.getSize());
                                                                                 
        // First phase inserts the prefix key to prohibit concurrent insertions of same key
        setKey(c, prefixKeyItem.Get());                                          
        c->set_value(c, emptyItem.Get());                                        
        ret = WT_OP_CHECK(c->insert(c));                                         
                                                                                 
        // An entry with prefix key already exists. This can happen only during rolling upgrade when
        // both timestamp unsafe and timestamp safe index format keys could be present.
        if (ret == WT_DUPLICATE_KEY) {                                           
            return dupKeyError(key);                                             
        }                                                                        
        invariantWTOK(ret);                                                      
                                                                                 
        // Remove the prefix key, our entry will continue to conflict with any concurrent
        // transactions, but will not conflict with any transaction that begins after this
        // operation commits.                                                    
        setKey(c, prefixKeyItem.Get());                                          
        ret = WT_OP_CHECK(c->remove(c));                                         
        invariantWTOK(ret);   

The test for removing entries from the index is scripted to remove and re-insert entries. So the regression there is from the insertion itself. On disabling the above code, similar to in insert, I was also able to get back performance.

Comment by Henrik Ingo (Inactive) [ 07/Aug/18 ]

Wow you were fast! I created and linked a BF ticket now.

Generated at Thu Feb 08 04:43:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.