[SERVER-79433] Experiment - Replace Rank with Id function to remove bottleneck Created: 27/Jul/23  Updated: 15/Aug/23  Resolved: 15/Aug/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Matt Kneiser Assignee: Matt Kneiser
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: Execution NAMR Team 2023-08-07, Execution NAMR Team 2023-08-21
Participants:

 Comments   
Comment by Matt Kneiser [ 15/Aug/23 ]

Drilling into one of the index cloner threads on the donor:

+    7.68%  conn5  [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterNonIdIndex
+    4.47%  conn5  [.] roaring_bitmap_contains
+    4.45%  conn5  [.] __wt_btcur_next
     3.00%  conn5  [.] operator new[] 
     2.96%  conn5  [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterIdIndexAndUpdateId 
     2.86%  conn5  [.] mongo::BufReader::skip 
     2.62%  conn5  [.] __curfile_next 
     2.17%  conn5  [.] mongo::IndexScan::doWork 
     2.07%  conn5  [.] __wt_cursor_get_keyv 
     2.03%  conn5  [.] mongo::ProjectionStageDefault::transform 
     1.71%  conn5  [.] __wt_txn_read 
     1.68%  conn5  [.] mongo::BSONArrayBuilderBase<mongo::BSONArrayBuilder, mongo::BSONObjBuilder>::append<mongo::BSONObj>
     1.60%  conn5  [.] tc_malloc
     1.51%  conn5  [.] tc_deletearray_sized
     1.48%  conn5  [.] mongo::MutableDocument::storage
     1.43%  conn5  [.] mongo::DocumentStorage::appendField<mongo::StringData>
     1.43%  conn5  [.] mongo::BSONObjBuilder::obj<mongo::BSONObj::DefaultSizeTrait>
     1.32%  conn5  [.] mongo::BSONObjBuilderBase<mongo::BSONObjBuilder, mongo::BufBuilder>::_done
     1.32%  conn5  [.] mongo::projection_executor::ProjectionNode::applyProjections
     1.19%  conn5  [.] snappy::internal::CompressFragment 

Comment by Matt Kneiser [ 15/Aug/23 ]

With the rank function removed, here is what perf says about the donor's time:

$ sudo perf report -G --no-children -i donor.perf
+    3.88%  conn3            [.] snappy::internal::CompressFragment
+    0.94%  conn5            [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterNonIdIndex
+    0.90%  conn4            [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterIdIndexAndUpdateId
+    0.89%  conn6            [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterNonIdIndex
+    0.87%  conn7            [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterNonIdIndex
+    0.83%  conn9            [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterNonIdIndex
+    0.73%  conn10           [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterNonIdIndex
+    0.66%  conn10           [.] operator new[]
+    0.60%  conn9            [.] operator new[]
+    0.57%  conn6            [.] __wt_btcur_next
+    0.55%  conn5            [.] roaring_bitmap_contains
+    0.55%  conn5            [.] __wt_btcur_next
     0.54%  conn6            [.] roaring_bitmap_contains
     0.52%  conn4            [.] __wt_btcur_next
     0.51%  conn8            [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterNonIdIndex
     0.49%  conn7            [.] operator new[]
     0.48%  conn7            [.] __wt_btcur_next
     0.48%  conn7            [.] roaring_bitmap_contains
     0.46%  conn9            [.] __wt_btcur_next
     0.45%  conn6            [.] operator new[]
     0.44%  conn8            [.] operator new[]
     0.44%  conn10           [.] __wt_btcur_next
     0.43%  read-ahead-s 1   [.] snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter>
     0.42%  read-ahead-s 7   [.] snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter>
     0.42%  read-ahead-s 2   [.] snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter>
     0.41%  read-ahead-s 5   [.] snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter>
     0.41%  read-ahead-s 8   [.] snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter>
     0.41%  read-ahead-s 6   [.] snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter>
     0.41%  conn8            [.] mongo::(anonymous namespace)::WiredTigerIndexCursorBase::filterIdIndexAndUpdateId
     0.41%  read-ahead-s 3   [.] snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter>
     0.41%  read-ahead-s 4   [.] snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter> 

Recipient:

sudo perf report -G --no-children -i recipient.perf
+   57.38%  CloneFi.lection  [.] snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter>
+   32.10%  CloneFi.lection  [.] mongo::(anonymous namespace)::ValidateBuffer<false, mongo::(anonymous namespace)::DefaultValidator>::_validateIterative
     6.06%  CloneFi.lection  [.] mongo::(anonymous namespace)::parseCursorResponse
     1.48%  CloneFi.lection  [.] mongo::(anonymous namespace)::ValidateBuffer<false, mongo::(anonymous namespace)::DefaultValidator>::_validateSpecial
     0.74%  CloneFi.lection  [.] mongo::Fetcher::_callbackExhaust
     0.55%  CloneFi.lection  [.] mongo::Fetcher::QueryResponse::QueryResponse
     0.36%  CloneFi.lection  [.] std::_Function_handler<void (mongo::StatusWith<mongo::Fetcher::QueryResponse> const&, mongo::Fetcher::NextAction*, mongo::BSONObjBuilder*), mongo::repl::(anonymous namespace)::CmdCloneFilteredCollection::run(mongo::OperationContext*, mongo::DatabaseName const&, mongo::BSONObj const&, mongo::BSONObjBuilder&)::{lambda(mongo::StatusWith<mongo::Fetcher::QueryResponse> const&, mongo::Fetcher::NextAction*, mongo::BSONObjBuilder*)#1}>::_M_invoke
     0.33%  CloneFi.lection  [.] mongo::BSONElement::computeSize
     0.29%  CloneFi.lection  [.] std::vector<mongo::BSONObj, std::allocator<mongo::BSONObj> >::_M_realloc_insert<mongo::BSONObj>
     0.14%  CloneFi.lection  [.] mongo::BSONElement::embeddedObjectUserCheck
     0.08%  CloneFi.lection  [.] mongo::BSONElement::Obj
     0.04%  CloneFi.lection  [.] asio::detail::reactive_socket_recv_op<asio::mutable_buffers_1, asio::detail::read_op<asio::basic_stream_socket<asio::generic::stream_protocol>, asio::mutable_buffers_1, asio::mutable_buffer const*, asio::detail::transfer_all_t, mongo::transport::UseFuture::Adapter<std::error_code, unsigned long>::Handler> >::do_complete
     0.03%  CloneFi.lection  [.] asio::asio_handler_deallocate
     0.03%  CloneFi.lection  [.] asio::detail::reactive_socket_recv_op_base<asio::mutable_buffers_1>::do_perform
     0.03%  CloneFi.lection  [.] mongo::Promise<unsigned long>::breakPromiseIfNeeded
     0.03%  CloneFi.lection  [.] tc_deletearray_sized

 

Generated at Thu Feb 08 06:40:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.