[SERVER-8868] Concurrent index recalculation Created: 06/Mar/13  Updated: 06/Dec/22  Resolved: 22/Feb/16

Status: Closed
Project: Core Server
Component/s: Concurrency, Index Maintenance
Affects Version/s: 2.4.0-rc1
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: kyle wolfe Assignee: Backlog - Storage Execution Team
Resolution: Won't Fix Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Storage Execution
Participants:

 Description   

Index1:
------
op_date: 1,
cust_id: 1,
log.event_label: 1,
log.event_result: 1


Index2:
------
import_id: 1
gc_id: 1

{unique: true}

------

insert statement(php):
$op = array(
"event_label" => (integer) $data[2],
"event_result" => (integer) $data[3]
);
$this->mongoDb->$coll->update(array("import_id" => $importId, "gc_id" => $data[0], "op_date" => $callDate), array('$push' => array('log' => $op)), array("upsert" => true, "w" => 0));

Given this upsert statement, on an able server with several cores and high i/o, a bottlekneck is reached due to the recalculation of indexes. Running multiple of the same process will not increase performance as the single core is locked up for index recalculation.

Is it possible to thread out index recalculation? In my case, I have 24 cores, over 1gb/s write speed and well over enough ram to fit my active set, but can't use the other cores for these upserts.



 Comments   
Comment by James Wahlin [ 27/Mar/13 ]

Kyle, I am reopening this. I had misunderstood this as being a request to debug your update scenario where this is instead a feature request for concurrent index calculation.

Comment by James Wahlin [ 27/Mar/13 ]

I am closing this issue as conversation is progressing in the MongoDB Google group: https://groups.google.com/forum/?fromgroups=#!topic/mongodb-user/fAQkM45wXDs

Kyle, if you see a reason to maintain this ticket as well please feel free to update and reopen.

Thanks,
James

Generated at Thu Feb 08 03:18:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.