[SERVER-25268] OplogBufferCollection should cache last pushed/popped values and tryPop() should not remove documents from collection Created: 25/Jul/16 Updated: 25/Jan/17 Resolved: 20/Sep/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | None |
| Fix Version/s: | 3.3.14 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Judah Schvimer | Assignee: | Benety Goh |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Sprint: | Repl 2016-08-29, Repl 2016-09-19, Repl 2016-10-10 | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
The OplogBufferCollection appears to be about 20x slower than the OplogBufferInMemory. To improve it, we can: Data can be found here: Compared to here: |
| Comments |
| Comment by Githook User [ 20/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Githook User [ 20/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Githook User [ 20/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Githook User [ 20/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Githook User [ 16/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Githook User [ 15/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Githook User [ 15/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Githook User [ 15/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Githook User [ 15/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Githook User [ 15/Sep/16 ] |
|
Author: {u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}Message: |
| Comment by Scott Hernandez (Inactive) [ 01/Sep/16 ] |
|
Mathias suggests using the SortedFileWriter which creates temp files in insertion order, has an iterator (more/next) to retrieve documents, and also supports compression. This would allow us to reduce locking and other storage/contention associated with using a collection. It would also mean reduced IO, and allocation in mmapv1 since dropping collections only frees extents not files to the os/fs. |