[DOCS-871] Replication Internals: threaded application of operations Created: 11/Dec/12  Updated: 22/Feb/13  Resolved: 20/Feb/13

Status: Closed
Project: Documentation
Component/s: None
Affects Version/s: None
Fix Version/s: v1.1, mongodb-2.2

Type: Task Priority: Blocker - P1
Reporter: Sam Kleinman (Inactive) Assignee: Bob Grabar
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:
Days since reply: 10 years, 51 weeks, 6 days ago

 Description   

In order to support multithreaded replication, we apply operations in batches on secondary nodes. The batching enables us to guarantee that read queries on secondaries do not see operations applied out of order.

Each batch is divided up among many threads, and while they are writing, we block all readers, even if the writing threads yield. The multithreaded writing allows for concurrent CPU usage.

In addition, before the batched writing begins, we prefetch all the memory pages which contain records we are about to write, including the pages we need to traverse in all indexes. This prefetch stage provides concurrency for IO, and is probably providing the majority of the speedup that users are seeing. It also allows us to hold the write lock for a minimal amount of time, since there should not be any page faults taken in the write phase.



 Comments   
Comment by auto [ 21/Feb/13 ]

Author:

{u'date': u'2013-02-21T20:49:14Z', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: DOCS-871: edits
Branch: master
https://github.com/mongodb/docs/commit/b57d6cfddeabe51db8a111707be58d032b818a77

Comment by Scott Hernandez (Inactive) [ 21/Feb/13 ]

For some of these sections we should link to the settings which control them:

+To help improve the performance of operation application, MongoDB
+fetches all the memory pages that hold data and indexes that the
+operations in the batch will affect. The prefetch stage minimizes the
+amount of time MongoDB must hold the write lock to apply operations.

This should link to the preFetch setting, see DOCS-417.

Comment by auto [ 21/Feb/13 ]

Author:

{u'date': u'2013-02-21T19:46:40Z', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: DOCS-871: edits
Branch: master
https://github.com/mongodb/docs/commit/1f8158fc436f82a1a3147fd71ac56d7df1989f69

Comment by auto [ 21/Feb/13 ]

Author:

{u'date': u'2013-02-21T19:38:54Z', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: merge: DOCS-871
Branch: master
https://github.com/mongodb/docs/commit/a9f325fa5aed9fb2b2a7670f4cd30a9413af3977

Comment by auto [ 21/Feb/13 ]

Author:

{u'date': u'2013-02-19T18:08:31Z', u'name': u'Bob Grabar', u'email': u'bob.grabar@10gen.com'}

Message: DOCS-871 multithreaded replication
Branch: master
https://github.com/mongodb/docs/commit/b7eed97549384fc0d9073a828965370c780df871

Generated at Thu Feb 08 07:39:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.