Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17487

cloner dropDups removes _id entries belonging to other records

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.1, 3.1.0
    • Component/s: Replication, WiredTiger
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:
    • Steps To Reproduce:
      Hide

      -replica set
      -insert 2M doc
      -run mix traffic (insert/deletion, update) and re-sync one member

      • all wiredTiger as storageEngine
      • multiple member replica set
      Show
      -replica set -insert 2M doc -run mix traffic (insert/deletion, update) and re-sync one member all wiredTiger as storageEngine multiple member replica set

      Description

      after re-sync is done, the _id index count, secondary index count and data table count could be different.

              "index" : 2000000,
              "secondary_index" : 2047102,
              "table" : 2047102

      and log file show (with Dan's patch for more printout)

      2015-03-06T01:05:28.851+0000 I REPL     [rsSync] ******
      2015-03-06T01:05:28.851+0000 I REPL     [rsSync] creating replication oplog of size: 4094MB...
      2015-03-06T01:05:28.853+0000 I STORAGE  [rsSync] Starting WiredTigerRecordStoreThread local.oplog.rs
      2015-03-06T01:05:28.870+0000 I REPL     [rsSync] ******
      2015-03-06T01:05:28.870+0000 I REPL     [rsSync] initial sync pending
      2015-03-06T01:05:28.870+0000 I REPL     [ReplicationExecutor] syncing from: 10.1.1.107:27017
      2015-03-06T01:05:28.874+0000 I REPL     [rsSync] initial sync drop all databases
      2015-03-06T01:05:28.874+0000 I STORAGE  [rsSync] dropAllDatabasesExceptLocal 1
      2015-03-06T01:05:28.874+0000 I REPL     [rsSync] initial sync clone all databases
      2015-03-06T01:05:28.875+0000 I REPL     [rsSync] initial sync cloning db: sbtest
      2015-03-06T01:05:42.605+0000 I INDEX    [rsSync] build index on: sbtest.sbtest1 properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "sbtest.sbtest1" }
      2015-03-06T01:05:42.605+0000 I INDEX    [rsSync]         building index using bulk method
      2015-03-06T01:05:47.969+0000 I INDEX    [rsSync] build index done.  scanned 2047128 total records. 5 secs
      2015-03-06T01:05:47.969+0000 I STORAGE  [rsSync] Deleting: RecordId(1955618)
      2015-03-06T01:05:47.969+0000 I STORAGE  [rsSync] Deleting: RecordId(1955661)
      2015-03-06T01:05:47.970+0000 I STORAGE  [rsSync] Deleting: RecordId(1955710)
      2015-03-06T01:05:47.970+0000 I STORAGE  [rsSync] Deleting: RecordId(1955778)
      2015-03-06T01:05:47.970+0000 I STORAGE  [rsSync] Deleting: RecordId(1955802)
      2015-03-06T01:05:47.970+0000 I STORAGE  [rsSync] Deleting: RecordId(1955996)
      2015-03-06T01:05:47.970+0000 I STORAGE  [rsSync] Deleting: RecordId(1956089)
      2015-03-06T01:05:47.970+0000 I STORAGE  [rsSync] Deleting: RecordId(1956102)
      2015-03-06T01:05:47.970+0000 I STORAGE  [rsSync] Deleting: RecordId(1956127)
      2015-03-06T01:05:47.970+0000 I STORAGE  [rsSync] Deleting: RecordId(1956187)
      2015-03-06T01:05:47.970+0000 I STORAGE  [rsSync] Deleting: RecordId(1956305)
      ....
       
      2015-03-06T01:05:49.275+0000 I STORAGE  [rsSync] Deleting: RecordId(2047128)
      2015-03-06T01:05:49.275+0000 I STORAGE  [rsSync] index build dropped: 47128 dups
      2015-03-06T01:05:49.449+0000 I REPL     [rsSync] initial sync data copy, starting syncup
      2015-03-06T01:05:49.449+0000 I REPL     [rsSync] oplog sync 1 of 3

      • not sure why there is dup for _id
      • dropped dups are not removed from data table and secondary index tree
      • saw one case when mixed traffic stopped, re-sync still have issue with dups and out of sync, but cannot reproduce it now.
      1. cloner_dups.js
        2 kB
        Dan Pasette
      2. SERVER-17487.patch
        1 kB
        David Hows

        Activity

        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

        Message: SERVER-17487 Throttle bg insert thread in test
        Branch: master
        https://github.com/mongodb/mongo/commit/06f4ea80a51ccea8da275e095cc5061707263957

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'} Message: SERVER-17487 Throttle bg insert thread in test Branch: master https://github.com/mongodb/mongo/commit/06f4ea80a51ccea8da275e095cc5061707263957
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

        Message: SERVER-17487 raise oplog size in test
        Branch: master
        https://github.com/mongodb/mongo/commit/4c7eb08931a607e9879aaf088998d3bff3720e92

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'} Message: SERVER-17487 raise oplog size in test Branch: master https://github.com/mongodb/mongo/commit/4c7eb08931a607e9879aaf088998d3bff3720e92
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

        Message: SERVER-17487 raise oplog size in test

        (cherry picked from commit 4c7eb08931a607e9879aaf088998d3bff3720e92)
        Branch: v3.0
        https://github.com/mongodb/mongo/commit/d00d2bc873ebeca1536f578d8a5255e4f776aad3

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'} Message: SERVER-17487 raise oplog size in test (cherry picked from commit 4c7eb08931a607e9879aaf088998d3bff3720e92) Branch: v3.0 https://github.com/mongodb/mongo/commit/d00d2bc873ebeca1536f578d8a5255e4f776aad3
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

        Message: SERVER-17487 Throttle bg insert thread in test

        (cherry picked from commit 06f4ea80a51ccea8da275e095cc5061707263957)
        Branch: v3.0
        https://github.com/mongodb/mongo/commit/03d7d9d46179425212ce1840c58c4325d450b950

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'} Message: SERVER-17487 Throttle bg insert thread in test (cherry picked from commit 06f4ea80a51ccea8da275e095cc5061707263957) Branch: v3.0 https://github.com/mongodb/mongo/commit/03d7d9d46179425212ce1840c58c4325d450b950
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

        Message: SERVER-17487 Raise awaitReplication timeout in test

        (cherry picked from commit 5f64a1c4c3eb6698dfa2e721972a158202229bb0)
        Branch: v3.0
        https://github.com/mongodb/mongo/commit/23d882d8f6423e27e4f3a01f7829861e77ba2e0e

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'} Message: SERVER-17487 Raise awaitReplication timeout in test (cherry picked from commit 5f64a1c4c3eb6698dfa2e721972a158202229bb0) Branch: v3.0 https://github.com/mongodb/mongo/commit/23d882d8f6423e27e4f3a01f7829861e77ba2e0e

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: