Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-12208

Got "exception: btree: key+recloc already in index code:10287". Related document vanished

    • Type: Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 2.4.4
    • Component/s: Index Maintenance
    • Labels:
    • Environment:
      Ubuntu 12.04 - On Digital Ocean - Single MongoDB node - Very low load, very small DB,
    • Linux
    • Hide

      N/A

      Show
      N/A

      We got the following on mongodb.log:

      Thu Dec 26 14:48:51.331 [conn265] update reef.companies query: { _id: ObjectId('521f7f99e4b00b44b9987a59') } update: { $push: { capitals: { legal: { proceeding: { re
      fid: null, state: "none" } }, id: "ecb646fa-9cb8-4a53-bdf0-1b4821ca4f4e", shareholders: [ { entity: { company: { refid: "52bc3f7fe4b07f4f0a49fc4a", name: "EOG Canada
       Holdings ULC (CANCELADA)" } }, amount_mo: 2125000000 }, { entity: { company: { refid: "52bc2d36e4b07f4f0a49fc29", name: "EOG Canada Investments ULC" } }, amount_mo:
       6375000000 } ], total_mo: 8500000000, since_dd: new Date(1323226800000), inc_mo: 8500000000, is_statute_change: true, is_art299: true } } } idhack:1 keyUpdates:0 ex
      ception: btree: key+recloc already in index code:10287 locks(micros) w:1905 1ms
      

      Then, right after that operation, the referenced document disappeared from the DB. Running db.companies.find({"_id" : ObjectId("521f7f99e4b00b44b9987a59")}); returns no results.

      We tried db.runCommand({validate:"companies"}); and everything looks OK.

      We also run a reIndex() on the collection. No errors. The document still gone.

      We know the doc was lost due to the above error because that's exactly when it disappeared. And our application codebase does not include a single "remove/delete" operation (we only do "soft" deletes).

      Additional info:

      • HINT: We DO have textSearchEnabled=true
      • It's running on a VM (on Digital Ocean)
      • It's a single node installation
      • Very very low load (a single user entering data via a UI)
      • Only 15 documents on that collection
      • The one being updated with $push (which was lost) was 3521 bytes
      • Config file:
        $ cat /etc/mongodb.conf 
        logappend=true
        master=true
        setParameter=textSearchEnabled=true
        rest=true
        logpath=/var/log/mongodb/mongodb.log
        dbpath=/var/lib/mongodb
        

      Right now we restored the lost document from our last backup. However, we are very concerned about losing data in the future. Specially since this is our first project using MongoDB and with very little volume and load we've already lost data.

      Thanks

            Assignee:
            Unassigned Unassigned
            Reporter:
            dobladez Fernando Dobladez
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: