Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-36428

WriteConflict inside a transaction on any concurrent db access

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.0.0
    • Component/s: None
    • None
    • Environment:
      Latest Node driver, Node v10.6.0, MongoDB 4.0 (local database, replica set with one member).
      Reproduced on Windows 10 and CentOS 7.4
    • ALL

      Multi-document transactions in their current implementation (MongoDB 4.0) work awfully unstable. I'm using transactions to consistently load high volumes of heterogeneous data (about 30 GB / 300 000 000 documents) and analyse it. I use latest Node driver, Node v10.6.0. Local database, replica set with one member only.

      A transaction is started like that:

       

      session.startTransaction({
        readConcern: {level: 'snapshot'},
        writeConcern: {w: 1}
      })
      
      await db.collection('collection1').insertMany({...})
      await db.collection('collection2').insertMany({...})
      
      await session.commitTransaction()

       

       

      During the last few weeks I experienced a full range of exceptions, just can't make it work stable. Here are some of my observations:

      1. When I run transaction in parallel with any database modification command (like create another collection or schedule building index), I always get WriteConflict exception: "MongoError: WriteConflict" (No details, only this message). I run the command from another application and it operates on separate collection, so in theory there can't be any write conflicts.

      2. When I try to run two transactions in parallel (from two different apps, on separate collections), I always get the same WriteConflict exception in one of the applications. Therefore, I can see no way to start two parallel transactions (on two different collections from separate applications), and that definitely kills the idea of using this code in production.

      3. Sometimes (once or twice a day) I get errors like MongoError: Transaction 161453 has been aborted. I can't figure out the reason behind it.

      4. If I try to run two write operations from the same transaction simultaneously, I get MongoError: Given transaction number 231 does not match any in-progress transactions.

      So this works:

       

      session.startTransaction({readConcern: {level: 'snapshot'},writeConcern: {w: 1}})
      
      await db.collection('collection1').insertMany({...})
      await db.collection('collection2').insertMany({...})
      
      await session.commitTransaction()
      

      and this code will eventually throw a "does not match any in-progress transactions" exception:

       

      session.startTransaction({readConcern: {level: 'snapshot'},writeConcern: {w: 1}})
      
      await Promise.all([
        db.collection('collection1').insertMany({...}),
        db.collection('collection2').insertMany({...})
      ])
      
      await session.commitTransaction()
      

      5. Calling db.stats() or db.collection.stats() (even on another database) frequently leads to MongoError: Unable to acquire lock '{8543186066763233601: Database, 1625657039122151745}' within a max lock request timeout of '5ms' milliseconds. error in the running transaction. Is there any way to increase lock waiting timeout? And why calling stats() on another collection/database leads to an exception?

       

      I get the same results on both my Windows 10 laptop and CentOS 7.4 server.

      Is there any way to get rid of those annoying exceptions? Maybe I'm using wrong read isolation level? Are there any parameters that can be tuned?

        1. test.sql.js
          4 kB
        2. test.js
          4 kB

            Assignee:
            alyson.cabral@mongodb.com Alyson Cabral (Inactive)
            Reporter:
            kreig Vyacheslav Stroy
            Votes:
            0 Vote for this issue
            Watchers:
            25 Start watching this issue

              Created:
              Updated:
              Resolved: