Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-50512

Mongodb migration causes duplicate key :

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.2 Required
    • Component/s: Write Ops
    • Labels:
      None
    • ALL

      Hello,

      In an old project we are trying to migrate from basic spring to spring boot 2.3.1. To do so and because we have a mongo database, we have to migrate from spring-data-mongodb:1.10.18 where this code was written :

      DBCollection contextCollection = this.mongoTemplate.getCollection("productStock");
      BulkWriteOperation builder = contextCollection.initializeUnorderedBulkOperation();
      StockType stockItem = stockMessage.getStockItem();
      
      final BasicDBObject id = new BasicDBObject("storeID", stockItem.getStoreID()).append("productID", stockItem.getProductID());
      BulkWriteRequestBuilder bulkWriteRequestBuilder = builder.find(new BasicDBObject("_id", id));
      HashMap<String, Object> stock = new HashMap<>();
      Date currentDate = Calendar.getInstance().getTime();
      
      stock.put("value", stockItem.getValue());
      if (stockItem.getAssociateDate() != null) {
          stock.put("associateDate", stockItem.getAssociateDate());
      }
      
      if (stockItem.getLastAccessDateSource() != null) {
          stock.put("lastAccessDateSource", stockItem.getLastAccessDateSource());
      
          // check
          BasicDBObject ltLast = new BasicDBObject("$lt", stockItem.getLastAccessDateSource());
          BasicDBList dbList = new BasicDBList();
          dbList.add(new BasicDBObject(stockItem.getStockCategory() + ".lastAccessDateSource", ltLast));
          dbList.add(new BasicDBObject(stockItem.getStockCategory() + ".lastAccessDateSource", null));
          bulkWriteRequestBuilder = builder.find(new BasicDBObject("_id", id).append("$or", dbList));
      } else {
          stock.put("lastAccessDateSource", currentDate);
      }
      
      stock.put("lastUpdateDate", currentDate);
      BasicDBObject set = new BasicDBObject(stockItem.getStockCategory(), new Document(stock));
      bulkWriteRequestBuilder.upsert().updateOne(new BasicDBObject("$set", set));
      builder.execute();
      

       

      to spring-boot-starter-data-mongodb (with mongodb-driver-sync-4.0.4) with this updated code :

       

      Map<String, List<StockType>> mapMultiUpdate = new HashMap<>();
      StockType stockItem = stockMessage.getStockItem();
      
      final Document id = new Document("storeID", stockItem.getStoreID()).append("productID", stockItem.getProductID());
      HashMap<String, Object> stock = new HashMap<>();
      Date currentDate = Calendar.getInstance().getTime();
      Document searchQuery = new Document("_id", id).append("$or", dbList);
      stock.put("value", stockItem.getValue());
      if (stockItem.getAssociateDate() != null) {
          stock.put("associateDate", stockItem.getAssociateDate());
      }
      
      if (stockItem.getLastAccessDateSource() != null) {
          stock.put("lastAccessDateSource", stockItem.getLastAccessDateSource());
      
          // check
          Document ltLast = new Document("$lt", stockItem.getLastAccessDateSource());
          List<Document> dbList = Lists.newArrayList();
          dbList.add(new Document(stockItem.getStockCategory() + ".lastAccessDateSource", ltLast));
          dbList.add(new Document(stockItem.getStockCategory() + ".lastAccessDateSource", null));
      } else {
          stock.put("lastAccessDateSource", currentDate);
      }
      
      
      //Bulk write options
      BulkWriteOptions bulkWriteOptions = new BulkWriteOptions();
      bulkWriteOptions.ordered(false);
      bulkWriteOptions.bypassDocumentValidation(true);
      
      MongoCollection<Document> mongoCollection = this.mongoTemplate.getCollection("productStoreStock");
      mongoCollection.bulkWrite(updateDocuments, bulkWriteOptions);
      
      

       

      But when the new code is executed on an already existing object we get a duplicated key error :

      om.mongodb.MongoBulkWriteException: Bulk write operation error on server localhost:27017. Write errors: [BulkWriteError{index=0, code=11000, message='E11000 duplicate key error collection: test.productStoreStock index: _id_ dup key: { : { storeID: 400, productID: 100000 } }', details={}}]*
      

      We use last version of mongodb 4.2.x

            Assignee:
            eric.sedor@mongodb.com Eric Sedor
            Reporter:
            tiengpub2020@gmail.com Tita Youlika
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: