Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-3858

Replicaset operation way too slow.

    • Type: Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.4.1
    • Component/s: Performance
    • Labels:
      None

      Detailed steps to reproduce the problem?

      run this python code,

       

      import time

      import pymongo

      url = "mongodb://192.168.1.31:27017,192,168.1.161:27017/testdb?replicaSet=shard01"

      class MyMongoClient(pymongo.MongoClient):

      def _init_(self, *args, **kwargs):
      super(MyMongoClient, self)._init_(*args, **kwargs)
      self.cached_server = None

      def _select_server(self, server_selector, session, address=None):
      if not self.cached_server:
      self.cached_server = super(MyMongoClient, self)._select_server(server_selector, session, address)
      return self.cached_server

      use_replicaset = 1
      if use_replicaset:
      mongo = MyMongoClient(url)
      else:
      mongo = MyMongoClient(
      host='192.168.1.161',
      directConnection=True,
      )

      db = mongo.get_database('testdb')

      coll = db.get_collection(
      'ums_user', write_concern=pymongo.WriteConcern(w=1, j=False))

      t1 = time.time()
      for i in range(100):
      rec = {'name': f'Tom

      {i}

      '}
      coll.insert_one(rec)

      t2 = time.time()
      print(t2 - t1)

      If using replica, it would cost 3 seconds with {w: 1, j: false}, 

      equavalent java code would be completed with around 400 milli seconds.

      public class MongoClientTest {

      public static void main(String[] args) {
      boolean useReplicaSet = false;
      MongoClient mongoClient = null;
      if (useReplicaSet)

      { mongoClient = MongoClients.create("mongodb://192.168.1.161,192,168.1.31/testdb?replicaset=shard01"); }

      else

      { mongoClient = MongoClients.create("mongodb://192.168.1.161:27017/testdb?directConnection=true"); }

      MongoDatabase db = mongoClient.getDatabase("testdb");
      MongoCollection<Document> collection = db.getCollection("ums_user").withWriteConcern(WriteConcern.W1.withJournal(false));
      long t1 = System.currentTimeMillis();
      for (int i = 0; i < 100; i++) {
      Document document = new Document();
      document.put("name", "Tom " + i);
      for (int j = 0; j < 20; j++)

      { document.put("attr" + j, "val" + j); }

      collection.insertOne(document);
      }

      Document document = new Document();
      document.put("name", "Jerry");
      db.getCollection("ums_user").withWriteConcern(WriteConcern.W2.withJournal(true)).insertOne(document);
      long t2 = System.currentTimeMillis();
      System.out.println(t2 - t1); // 370
      }

      }

       

      How ever, pymongo seems to be standalone mongo optimized, if directly connect to the primay, pymongo code would be completed with 0.2 seconds, while java code still takes about 300 milli seonds. The performance with java is fair, co's {w:1} shoudl be similary to connecting to a standalone mongod.

       

      The exact Python version used, with patch level:

      $ python -c "import sys; print(sys.version)"

      3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]

      The exact version of PyMongo used, with patch level:

      $ python -c "import pymongo; print(pymongo.version); print(pymongo.has_c())"

      4.4.1

      The operating system and version (e.g. Windows 7, OSX 10.8, ...)

      pymongo on window 11

      replicaset hosted on 2 windows 10 machines.

      Web framework or asynchronous network library used, if any, with version (e.g. Django 1.7, mod_wsgi 4.3.0, gevent 1.0.1, Tornado 4.0.2, ...)

      Not using yet, just a simple test.

            Assignee:
            noah.stapp@mongodb.com Noah Stapp
            Reporter:
            gbigloser@gmail.com Bigloser Felon
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: