[JAVA-777] Inserting LazyDBObject is slow Created: 28/Feb/13  Updated: 28/Feb/13  Resolved: 28/Feb/13

Status: Closed
Project: Java Driver
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Steve Briskin (Inactive) Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Inserting a LazyDBObject with a large array (20000 ints) is extremely slow...on the order of 500x slower.

Profiling shows the vast majority of the time is spent in the org.bson.io.BSONByteBuffer.getCString() call.

    @Test
    public void testLazyDocWithBigList() {
        DBCollection c = _db.getCollection("testLazyDotKeysPass");
        c.drop();
 
        DBObject obj = new BasicDBObject();
        ArrayList<Integer> arr = new ArrayList<Integer>();
        for(int i = 0; i < 20000; i++){
            arr.add(i);
        }
        
        obj.put("largeArray", arr);
        obj.put("randomstr", "asdfasdfa");
        
        long start = System.currentTimeMillis();
        c.insert(obj);
        System.out.println("Save obj: " + (System.currentTimeMillis() - start));
        
        //convert to a lazydbobject
        DefaultDBEncoder encoder = new DefaultDBEncoder();
        byte[] encodedBytes = encoder.encode(obj);
 
        LazyDBDecoder lazyDecoder = new LazyDBDecoder();
        DBObject lazyObj = lazyDecoder.decode(encodedBytes, c);
 
        start = System.currentTimeMillis();
        c.insert(lazyObj);
        System.out.println("Save lazy obj: " + (System.currentTimeMillis() - start));
        
        //convert lazy to a regular dbobject and save
        start = System.currentTimeMillis();
        LazyDBEncoder lazyEncoder = new LazyDBEncoder();
        BasicOutputBuffer outputBuffer = new BasicOutputBuffer();
        lazyEncoder.writeObject(outputBuffer, lazyObj);
        
        DefaultDBDecoder defDecode = new DefaultDBDecoder();
        DBObject normalObject = defDecode.decode(outputBuffer.toByteArray(), c);
        
        c.insert(normalObject);
        System.out.println("Lazy to Basic and save " + (System.currentTimeMillis() - start));
    }

Output

Save obj: 72
Save lazy obj: 35194
Lazy to Basic and save 131



 Comments   
Comment by Jeffrey Yemin [ 28/Feb/13 ]

Note line 27.

Comment by Jeffrey Yemin [ 28/Feb/13 ]

You have to use the LazyDBEncoder when you insert, or it's going to be awful:

        DB db = new MongoClient().getDB("test");
        DBCollection c = db.getCollection("testLazyDotKeysPass");
        c.drop();
 
        DBObject obj = new BasicDBObject();
        ArrayList<Integer> arr = new ArrayList<Integer>();
        for (int i = 0; i < 20000; i++) {
            arr.add(i);
        }
 
        obj.put("largeArray", arr);
        obj.put("randomstr", "asdfasdfa");
 
        long start = System.currentTimeMillis();
        c.insert(Arrays.asList(obj), WriteConcern.UNACKNOWLEDGED);
        System.out.println("Save obj: " + (System.currentTimeMillis() - start));
        c.remove(new BasicDBObject());
 
        //convert to a lazydbobject
        DefaultDBEncoder encoder = new DefaultDBEncoder();
        byte[] encodedBytes = encoder.encode(obj);
 
        LazyDBDecoder lazyDecoder = new LazyDBDecoder();
        DBObject lazyObj = lazyDecoder.decode(encodedBytes, c);
 
        start = System.currentTimeMillis();
        c.insert(Arrays.asList(lazyObj), WriteConcern.UNACKNOWLEDGED, new LazyDBEncoder());
        System.out.println("Save lazy obj: " + (System.currentTimeMillis() - start));
        c.remove(new BasicDBObject());
 
        //convert lazy to a regular DBObject and save
        start = System.currentTimeMillis();
        LazyDBEncoder lazyEncoder = new LazyDBEncoder();
        BasicOutputBuffer outputBuffer = new BasicOutputBuffer();
        lazyEncoder.writeObject(outputBuffer, lazyObj);
 
        DefaultDBDecoder defDecode = new DefaultDBDecoder();
        DBObject normalObject = defDecode.decode(outputBuffer.toByteArray(), c);
 
        c.insert(Arrays.asList(normalObject), WriteConcern.UNACKNOWLEDGED);
        System.out.println("Lazy to Basic and save " + (System.currentTimeMillis() - start));

Results:

Save obj: 51
Save lazy obj: 1
Lazy to Basic and save 120

Generated at Thu Feb 08 08:53:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.