[SERVER-33900] Optimize for loops in bulk_shard_insert.js Created: 15/Mar/18  Updated: 22/Mar/18  Resolved: 22/Mar/18

Status: Closed
Project: Core Server
Component/s: Sharding, Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Robert Guo (Inactive) Assignee: Samyukta Lanka
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Operating System: ALL
Sprint: TIG 2018-03-26, TIG 2018-04-09
Participants:
Linked BF Score: 0

 Description   

The JS code in bulk_shard_insert.js is causing the test to time out after we disabled the JIT.

Some improvements can be made to speed up the test:
1. Use itcount() instead of toArray() to take advantage of the gc() calls.

2. Cache the bulk array instead of creating it 250 times.

If these changes are still not sufficient, we should consider reducing the number of inserted documents.



 Comments   
Comment by Samyukta Lanka [ 22/Mar/18 ]

Resolved by SERVER-33635: https://jira.mongodb.org/browse/SERVER-33635

Note: Removing the toArray() call ended up not making a significant difference. The main source of time came from the number of documents being iterated.

The test no longer times out:

[2018/03/22 19:26:33.736] [executor:js_test:job0] 2018-03-22T19:26:33.736+0000 Running bulk_shard_insert.js...
[2018/03/22 19:26:33.736] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.storageEngineCacheSizeGB = "1"; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.failIfUnterminatedProcesses = true; TestData.testName = "bulk_shard_insert"; TestData.serviceExecutor = ""; TestData.storageEngine = "wiredTiger"; TestData.minPort = 20010; TestData.transportLayer = ""; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/"; load('jstests/libs/override_methods/validate_collections_on_shutdown.js');; load('jstests/libs/override_methods/check_uuids_consistent_across_cluster.js'); --readMode=commands --nodb jstests/sharding/bulk_shard_insert.js
[2018/03/22 19:26:33.784] [executor:js_test:job0] 2018-03-22T19:26:33.784+0000 Writing output of bulk_shard_insert to https://logkeeper.mongodb.org/build/04013a1a9e2b529ccbd9462333cf217f/test/5ab403699041305b7716eea6.
[2018/03/22 19:53:01.987] [executor:js_test:job0] 2018-03-22T19:53:01.986+0000 bulk_shard_insert.js ran in 1588.25 seconds.

https://evergreen.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_62_64_bit_coverage_sharding_d80f08d108569de8fb39c62339bb52978a4c6133_18_03_22_17_33_25

Generated at Thu Feb 08 04:34:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.