Loading...

Type: Bug
Resolution: Done
Priority: Major - P3
Fix Version/s: None
Affects Version/s: 3.0.8, 3.2.0
Component/s: Tools
Labels:
None

Operating System:
ALL
Steps To Reproduce:
Hide

All following tests are done with the primer dataset as mentioned in our documentation.

In 2.6.9, it works as expected.

2.6.9 mongoimport

mongoimport --db test --collection restaurants --file primer-dataset.json --drop connected to: 127.0.0.1 2016-01-04T11:23:04.064-0500 dropping: test.restaurants 2016-01-04T11:23:06.012-0500 check 9 25359 2016-01-04T11:23:06.013-0500 imported 25359 objects

2.6.9 mongoimport with upsert

mongoimport --db test --collection restaurants --file primer-dataset.json --drop --upsert connected to: 127.0.0.1 2016-01-04T11:23:13.559-0500 dropping: test.restaurants 2016-01-04T11:23:15.452-0500 check 9 25359 2016-01-04T11:23:15.455-0500 imported 25359 objects

However, in 3.x the upserted import takes a long time to run. Tested on multiple versions including 3.0.8 and 3.2.0.

3.0.8 mongoimport

mongoimport --db test --collection restaurants --file primer-dataset.json --drop 2016-01-04T11:19:22.904-0500 connected to: localhost 2016-01-04T11:19:22.904-0500 dropping: test.restaurants 2016-01-04T11:19:24.406-0500 imported 25359 documents

3.0.8 mongoimport with upset

mongoimport --db test --collection restaurants --file primer-dataset.json --drop --upsert 2016-01-04T11:19:29.025-0500 connected to: localhost 2016-01-04T11:19:29.025-0500 dropping: test.restaurants 2016-01-04T11:19:32.022-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:35.023-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:38.023-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:41.027-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:44.025-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:47.026-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:50.024-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:53.026-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:56.023-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:59.022-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:02.025-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:05.023-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:08.026-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:11.025-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:14.023-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:17.026-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:20.023-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:23.026-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:26.022-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:29.024-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:32.027-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:35.022-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:38.026-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:39.418-0500 imported 25359 documents
Show
All following tests are done with the primer dataset as mentioned in our documentation . In 2.6.9, it works as expected. 2.6.9 mongoimport mongoimport --db test --collection restaurants --file primer-dataset.json --drop connected to: 127.0.0.1 2016-01-04T11:23:04.064-0500 dropping: test.restaurants 2016-01-04T11:23:06.012-0500 check 9 25359 2016-01-04T11:23:06.013-0500 imported 25359 objects 2.6.9 mongoimport with upsert mongoimport --db test --collection restaurants --file primer-dataset.json --drop --upsert connected to: 127.0.0.1 2016-01-04T11:23:13.559-0500 dropping: test.restaurants 2016-01-04T11:23:15.452-0500 check 9 25359 2016-01-04T11:23:15.455-0500 imported 25359 objects However, in 3.x the upserted import takes a long time to run. Tested on multiple versions including 3.0.8 and 3.2.0. 3.0.8 mongoimport mongoimport --db test --collection restaurants --file primer-dataset.json --drop 2016-01-04T11:19:22.904-0500 connected to: localhost 2016-01-04T11:19:22.904-0500 dropping: test.restaurants 2016-01-04T11:19:24.406-0500 imported 25359 documents 3.0.8 mongoimport with upset mongoimport --db test --collection restaurants --file primer-dataset.json --drop --upsert 2016-01-04T11:19:29.025-0500 connected to: localhost 2016-01-04T11:19:29.025-0500 dropping: test.restaurants 2016-01-04T11:19:32.022-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:35.023-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:38.023-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:41.027-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:44.025-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:47.026-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:50.024-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:53.026-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:56.023-0500 [###########.............] test.restaurants 5.3 MB/11.3 MB (46.7%) 2016-01-04T11:19:59.022-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:02.025-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:05.023-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:08.026-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:11.025-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:14.023-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:17.026-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:20.023-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:23.026-0500 [####################....] test.restaurants 9.9 MB/11.3 MB (87.1%) 2016-01-04T11:20:26.022-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:29.024-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:32.027-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:35.022-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:38.026-0500 [########################] test.restaurants 11.3 MB/11.3 MB (100.0%) 2016-01-04T11:20:39.418-0500 imported 25359 documents

mongoimport with the --upsert flag is much slower than expected on 3.x. When running mongoimport with the --upsert flag on an empty collection, performance should be about equal to mongoimport without the flag. This was the case in 2.6.9 but is no longer true in 3.x.

Details

Description

Attachments

Activity

People

Dates