[SERVER-11553] mongoimport cause to crash mongod on sharder cluster Created: 04/Nov/13  Updated: 10/Dec/14  Resolved: 30/Dec/13

Status: Closed
Project: Core Server
Component/s: Internal Code, Stability, Tools
Affects Version/s: 2.4.7
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Roman Shekhovtsov Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: Windows, crash, mongoimport, replicaset, sharding
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows7 32bit
sharder cluster with 2x replica set
each replica set has 2 nodes
1 config server
1 mongos
All on the same PC (localhost)


Attachments: Text File mongod.log     Text File mongos.log     File test_coll_import.json    
Operating System: Windows
Steps To Reproduce:

I use commands below to create a cluster:
@cd c:\mongodb\
@start "rs0:27001" bin\mongod.exe --port 27001 --replSet rs0 --config mongod.cfg --dbpath db_rs0
@start "rs0:27002" bin\mongod.exe --port 27002 --replSet rs0 --config mongod.cfg --dbpath db_rs0s
@start "rs1:27003" bin\mongod.exe --port 27003 --replSet rs1 --config mongod.cfg --dbpath db_rs1
@start "rs1:27004" bin\mongod.exe --port 27004 --replSet rs1 --config mongod.cfg --dbpath db_rs1s
@start "cfg:27005" bin\mongod.exe --port 27005 --configsvr --config mongod.cfg --dbpath configdb
@start "shr:27006" bin\mongos.exe --port 27006 --configdb localhost:27005 --config mongos.cfg
mongo.cmd 27001

rs.initiate({ _id: 'rs0', version: 1, members: [{_id:0, host:'localhost:27001'}] })
rs.add('localhost:27002')
exit

mongo.cmd 27003

rs.initiate({ _id: 'rs1', version: 1, members: [{_id:0, host:'localhost:27003'}] })
rs.add('localhost:27004')
exit

mongo.cmd 27006

sh.addShard( "rs0/localhost:27001" )
sh.addShard( "rs1/localhost:27003" )
sh.enableSharding("test")

use test

<GENERATE TEST DATA
people = ["Marc", "Bill", "George", "Eliot", "Matt", "Trey", "Tracy", "Greg", "Steve", "Kristina", "Katie", "Jeff"];
for(var i=0; i<1000000; i++) {
name = people[Math.floor(Math.random()*people.length)];
user_id = i;
boolean = [true, false][Math.floor(Math.random()*2)];
added_at = new Date();
number = Math.floor(Math.random()*10001);

db.test_collection.save(

{ "name":name, "user_id":user_id, "boolean": boolean, "added_at":added_at, "number":number }

);
}
GENERATE TEST DATA>

db.test_collection.ensureIndex(

{ _id : "hashed" }

)
sh.shardCollection("test.test_collection",

{ "_id": "hashed" }

)

THEN:
bin\mongoexport.exe --port 27006 --collection test_collection --out test_coll_import.json
bin\mongoimport.exe --port 27006 --collection test_coll_import --file test_coll_import.json

Participants:

 Description   

I tried to export and then import with different name small collection (~8K records) using mongoexport and then mongoimport.

One of the cluster node crushed.

mongod log:
Mon Nov 04 22:44:39.551 [conn37] build index test.test_coll_import

{ _id: 1 }

Mon Nov 04 22:44:39.553 [conn37] build index done. scanned 0 total records. 0.001 secs
Mon Nov 04 22:44:39.555 [repl writer worker 1] build index test.test_coll_import

{ _id: 1 }

Mon Nov 04 22:44:39.557 [repl writer worker 1] build index done. scanned 0 total records. 0.001 secs
Mon Nov 04 22:44:39.558 [conn4] getMore: cursorid not found local.oplog.rs 51171225181354
Mon Nov 04 22:44:39.558 [rsSyncNotifier] repl: old cursor isDead, will initiate a new one
Mon Nov 04 22:44:40.731 [conn25] command admin.$cmd command:

{ writebacklisten: ObjectId('5277e3da81f5fbe6b00d97ef') }

ntoreturn:1 keyUpdates:0 reslen:44 300000ms
Mon Nov 04 22:44:40.733 [conn21] command admin.$cmd command:

{ writebacklisten: ObjectId('5277e3da81f5fbe6b00d97ef') }

ntoreturn:1 keyUpdates:0 reslen:44 300000ms
Mon Nov 04 22:44:40.737 [conn26] command admin.$cmd command:

{ writebacklisten: ObjectId('5277e3da81f5fbe6b00d97ef') }

ntoreturn:1 keyUpdates:0 reslen:44 300000ms
Mon Nov 04 22:44:40.739 [conn21] command admin.$cmd command:

{ writebacklisten: ObjectId('5277e3da81f5fbe6b00d97ef') }

ntoreturn:1 keyUpdates:0 reslen:44 300000ms
Mon Nov 04 22:44:40.800 [journal] MapViewOfFileEx for C:/mongodb/db_rs0/test.ns failed with error errno:487 Attempt to access invalid address. (file size is 16777216) in MemoryMappedFile::remapPrivateView
Mon Nov 04 22:44:40.800 [journal] Fatal Assertion 16148
Mon Nov 04 22:44:42.165 [journal] mongod.exe ...\src\mongo\util\assert_util.cpp(129) mongo::fassertFailed+0x75
Mon Nov 04 22:44:42.165 [journal] mongod.exe ...\src\mongo\util\mmap_win.cpp(326) mongo::MemoryMappedFile::remapPrivateView+0x291
Mon Nov 04 22:44:42.165 [journal] mongod.exe ...\src\mongo\db\dur.cpp(523) mongo::dur::_REMAPPRIVATEVIEW+0x261
Mon Nov 04 22:44:42.165 [journal] mongod.exe ...\src\mongo\db\dur.cpp(538) mongo::dur::REMAPPRIVATEVIEW+0x11
Mon Nov 04 22:44:42.165 [journal] mongod.exe ...\src\mongo\db\dur.cpp(700) mongo::dur::_groupCommit+0x227
Mon Nov 04 22:44:42.165 [journal] mongod.exe ...\src\mongo\db\dur.cpp(711) mongo::dur::groupCommit+0x3e
Mon Nov 04 22:44:42.165 [journal] mongod.exe ...\src\mongo\db\dur.cpp(751) mongo::dur::durThreadGroupCommit+0xaf
Mon Nov 04 22:44:42.165 [journal] mongod.exe ...\src\mongo\db\dur.cpp(16707566) mongo::dur::durThread+0x227
Mon Nov 04 22:44:42.165 [journal] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(16707566) boost::`anonymous namespace'::thread_start_function+0x47
Mon Nov 04 22:44:42.165 [journal] mongod.exe f:\dd\vctools\crt_bld\self_x86\crt\src\threadex.c(314) _callthreadstartex+0x1b
Mon Nov 04 22:44:42.165 [journal] mongod.exe f:\dd\vctools\crt_bld\self_x86\crt\src\threadex.c(292) _threadstartex+0x64
Mon Nov 04 22:44:42.165 [journal] kernel32.dll BaseThreadInitThunk+0x12
Mon Nov 04 22:44:42.165 [journal] ntdll.dll RtlInitializeExceptionChain+0xef



 Comments   
Comment by Daniel Pasette (Inactive) [ 30/Dec/13 ]

As noted on the MongoDB downloads page

32-bit builds are limited to around 2GB of data. In general you should use the 64 bit builds. The 32 bit binaries are ok for replica set arbiters and mongos' but not for production mongod's. See here for more information.

Generated at Thu Feb 08 03:26:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.