[SERVER-4534] Index creation opens too many files for large array Created: 20/Dec/11  Updated: 11/Jul/16  Resolved: 10/Jul/13

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: None
Fix Version/s: 2.5.1

Type: Bug Priority: Critical - P2
Reporter: Mathias Stearn Assignee: Mathias Stearn
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-6157 Refactor BSONObjExtSort Closed
is related to DOCS-1206 Add note to text index build document... Closed
Operating System: ALL
Participants:

 Description   

Background indexing is ok, forground inding crashes.

Tue Dec 20 14:48:40 [conn2] build index test.huge_multikey_index

{ a: 1.0 }

Tue Dec 20 14:48:45 [conn2] couldn't open /data/db/sconsTests/_tmp/esort.1324410520.1552787748//file.1010 errno:24 Too many open files
Tue Dec 20 14:48:45 [conn2] Assertion: 10308:mmap failed
0x5b01f0 0x5b2a0e 0x8731db 0x873418 0x8580cd 0x84d3fd 0x84e1b0 0x84f9ae 0x85062e 0x82f6a0 0x831c8a 0x83600b 0x836842 0x96e0c8 0x6371f9 0x7f5eaf522df0 0x7f5eae2292ed
/home/mstearn/10gen/mongo/mongod() [0x5b01f0]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo11msgassertedEiPKc+0xbe) [0x5b2a0e]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo21BSONObjExternalSorter12FileIteratorC1ESs+0xab) [0x8731db]
/home/mstearn/10gen/mongo/mongod(ZN5mongo21BSONObjExternalSorter8IteratorC1EPS0+0x1f8) [0x873418]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo24buildBottomUpPhases2And3INS_12BtreeData_V1EEEvbRNS_12IndexDetailsERNS_21BSONObjExternalSorterEbRSt4listINS_7DiskLocESaIS7_EEPNS_5CurOpEPNS_12SortPhaseOneERNS_19ProgressMeterHolderERNS_5TimerE+0x7d) [0x8580cd]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo14fastBuildIndexEPKcPNS_16NamespaceDetailsERNS_12IndexDetailsEi+0x60d) [0x84d3fd]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo16insert_makeIndexEPNS_16NamespaceDetailsERKSsRKNS_7DiskLocE+0x360) [0x84e1b0]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibbPb+0x30e) [0x84f9ae]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo11DataFileMgr16insertWithObjModEPKcRNS_7BSONObjEb+0x3e) [0x85062e]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo14checkAndInsertEPKcRNS_7BSONObjE+0xb0) [0x82f6a0]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo14receivedInsertERNS_7MessageERNS_5CurOpE+0x25a) [0x831c8a]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo17_assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x171b) [0x83600b]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x32) [0x836842]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x88) [0x96e0c8]
/home/mstearn/10gen/mongo/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x289) [0x6371f9]
/lib/libpthread.so.0(+0x7df0) [0x7f5eaf522df0]
/lib/libc.so.6(clone+0x6d) [0x7f5eae2292ed]
Tue Dec 20 14:48:45 terminate() called, printing stack:
...

slowNightly test will be committed shortly

Fails with 1.8, 2.0, and master.



 Comments   
Comment by auto [ 10/Jul/13 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: Enable test for SERVER-4534

It was fixed when index building switched to using the new Sorter.
Branch: master
https://github.com/mongodb/mongo/commit/b6d259180642b0bb3296928ad4d6740e63ac30bf

Comment by Mathias Stearn [ 27/Jun/13 ]

Just need to reenable huge_multikey_index.js. Already passes after extsort refacter

Comment by Eliot Horowitz (Inactive) [ 21/Mar/12 ]

Can you set ulimit to unlimited?

Comment by Paul Wang [ 21/Mar/12 ]

Have the same problem, working on a collection size of 1 billion with 4 fields. Trying to create a single index on 3 fields, getting to 99% complete them the error message above hits. Some people suggested to increase the ulimit -n 10000, but still fails. Any luck on solving this problem yet, the index cannot be performed in the background since it will take too long.

Thanks in advance,
Paul

Comment by Mathias Stearn [ 23/Dec/11 ]

After some examination, this only effects collections with < 1,000,000 objects or an average of > 100 elements per array. This is because we will use the smaller of 1M or db.foo.count()+100 as the size of the InMemory array in ExternalSort. This has bad effects for small collections with huge arrays like this test.

Comment by auto [ 20/Dec/11 ]

Author:

{u'login': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: Disabled test for SERVER-4534
Branch: master
https://github.com/mongodb/mongo/commit/88e4e699dc4db5efd20f393ac587fbe8d66673ac

Generated at Thu Feb 08 03:06:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.