[SERVER-16886] Segmentation fault when loading data into WiredTiger. Created: 15/Jan/15  Updated: 03/Dec/21  Resolved: 21/Jan/15

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 2.8.0-rc5
Fix Version/s: 3.0.0-rc6

Type: Bug Priority: Major - P3
Reporter: John Page Assignee: Michael Cahill (Inactive)
Resolution: Done Votes: 1
Labels: wiredtiger
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Amazon Linux.


Issue Links:
Duplicate
is duplicated by SERVER-16993 mongod crashes with Got signal: 11 (S... Closed
is duplicated by SERVER-16926 mongod sefgault with wiredtiger durin... Closed
is duplicated by SERVER-16929 null pointer dereference in wiredTige... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

I can give you access to the servers , I am of course based in Europe so w need to work out timing.

Participants:

 Description   

Loading data on a high spec server on EC2 + EBS
15 x 400GB EBS SSD Diks as Raid 0.

Using custom C based multi threaded loader to load 500,000 real records (~270GB).

Wired tiger segfaults, seemingly at different points but reproducibly.

Stack trace.

2015-01-15T22:40:21.108+0000 F -        [conn3] Invalid access at address: 0x20
2015-01-15T22:40:21.119+0000 F -        [conn3] Got signal: 11 (Segmentation fault).
 
 0xf25749 0xf24df2 0xf2511e 0x7faab058d5b0 0x12d11d3 0x13092ce 0x1309896 0x130756d 0x12c62cd 0x12df145 0x12ba500 0x12f4eb6 0xd47aed 0x8f073e 0x8f0f0c 0xa8f868 0xa8fce0 0xa92442 0xa94d48 0x7e5320 0xee37d1 0x7faab0585f18 0x7faaaf697b9d
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"B25749"},{"b":"400000","o":"B24DF2"},{"b":"400000","o":"B2511E"},{"b":"7FAAB057E000","o":"F5B0"},{"b":"400000","o":"ED11D3"},{"b":"400000","o":"F092CE"},{"b":"400000","o":"F09896"},{"b":"400000","o":"F0756D"},{"b":"400000","o":"EC62CD"},{"b":"400000","o":"EDF145"},{"b":"400000","o":"EBA500"},{"b":"400000","o":"EF4EB6"},{"b":"400000","o":"947AED"},{"b":"400000","o":"4F073E"},{"b":"400000","o":"4F0F0C"},{"b":"400000","o":"68F868"},{"b":"400000","o":"68FCE0"},{"b":"400000","o":"692442"},{"b":"400000","o":"694D48"},{"b":"400000","o":"3E5320"},{"b":"400000","o":"AE37D1"},{"b":"7FAAB057E000","o":"7F18"},{"b":"7FAAAF5B5000","o":"E2B9D"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc5", "gitVersion" : "74b351de21c84438b12a83b28e155f5e69e3c1eb", "uname" : { "sysname" : "Linux", "release" : "3.14.20-20.44.amzn1.x86_64", "version" : "#1 SMP Mon Oct 6 22:52:46 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000" }, { "b" : "7FFFB9AFE000", "elfType" : 3 }, { "b" : "7FAAB057E000", "path" : "/lib64/libpthread.so.0", "elfType" : 3 }, { "b" : "7FAAB0376000", "path" : "/lib64/librt.so.1", "elfType" : 3 }, { "b" : "7FAAB0172000", "path" : "/lib64/libdl.so.2", "elfType" : 3 }, { "b" : "7FAAAFE6E000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3 }, { "b" : "7FAAAFB70000", "path" : "/lib64/libm.so.6", "elfType" : 3 }, { "b" : "7FAAAF95A000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7FAAAF5B5000", "path" : "/lib64/libc.so.6", "elfType" : 3 }, { "b" : "7FAAB079A000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3 } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf25749]
 mongod(+0xB24DF2) [0xf24df2]
 mongod(+0xB2511E) [0xf2511e]
 libpthread.so.0(+0xF5B0) [0x7faab058d5b0]
 mongod(__wt_split_insert+0x813) [0x12d11d3]
 mongod(+0xF092CE) [0x13092ce]
 mongod(__wt_evict+0x96) [0x1309896]
 mongod(__wt_evict_page+0x2D) [0x130756d]
 mongod(__wt_page_in_func+0x64D) [0x12c62cd]
 mongod(__wt_row_search+0x9D5) [0x12df145]
 mongod(__wt_btcur_insert+0x300) [0x12ba500]
 mongod(+0xEF4EB6) [0x12f4eb6]
 mongod(_ZN5mongo21WiredTigerRecordStore12insertRecordEPNS_16OperationContextEPKcib+0xCD) [0xd47aed]
 mongod(_ZN5mongo10Collection15_insertDocumentEPNS_16OperationContextERKNS_7BSONObjEb+0x4E) [0x8f073e]
 mongod(_ZN5mongo10Collection14insertDocumentEPNS_16OperationContextERKNS_7BSONObjEb+0x8C) [0x8f0f0c]
 mongod(_ZN5mongo14checkAndInsertEPNS_16OperationContextERNS_6Client7ContextEPKcRNS_7BSONObjE+0x108) [0xa8f868]
 mongod(_ZN5mongo11insertMultiEPNS_16OperationContextERNS_6Client7ContextEbPKcRSt6vectorINS_7BSONObjESaIS8_EERNS_5CurOpE+0xB0) [0xa8fce0]
 mongod(_ZN5mongo14receivedInsertEPNS_16OperationContextERNS_7MessageERNS_5CurOpE+0x522) [0xa92442]
 mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortEb+0x1658) [0xa94d48]
 mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0xE0) [0x7e5320]
 mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x321) [0xee37d1]
 libpthread.so.0(+0x7F18) [0x7faab0585f18]
 libc.so.6(clone+0x6D) [0x7faaaf697b9d]
-----  END BACKTRACE  -----
 



 Comments   
Comment by John Page [ 18/Jan/15 ]

Autocorrect sucks, I mean MongoDB source tree of course.

Comment by John Page [ 18/Jan/15 ]

When does that hit the monsoon source tree?

Comment by Daniel Pasette (Inactive) [ 18/Jan/15 ]

This should be fixed with the next drop from WiredTiger:
https://github.com/wiredtiger/wiredtiger/commit/e0031209183c880fb1a1b99399013e7675a75e88

Comment by Mark Callaghan [ 17/Jan/15 ]

Another crash from the same test but using the WT LSM via: --storageEngine wiredTiger --wiredTigerIndexConfigString="type=lsm"

2015-01-16T18:38:17.804-0800 F -        [conn11] Invalid access at address: 0
2015-01-16T18:38:17.814-0800 F -        [conn11] Got signal: 11 (Segmentation fault).
 
 0xfa0459 0xf9fb02 0xf9fe2e 0x7f0ecb02a550 0x133cd53 0x13749de 0x1374fa6 0x1372c7d 0x133265d 0x134ac85 0x1326930 0x13605c6 0xdc42dd 0x961bee 0x9623bc 0x9fe6c4 0x9ffac2 0xa01264 0xa01965 0xa0393d 0xa23914 0xa24853 0xa2530b 0xbee2b5 0xb030aa 0x857830 0xf5e751 0x7f0ecb021fa8 0x7f0ecad445ad 0
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"BA0459"},{"b":"400000","o":"B9FB02"},{"b":"400000","o":"B9FE2E"},{"b":"7F0ECB01A000","o":"10550"},{"b":"400000","o":"F3CD53"},{"b":"400000","o":"F749DE"},{"b":"400000","o":"F74FA6"},{"b":"400000","o":"F72C7D"},{"b":"400000","o":"F3265D"},{"b":"400000","o":"F4AC85"},{"b":"400000","o":"F26930"},{"b":"400000","o":"F605C6"},{"b":"400000","o":"9C42DD"},{"b":"400000","o":"561BEE"},{"b":"400000","o":"5623BC"},{"b":"400000","o":"5FE6C4"},{"b":"400000","o":"5FFAC2"},{"b":"400000","o":"601264"},{"b":"400000","o":"601965"},{"b":"400000","o":"60393D"},{"b":"400000","o":"623914"},{"b":"400000","o":"624853"},{"b":"400000","o":"62530B"},{"b":"400000","o":"7EE2B5"},{"b":"400000","o":"7030AA"},{"b":"400000","o":"457830"},{"b":"400000","o":"B5E751"},{"b":"7F0ECB01A000","o":"7FA8"},{"b":"7F0ECAC41000","o":"1035AD"},{"b":"0","o":"0"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc5", "gitVersion" : "nogitversion", "uname" : { "sysname" : "Linux", "release" : "3.10.53-43_fbk14_02627_gb5c286c", "version" : "#1 SMP Mon Oct 20 15:06:57 PDT 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "EFA03B085CCE356E620544EE3F30EA3F04FF83BA" }, { "b" : "7FFF2A4CA000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "50103D521098AC1A8DA637FE9F84CCFE48B62B25" }, { "b" : "7F0ECC08C000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libz.so.1", "elfType" : 3, "buildId" : "34A0EEADD720B217827BB606C2130376FCC642B9" }, { "b" : "7F0ECBE79000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libbz2.so.1", "elfType" : 3, "buildId" : "BF497A8E3985BF4C86572B1529689DE5B09E836A" }, { "b" : "7F0ECBC70000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/librt.so.1", "elfType" : 3, "buildId" : "344F63D30C2E97FB03EAA270E6D2FD3C3B70CA91" }, { "b" : "7F0ECBA6C000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libdl.so.2", "elfType" : 3, "buildId" : "0B5B808A05DA9CA8B4D0BE4CC3634163BB9B2CA5" }, { "b" : "7F0ECB753000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libstdc++.so.6", "elfType" : 3, "buildId" : "0E2E313B4183BB49B69B226B5856AF686A139922" }, { "b" : "7F0ECB44F000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libm.so.6", "elfType" : 3, "buildId" : "1FD2602F81D42C1B191D6ED62A9A67F24F361137" }, { "b" : "7F0ECB238000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libgcc_s.so.1", "elfType" : 3, "buildId" : "116105FC268AE8300E69DA4D80BC807336854680" }, { "b" : "7F0ECB01A000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libpthread.so.0", "elfType" : 3, "buildId" : "A1DB5EF502F4CA68FC0513EA2C3A9F4A57C1FDFB" }, { "b" : "7F0ECAC41000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libc.so.6", "elfType" : 3, "buildId" : "90ED6B4A89B0CD0DCE4D2D9DEA3E12BCE6EB115B" }, { "b" : "7F0ECC2A7000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/ld-2.17.so", "elfType" : 3, "buildId" : "C078595551DB92FD545F998822BE749AB997A0DC" } ] }}
 mongod.rc5.jem(_ZN5mongo15printStackTraceERSo+0x29) [0xfa0459]
 mongod.rc5.jem(+0xB9FB02) [0xf9fb02]
 mongod.rc5.jem(+0xB9FE2E) [0xf9fe2e]
 libpthread.so.0(+0x10550) [0x7f0ecb02a550]
 mongod.rc5.jem(__wt_split_insert+0x813) [0x133cd53]
 mongod.rc5.jem(+0xF749DE) [0x13749de]
 mongod.rc5.jem(__wt_evict+0x96) [0x1374fa6]
 mongod.rc5.jem(__wt_evict_page+0x2D) [0x1372c7d]
 mongod.rc5.jem(__wt_page_in_func+0x64D) [0x133265d]
 mongod.rc5.jem(__wt_row_search+0x9D5) [0x134ac85]
 mongod.rc5.jem(__wt_btcur_insert+0x300) [0x1326930]
 mongod.rc5.jem(+0xF605C6) [0x13605c6]
 mongod.rc5.jem(_ZN5mongo21WiredTigerRecordStore12insertRecordEPNS_16OperationContextEPKcib+0xCD) [0xdc42dd]
 mongod.rc5.jem(_ZN5mongo10Collection15_insertDocumentEPNS_16OperationContextERKNS_7BSONObjEb+0x4E) [0x961bee]
 mongod.rc5.jem(_ZN5mongo10Collection14insertDocumentEPNS_16OperationContextERKNS_7BSONObjEb+0x8C) [0x9623bc]
 mongod.rc5.jem(_ZN5mongo18WriteBatchExecutor13execOneInsertEPNS0_16ExecInsertsStateEPPNS_16WriteErrorDetailE+0x214) [0x9fe6c4]
 mongod.rc5.jem(_ZN5mongo18WriteBatchExecutor11execInsertsERKNS_21BatchedCommandRequestEPSt6vectorIPNS_16WriteErrorDetailESaIS6_EE+0x262) [0x9ffac2]
 mongod.rc5.jem(_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE+0x34) [0xa01264]
 mongod.rc5.jem(_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE+0x395) [0xa01965]
 mongod.rc5.jem(_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x15D) [0xa0393d]
 mongod.rc5.jem(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x34) [0xa23914]
 mongod.rc5.jem(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xC13) [0xa24853]
 mongod.rc5.jem(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x28B) [0xa2530b]
 mongod.rc5.jem(_ZN5mongo8runQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERKNS_15NamespaceStringERNS_5CurOpES3_b+0x755) [0xbee2b5]
 mongod.rc5.jem(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortEb+0xB0A) [0xb030aa]
 mongod.rc5.jem(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0xE0) [0x857830]
 mongod.rc5.jem(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x321) [0xf5e751]
 libpthread.so.0(+0x7FA8) [0x7f0ecb021fa8]
 libc.so.6(clone+0x6D) [0x7f0ecad445ad]
 ??? [0]
-----  END BACKTRACE  -----

Comment by John Page [ 16/Jan/15 ]

Feel free to tell me when it has

Comment by Mark Callaghan [ 16/Jan/15 ]

I get this from rc5 while running https://jira.mongodb.org/browse/SERVER-16886. I did not get this in rc4.

2015-01-16T09:45:56.486-0800 F -        [conn3] Invalid access at address: 0x20
2015-01-16T09:45:56.495-0800 F -        [conn3] Got signal: 11 (Segmentation fault).
 
 0xfa0459 0xf9fb02 0xf9fe2e 0x7fd6b0e89550 0x133cd53 0x13749de 0x1374fa6 0x1372c7d 0x133265d 0x134ac85 0x1326930 0x13605c6 0xdc42dd 0x961bee 0x9623bc 0x9fe6c4 0x9ffac2 0xa01264 0xa01965 0xa0393d 0xa23914 0xa24853 0xa2530b 0xbee2b5 0xb030aa 0x857830 0xf5e751 0x7fd6b0e80fa8 0x7fd6b0ba35ad 0
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"BA0459"},{"b":"400000","o":"B9FB02"},{"b":"400000","o":"B9FE2E"},{"b":"7FD6B0E79000","o":"10550"},{"b":"400000","o":"F3CD53"},{"b":"400000","o":"F749DE"},{"b":"400000","o":"F74FA6"},{"b":"400000","o":"F72C7D"},{"b":"400000","o":"F3265D"},{"b":"400000","o":"F4AC85"},{"b":"400000","o":"F26930"},{"b":"400000","o":"F605C6"},{"b":"400000","o":"9C42DD"},{"b":"400000","o":"561BEE"},{"b":"400000","o":"5623BC"},{"b":"400000","o":"5FE6C4"},{"b":"400000","o":"5FFAC2"},{"b":"400000","o":"601264"},{"b":"400000","o":"601965"},{"b":"400000","o":"60393D"},{"b":"400000","o":"623914"},{"b":"400000","o":"624853"},{"b":"400000","o":"62530B"},{"b":"400000","o":"7EE2B5"},{"b":"400000","o":"7030AA"},{"b":"400000","o":"457830"},{"b":"400000","o":"B5E751"},{"b":"7FD6B0E79000","o":"7FA8"},{"b":"7FD6B0AA0000","o":"1035AD"},{"b":"0","o":"0"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc5", "gitVersion" : "nogitversion", "uname" : { "sysname" : "Linux", "release" : "3.10.53-43_fbk14_02627_gb5c286c", "version" : "#1 SMP Mon Oct 20 15:06:57 PDT 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "EFA03B085CCE356E620544EE3F30EA3F04FF83BA" }, { "b" : "7FFFDA6C9000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "50103D521098AC1A8DA637FE9F84CCFE48B62B25" }, { "b" : "7FD6B1EEB000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libz.so.1", "elfType" : 3, "buildId" : "34A0EEADD720B217827BB606C2130376FCC642B9" }, { "b" : "7FD6B1CD8000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libbz2.so.1", "elfType" : 3, "buildId" : "BF497A8E3985BF4C86572B1529689DE5B09E836A" }, { "b" : "7FD6B1ACF000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/librt.so.1", "elfType" : 3, "buildId" : "344F63D30C2E97FB03EAA270E6D2FD3C3B70CA91" }, { "b" : "7FD6B18CB000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libdl.so.2", "elfType" : 3, "buildId" : "0B5B808A05DA9CA8B4D0BE4CC3634163BB9B2CA5" }, { "b" : "7FD6B15B2000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libstdc++.so.6", "elfType" : 3, "buildId" : "0E2E313B4183BB49B69B226B5856AF686A139922" }, { "b" : "7FD6B12AE000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libm.so.6", "elfType" : 3, "buildId" : "1FD2602F81D42C1B191D6ED62A9A67F24F361137" }, { "b" : "7FD6B1097000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libgcc_s.so.1", "elfType" : 3, "buildId" : "116105FC268AE8300E69DA4D80BC807336854680" }, { "b" : "7FD6B0E79000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libpthread.so.0", "elfType" : 3, "buildId" : "A1DB5EF502F4CA68FC0513EA2C3A9F4A57C1FDFB" }, { "b" : "7FD6B0AA0000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libc.so.6", "elfType" : 3, "buildId" : "90ED6B4A89B0CD0DCE4D2D9DEA3E12BCE6EB115B" }, { "b" : "7FD6B2106000", "path" : "/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/ld-2.17.so", "elfType" : 3, "buildId" : "C078595551DB92FD545F998822BE749AB997A0DC" } ] }}
 mongod.rc5.jem(_ZN5mongo15printStackTraceERSo+0x29) [0xfa0459]
 mongod.rc5.jem(+0xB9FB02) [0xf9fb02]
 mongod.rc5.jem(+0xB9FE2E) [0xf9fe2e]
 libpthread.so.0(+0x10550) [0x7fd6b0e89550]
 mongod.rc5.jem(__wt_split_insert+0x813) [0x133cd53]
 mongod.rc5.jem(+0xF749DE) [0x13749de]
 mongod.rc5.jem(__wt_evict+0x96) [0x1374fa6]
 mongod.rc5.jem(__wt_evict_page+0x2D) [0x1372c7d]
 mongod.rc5.jem(__wt_page_in_func+0x64D) [0x133265d]
 mongod.rc5.jem(__wt_row_search+0x9D5) [0x134ac85]
 mongod.rc5.jem(__wt_btcur_insert+0x300) [0x1326930]
 mongod.rc5.jem(+0xF605C6) [0x13605c6]
 mongod.rc5.jem(_ZN5mongo21WiredTigerRecordStore12insertRecordEPNS_16OperationContextEPKcib+0xCD) [0xdc42dd]
 mongod.rc5.jem(_ZN5mongo10Collection15_insertDocumentEPNS_16OperationContextERKNS_7BSONObjEb+0x4E) [0x961bee]
 mongod.rc5.jem(_ZN5mongo10Collection14insertDocumentEPNS_16OperationContextERKNS_7BSONObjEb+0x8C) [0x9623bc]
 mongod.rc5.jem(_ZN5mongo18WriteBatchExecutor13execOneInsertEPNS0_16ExecInsertsStateEPPNS_16WriteErrorDetailE+0x214) [0x9fe6c4]
 mongod.rc5.jem(_ZN5mongo18WriteBatchExecutor11execInsertsERKNS_21BatchedCommandRequestEPSt6vectorIPNS_16WriteErrorDetailESaIS6_EE+0x262) [0x9ffac2]
 mongod.rc5.jem(_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE+0x34) [0xa01264]
 mongod.rc5.jem(_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE+0x395) [0xa01965]
 mongod.rc5.jem(_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x15D) [0xa0393d]
 mongod.rc5.jem(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x34) [0xa23914]
 mongod.rc5.jem(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xC13) [0xa24853]
 mongod.rc5.jem(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x28B) [0xa2530b]
 mongod.rc5.jem(_ZN5mongo8runQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERKNS_15NamespaceStringERNS_5CurOpES3_b+0x755) [0xbee2b5]
 mongod.rc5.jem(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortEb+0xB0A) [0xb030aa]
 mongod.rc5.jem(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0xE0) [0x857830]
 mongod.rc5.jem(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x321) [0xf5e751]
 libpthread.so.0(+0x7FA8) [0x7fd6b0e80fa8]
 libc.so.6(clone+0x6D) [0x7fd6b0ba35ad]
 ??? [0]

I changed the run.simple.bash script used to start iibench with:

export MAX_ROWS=400000000
 
export RUN_MINUTES=$1 --> use a large value for this
export RUN_SECONDS=$[RUN_MINUTES*60]
export NUM_DOCUMENTS_PER_INSERT=10
export NUM_LOADER_THREADS=10
export LENGTH_CHAR_FIELDS=2000
export PERCENT_COMPRESSIBLE=50

mongo.conf

processManagement:
  fork: true
systemLog:
  destination: file
  path: /data/mysql/mongo.280rc4/log
  logAppend: true
storage:
  syncPeriodSecs: 60
  dbPath: /data/mysql/mongo.280rc4/data
  journal:
    enabled: true
  mmapv1:
    journal:
      commitIntervalMs: 100
storage.wiredTiger.collectionConfig.blockCompressor: snappy
storage.wiredTiger.engineConfig.journalCompressor: snappy
operationProfiling.slowOpThresholdMs: 2000

Command line:
numactl --interleave=all bin/mongod --config /data/mysql/mongo.280rc4/mongo.conf --storageEngine wiredTiger

Comment by John Page [ 16/Jan/15 ]

I'm not a git afficianado - how do I check out the right branch ?

Comment by John Page [ 16/Jan/15 ]

Just pulled the latest master from git, built it and see the same result.

Comment by John Page [ 16/Jan/15 ]

Do I need to build it myself to test that or is it in the nightly build

Generated at Thu Feb 08 03:42:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.