[SERVER-17324] NULL pointer exception during insert only stress under low memory condition Created: 18/Feb/15  Updated: 12/Oct/15  Resolved: 12/Oct/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.0.0-rc8
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Eitan Klein Assignee: Michael Cahill (Inactive)
Resolution: Cannot Reproduce Votes: 0
Labels: 28qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File rs2.txt.2015-02-16T03-13-36    
Issue Links:
Related
is related to SERVER-17542 Out of Memory crash with wiredTiger Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

v3.0.0-rc9-pre
GitHub # - c7d4eac9eb895886086eb6f0b763147ec7f3b90

Environment:

  • Single machine replica set with 2 nodes.
  • Windows 8 w/ limited amount of available memory (VM w/o page file, limited to 2GB RAM)
  • Each instance uses default 1GB RAM

Workload:

  • Used hammer.mongo to do insert only workload
  • The machine complained about out of memory before crashing
  • When cache fills up, eventually hit a NPE. Location varies according to run.

Stack trace:

 
0:015> k
Child-SP          RetAddr           Call Site
00000051`8cb1e7f0 00007ff6`bc5a86e0 mongod!__rec_txn_read+0x70 [c:\data\mci\shell\src\src\third_party\wiredtiger\src\reconcile\rec_write.c @ 855]
00000051`8cb1e890 00007ff6`bc5a852c mongod!__rec_row_leaf_insert+0x60 [c:\data\mci\shell\src\src\third_party\wiredtiger\src\reconcile\rec_write.c @ 4619]
00000051`8cb1e8f0 00007ff6`bc5ad15c mongod!__rec_row_leaf+0x70c [c:\data\mci\shell\src\src\third_party\wiredtiger\src\reconcile\rec_write.c @ 4589]
00000051`8cb1ea10 00007ff6`bc58c9de mongod!__wt_reconcile+0x19c [c:\data\mci\shell\src\src\third_party\wiredtiger\src\reconcile\rec_write.c @ 414]
00000051`8cb1ea70 00007ff6`bc58cbfd mongod!__evict_review+0x21e [c:\data\mci\shell\src\src\third_party\wiredtiger\src\evict\evict_page.c @ 481]
00000051`8cb1ead0 00007ff6`bc58c298 mongod!__wt_evict+0xcd [c:\data\mci\shell\src\src\third_party\wiredtiger\src\evict\evict_page.c @ 59]
00000051`8cb1eb50 00007ff6`bc568a78 mongod!__wt_evict_page+0x38 [c:\data\mci\shell\src\src\third_party\wiredtiger\src\evict\evict_lru.c @ 630]
00000051`8cb1eb80 00007ff6`bc55acb5 mongod!__wt_page_release_evict+0x98 [c:\data\mci\shell\src\src\third_party\wiredtiger\src\include\btree.i @ 972]
00000051`8cb1ebc0 00007ff6`bc56a55a mongod!__wt_page_in_func+0x1d5 [c:\data\mci\shell\src\src\third_party\wiredtiger\src\btree\bt_page.c @ 144]
00000051`8cb1ec10 00007ff6`bc56ac75 mongod!__wt_page_swap_func+0x2a [c:\data\mci\shell\src\src\third_party\wiredtiger\src\include\btree.i @ 1063]
00000051`8cb1ec50 00007ff6`bc554044 mongod!__wt_row_search+0x405 [c:\data\mci\shell\src\src\third_party\wiredtiger\src\btree\row_srch.c @ 297]
00000051`8cb1ed10 00007ff6`bc57c3fc mongod!__wt_btcur_insert+0x214 [c:\data\mci\shell\src\src\third_party\wiredtiger\src\btree\bt_cursor.c @ 484]
00000051`8cb1ed70 00007ff6`bc1b9852 mongod!__curfile_insert+0x2ac [c:\data\mci\shell\src\src\third_party\wiredtiger\src\cursor\cur_file.c @ 245]
00000051`8cb1ee10 00007ff6`bbe50bdb mongod!mongo::WiredTigerRecordStore::insertRecord+0x3b2 [c:\data\mci\shell\src\src\mongo\db\storage\wiredtiger\wiredtiger_record_store.cpp @ 579]
00000051`8cb1ef60 00007ff6`bbe51a67 mongod!mongo::Collection::_insertDocument+0x6b [c:\data\mci\shell\src\src\mongo\db\catalog\collection.cpp @ 245]
00000051`8cb1efe0 00007ff6`bc0d2bea mongod!mongo::Collection::insertDocument+0x1b7 [c:\data\mci\shell\src\src\mongo\db\catalog\collection.cpp @ 201]
00000051`8cb1f0e0 00007ff6`bc11afe6 mongod!mongo::repl::writeOpsToOplog+0x3ca [c:\data\mci\shell\src\src\mongo\db\repl\oplog.cpp @ 445]
00000051`8cb1f380 00007ff6`bc11b654 mongod!mongo::repl::SyncTail::multiApply+0x2b6 [c:\data\mci\shell\src\src\mongo\db\repl\sync_tail.cpp @ 300]
00000051`8cb1f530 00007ff6`bc11579c mongod!mongo::repl::SyncTail::oplogApplication+0x3d4 [c:\data\mci\shell\src\src\mongo\db\repl\sync_tail.cpp @ 510]
00000051`8cb1f6a0 00007ff6`bc307d81 mongod!mongo::repl::runSyncThread+0x37c [c:\data\mci\shell\src\src\mongo\db\repl\rs_sync.cpp @ 135]
00000051`8cb1f900 00007ff6`bc60e207 mongod!boost::`anonymous namespace'::thread_start_function+0x21 [c:\data\mci\shell\src\src\third_party\boost\libs\thread\src\win32\thread.cpp @ 185]
00000051`8cb1f940 00007ff6`bc60e3ae mongod!_callthreadstartex+0x17 [f:\dd\vctools\crt\crtw32\startup\threadex.c @ 376]
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\system32\KERNEL32.DLL - 
00000051`8cb1f970 00007fff`e37615bd mongod!_threadstartex+0x102 [f:\dd\vctools\crt\crtw32\startup\threadex.c @ 354]
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\SYSTEM32\ntdll.dll - 
00000051`8cb1f9a0 00007fff`e5b843d1 KERNEL32!BaseThreadInitThunk+0xd
00000051`8cb1f9d0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d
 
0:015> .srcpath \\eitan6\f$\src\mongo\src

Variables:

> dump variables
        session = 0x00000051`8f992e10
              r = 0x00007ff6`bc5bce2e
            ins = 0x00000051`9912e030
            rip = 0x00000000`00000000
          vpack = 0x00000000`00000000
           updp = 0x00000051`8cb1e8f8
        notused = 0x00007ff6`bc5bce2e
       upd_ovfl = 0x00000051`8cb1e8f8
           ovfl = struct __wt_item
 
> dump pointer 0x00000051`8cb1e8f8
00000051`8cb1e8f8  00000000`00000000 00000051`8f992e10
00000051`8cb1e908  00000051`8cb1e9a9 00000051`8cb1ea10
00000051`8cb1e918 

Code at point of crash:

 
for (max_txn = WT_TXN_NONE, min_txn = UINT64_MAX, upd = upd_list;
        upd != NULL; upd = upd->next) {  <== upd->next is NULL
        if ((txnid = upd->txnid) == WT_TXN_ABORTED)
            continue;

Windows dumpfile:



 Comments   
Comment by Michael Cahill (Inactive) [ 12/Oct/15 ]

May have been fixed by WT-1992 or WT-1993.

Generated at Thu Feb 08 03:44:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.