[SERVER-16664] Segmentation fault on insert operation with Tiger engine / zlib compressor Created: 24/Dec/14  Updated: 23/Jan/15  Resolved: 21/Jan/15

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 2.8.0-rc4
Fix Version/s: 3.0.0-rc6

Type: Bug Priority: Major - P3
Reporter: Konstantin Koshelyaev Assignee: Michael Cahill (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS release 6.4 (Final), running on on VMWare.


Attachments: File lol-frequent-connect.pl     File lol.pl     Text File mongo-crash.log     File mongodb.log-crash    
Backwards Compatibility: Fully Compatible
Operating System: Linux
Steps To Reproduce:

1. Start Mongo with extra options;
2. Insert documents into collection (~200 100kb XMLs through perl driver).

Participants:

 Description   

Segmentation fault occures on insert operation with Tiger engine.

Server startup options:

/usr/bin/mongod --quiet -f /etc/mongodb.conf --storageEngine wiredtiger --wiredTigerJournalCompressor snappy --wiredTigerCollectionBlockCompressor snappy run

Used version:

mongodb-linux-x86_64-6105f06402fe1e7578d41f3e4e583a1476ef2455-2014-12-23

Happened twice - with snappy and zlib compressors.
Couldn't reproduce with higher log levels.



 Comments   
Comment by Ramon Fernandez Marina [ 12/Jan/15 ]

Thanks tirramissu, we no longer need the XML data. We're able to reproduce with a XML file set we already have access to and we're investigating.

Comment by Konstantin Koshelyaev [ 12/Jan/15 ]

You still need an XML?

Comment by Ramon Fernandez Marina [ 12/Jan/15 ]

Nevermind – the segfault does trigger in 2.8.0-rc4 as well, it jut took a tad longer.

Comment by Ramon Fernandez Marina [ 12/Jan/15 ]

All,

I'm able to easily reproduce the problem using 2.8.0-rc3, but not with 2.8.0-rc4. tirramissu, can you please confirm whether you see the segfault on your end when using 2.8.0-rc4?

Comment by Konstantin Koshelyaev [ 12/Jan/15 ]

Sure. How can I attach it privately? I've got only 'Users' and 'All users' group available in attachment dialog.

Comment by Ramon Fernandez Marina [ 12/Jan/15 ]

tirramissu, nevermind, I can see all the options used in the log files. We still need some sample documents, as so far we haven't been able to reproduce on our end. Alternatively, would you consider sharing your XML data? If so, please let me know and I'll be happy to provide instructions to upload your data privately and securely (it would only be shared with MongoDB staff).

Comment by Ramon Fernandez Marina [ 12/Jan/15 ]

tirramissu, in addition to a sample document, can you post the contents of your /etc/mongod.conf file?

Comment by David Golden [ 12/Jan/15 ]

tirramissu, do you have a sample document you could sanitize and upload? Ideally it would be one that crashed. I'd like to try to replicate with something as close to your actual data set as possible.
Thanks,
David

Comment by Konstantin Koshelyaev [ 11/Jan/15 ]

Using MariaDB or MySQL isn't necessary, you can take any large XML file and repeatably insert it in the collection.

If you need I can do any checks on my data set.

Comment by Konstantin Koshelyaev [ 11/Jan/15 ]

'lol-frequent-connect.pl':
Same as lol.pl.
It connects on each new insert, crashed more frequent.
It's a bad idea to do such things, but stable server shouldn't crash.

Comment by Konstantin Koshelyaev [ 11/Jan/15 ]

'lol.pl':
MariaDB to Mongo.
Take compressed XML from MariaDB, uncompress, insert into Mongo collection.

Comment by Konstantin Koshelyaev [ 11/Jan/15 ]

Hello, Dan.
Sorry for the delay, I was on holidays.

Comment by Daniel Pasette (Inactive) [ 11/Jan/15 ]

Hi Konstantin, any update here? I'd love to get my hands on a reproduction for this issue.

Comment by Daniel Pasette (Inactive) [ 31/Dec/14 ]

tirramissu, would you be able to attach your perl script which reproduces the issue to this ticket?

Thanks
Dan

Comment by Asya Kamsky [ 26/Dec/14 ]

tirramissu change "wiredtiger" to "wiredTiger" - that should remove the message about non-active storage engine.

Can you then let us know if the crash still happens and attach new logs please?

Comment by Daniel Pasette (Inactive) [ 26/Dec/14 ]

Can you attach the perl script and how you invoke it to this ticket?

Regarding the log message, mongod accepts both "wiredtiger" and "wiredTiger" as the storageEngine parameter. "wiredTiger" is the documented name though.

I'm not sure why you are getting the "Detected configuration for non-active storage..." message when you are starting up with the options you show on the command line. That warning only shows up if there are some mmapv1 storage engine options when you've selected wiredTiger.

Comment by Konstantin Koshelyaev [ 25/Dec/14 ]

Crash log for dry start.

Comment by Konstantin Koshelyaev [ 25/Dec/14 ]

Dry start for the same version. Clean DB directory.
CL:

/usr/bin/mongod --storageEngine wiredtiger --wiredTigerJournalCompressor zlib --wiredTigerCollectionBlockCompressor zlib --dbpath /var/lib/mongodb --logpath /var/log/mongodb/mongod.log run

Please see attached log file: 'mongo-crash.log'.

Odd notice - storageEngine is set to 'wiredtiger' by CL. Looks like Mongo sees it as 'wiredTiger'.

Comment by Konstantin Koshelyaev [ 24/Dec/14 ]

Data directory was clean, I deleted all files in /var/lib/mongodb.
I used mongo init.d script from old centos package, I'll check that can file.

Comment by J Rassi [ 24/Dec/14 ]

I see the following message in the attached log:

2014-12-24T18:18:58.732+0300 W STORAGE  [initandlisten] Detected configuration for non-active storage engine wiredTiger when current storage engine is wiredtiger

I infer from the presence of this message that you started the 6105f064 nightly version of the server on a data directory created with an earlier version of the server. Note that the WiredTiger data file format is not stable between different versions of the server during the release candidate process; I suspect that you may be affected by a file format change.

Are you able to reproduce this issue with the 6105f064 nightly on a clean data directory?

Generated at Thu Feb 08 03:41:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.