[SERVER-8431] Large update to text index can cause group commit to exceed max size Created: 31/Jan/13  Updated: 06/Jul/21  Resolved: 05/Mar/13

Status: Closed
Project: Core Server
Component/s: Storage, Text Search
Affects Version/s: 2.3.2
Fix Version/s: 2.4.0-rc2

Type: Bug Priority: Major - P3
Reporter: J Rassi Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

OSX 10.8, SSD storage


Attachments: Text File mongod.log    
Issue Links:
Related
related to SERVER-22078 Remove term list limits for text inde... Closed
Operating System: ALL
Participants:
Case:

 Description   

Inserting a document with 500,000 unique words can generate a group commit of >512MB. Trips assertion failure in AlignedBuilder::growReallocate, in util/alignedbuffer.cpp.

 91             verify( a <= 512*1024*1024 );

This leads to a server shutdown. Reproduce with (after creating a {{

{content: "text"}

}} index):

>>> import pymongo
>>> testdb = pymongo.MongoClient()['test']
>>> num_words_in_doc = 500 * 1000
>>> large_doc = {'content':" ".join(map(str, range(1,num_words_in_doc+1)))}
>>> num_docs = 50
>>> for i in xrange(num_docs):
...   large_doc['_id'] = i
...   testdb.foo.insert(large_doc)
... 

In my environment above, assertion trips on insert #32. Logfile attached.



 Comments   
Comment by auto [ 05/Mar/13 ]

Author:

{u'date': u'2013-03-05T05:51:07Z', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: SERVER-8431 - don't index docs where text index is too large for journal
Branch: master
https://github.com/mongodb/mongo/commit/9d9ea7b87d170711ddc17ac9abc2ead80d9cda60

Comment by auto [ 04/Mar/13 ]

Author:

{u'date': u'2013-03-04T19:19:24Z', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: SERVER-8431: make large index updates much less impactful
Branch: master
https://github.com/mongodb/mongo/commit/00c93cd20c7aed10c5f26d42272182d79b4a8972

Generated at Thu Feb 08 03:17:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.