[SERVER-7511] update code does not properly determine if a positional mod is indexed, fix Mod::isIndexed() Created: 30/Oct/12  Updated: 11/Jul/16  Resolved: 15/Jan/13

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: 2.2.2, 2.3.1
Fix Version/s: 2.2.3, 2.3.2

Type: Bug Priority: Major - P3
Reporter: Aaron Staple Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-8095 Update with positional operator does ... Closed
Related
related to SERVER-7928 Deleting documents constantly corrupt... Closed
related to SERVER-8173 Mod::isIndexed still not correct for ... Closed
Operating System: ALL
Participants:

 Description   

Certain mods are considered not to affect index entries that they actually do affect, in particular when the positional operator is used. This can result in index corruption when a document becomes out of sync with its index entries.

Test

c = db.c;
 
[ 'a.b', 'a.0', 'a.0.b' ].forEach( function( indexField ) {
 
c.drop();
 
indexSpec = {};
indexSpec[ indexField ] = 1;
 
c.ensureIndex( indexSpec );
 
c.save( { _id:0, a:[ { b:1 } ] } );
c.update( { 'a.b':1 }, { $set:{ 'a.$':{ b:2 } } } );
 
c.remove( { _id:0 } );
c.find( {} ).hint( indexSpec ).itcount();
 
} );

Output:

Tue Oct 30 11:46:06 [conn11] unindex failed (key too big?) test.c.$a.0_1 key: { : { b: 2.0 } } _id: 0.0
Tue Oct 30 11:46:06 [conn11] Assertion: 10334:Invalid BSONObj size: -286331154 (0xEEEEEEEE) first element: _id: 0.0
0x10de09165 0x10ddcfe1b 0x10ddb3df3 0x10ddb400a 0x10d71fbac 0x10d70aa9c 0x10d6764c4 0x10d67f3c7 0x10d7ab255 0x10da26fea 0x10da22a3d 0x10da250ec 0x10da28f95 0x10da2dab1 0x10d960832 0x10d965730 0x10d654a67 0x10ddef783 0x10ddf0cc0 0x10ddf0d22 
 0   mongod                              0x000000010de09165 _ZN5mongo15printStackTraceERSo + 37
 1   mongod                              0x000000010ddcfe1b _ZN5mongo10logContextEPKc + 123
 2   mongod                              0x000000010ddb3df3 _ZN5mongo11msgassertedEiPKc + 339
 3   mongod                              0x000000010ddb400a _ZNK5mongo13ExceptionInfo6appendERNS_14BSONObjBuilderEPKcS4_ + 0
 4   mongod                              0x000000010d71fbac _ZNK5mongo7BSONObj14_assertInvalidEv + 732
 5   mongod                              0x000000010d70aa9c _ZN5mongo7BSONObj4initEPKc + 76
 6   mongod                              0x000000010d6764c4 _ZN5mongo7BSONObjC1EPKc + 60
 7   mongod                              0x000000010d67f3c7 _ZN5mongo7BSONObj4makeEPKNS_6RecordE + 55
 8   mongod                              0x000000010d7ab255 _ZN5mongo11BtreeCursor7currentEv + 63
 9   mongod                              0x000000010da26fea _ZNK5mongo21ResponseBuildStrategy7currentEb + 944
 10  mongod                              0x000000010da22a3d _ZN5mongo20OrderedBuildStrategy11handleMatchERbRNS_12MatchDetailsE + 379
 11  mongod                              0x000000010da250ec _ZN5mongo20QueryResponseBuilder8addMatchEv + 342
 12  mongod                              0x000000010da28f95 _ZN5mongo23queryWithQueryOptimizerEiRKSsRKNS_7BSONObjERNS_5CurOpES4_S4_RKN5boost10shared_ptrINS_11ParsedQueryEEES4_RKNS_17ShardChunkVersionERNS7_10scoped_ptrINS_25PageFaultRetryableSectionEEERNSG_INS_19NoPageFaultsAllowedEEERNS_7MessageE + 1893
 13  mongod                              0x000000010da2dab1 _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 7505
 14  mongod                              0x000000010d960832 _ZN5mongoL13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 418
 15  mongod                              0x000000010d965730 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 1136
 16  mongod                              0x000000010d654a67 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 261
 17  mongod                              0x000000010ddef783 _ZN5mongo3pms9threadRunEPNS_13MessagingPortE + 2275
 18  mongod                              0x000000010ddf0cc0 _ZN5boost3_bi5list1INS0_5valueIPN5mongo13MessagingPortEEEEclIPFvS5_ENS0_5list0EEEvNS0_4typeIvEERT_RT0_i + 78
 19  mongod                              0x000000010ddf0d22 _ZN5boost3_bi6bind_tIvPFvPN5mongo13MessagingPortEENS0_5list1INS0_5valueIS4_EEEEEclEv + 92
Tue Oct 30 11:46:06 [conn11] assertion 10334 Invalid BSONObj size: -286331154 (0xEEEEEEEE) first element: _id: 0.0 ns:test.c query:{ query: {}, $hint: { a.0: 1.0 } }
Tue Oct 30 11:46:06 [conn11] problem detected during query over test.c : { $err: "Invalid BSONObj size: -286331154 (0xEEEEEEEE) first element: _id: 0.0", code: 10334 }
 

See Mod::isIndexed().

We should also:

  • Fix this out of bounds std::string read

                        if ( ! isdigit( fullName[i+1] ) )
                            continue;

  • Audit isIndexed for any other problems.


 Comments   
Comment by auto [ 09/Jan/13 ]

Author:

{u'date': u'2013-01-09T19:47:34Z', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-7511: fix backport for 2.2
Branch: v2.2
https://github.com/mongodb/mongo/commit/0a6bac4569c92c7ef92cb4b66f5ec0797b1334af

Comment by auto [ 09/Jan/13 ]

Author:

{u'date': u'2013-01-09T17:12:05Z', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-7511: fix update with positional mod or index offset
Branch: v2.2
https://github.com/mongodb/mongo/commit/38f42b2a42011c23385ba899ab84f78bfa9dc1af

Comment by auto [ 09/Jan/13 ]

Author:

{u'date': u'2013-01-09T17:12:05Z', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-7511: fix update with positional mod or index offset
Branch: master
https://github.com/mongodb/mongo/commit/97eca5fe66c68f290e9c29535fce1e058988660d

Comment by Thomas Rueckstiess [ 07/Jan/13 ]

The same bug occurs when using the "array index" syntax in the update, for example:

c.update( { _id:0 }, { $set:{ 'a.0':{ b:2 } } } );

Possible work-arounds for either syntax:

  1. Instead of updating individual elements of an array, update the full array. Example:

    a_array = db.coll.findOne({_id:0}, {a:1})['a']
    // modify a_array on the client-side
    c.update({_id:0}, {$set: {a: a_array}})

  2. Instead of setting the complete document in the array ( {b:1}

    ), reach into the document and set its fields individually. Example:

    c.update({_id:0}, {$set: {'a.0.b': 1}})

For the second alternative, several fields can still be updated in a single update.

Generated at Thu Feb 08 03:14:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.