|
There is a new official build up that should have this as well.
|
|
I'm leaving this building indexes, but both my test case has passed and its farther along in index build than its been before.
Thanks Eliot.
-N
|
|
Am currently fighting with boost, but I built a version of Mongod on an older boost and it looks like this does fix things. I'll confirm further with a newer boost shortly. The automated build seems to have failed on a replication unit test.
-N
|
|
Think i fixed that case as well.
Pushed a build - will be about 90 minutes
http://buildbot.mongodb.org/waterfall?show=V1.4%20Linux%2064-bit
|
|
Author:
{'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}
Message: another case for SERVER-1082
http://github.com/mongodb/mongo/commit/4e8cafb9e909fe9a71ba12fc1863251cfa7455b0
|
|
Author:
{'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}
Message: another case for SERVER-1082
http://github.com/mongodb/mongo/commit/5d4d3c74f1b9e33fc5b0c6d8d3910eab572a7bfe
|
|
Reproduced on 1.5.1:
> db.fields_current.node.ensureIndex(
{ status: 1, "@fields.ex_node_group.value": 1, "@fields.current.value": 1, "@fields.promoted_media.fid": 1 }
);
!e.eoo()
Tue May 4 18:41:08 connection accepted from 127.0.0.1:55704 #3
Tue May 4 18:42:25 building new index on
{ status: 1.0, @fields.ex_node_group.value: 1.0, @fields.current.value: 1.0, @fields.promoted_media.fid: 1.0 }
for examiner_migration.fields_current.node
Tue May 4 18:42:25 Buildindex examiner_migration.fields_current.node idxNo:1 { _id: ObjectId('4be0a2d196b388ebbbcbdb00'), ns: "examiner_migration.fields_current.node", key:
{ status: 1.0, @fields.ex_node_group.value: 1.0, @fields.current.value: 1.0, @fields.promoted_media.fid: 1.0 }
, name: "status_1_@fields.ex_node_group.value_1_@fields.current.value_1_@fields..." }
Tue May 4 18:42:26 examiner_migration.system.indexes Assertion failure !e.eoo() db/../bson/bsonobjbuilder.h 65
0x503dc6 0x50e334 0x4bcfcf 0x5834f0 0x58402a 0x653ccc 0x655b1a 0x65703c 0x658c25 0x63a79e 0x63fb1c 0x6ea3b2 0x6fd800 0x2aaaaacce617 0x2aaaab748c2d
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZN5mongo12sayDbContextEPKc+0xe6) [0x503dc6]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZN5mongo8assertedEPKcS1_j+0x154) [0x50e334]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZN5mongo14BSONObjBuilder8appendAsERKNS_11BSONElementEPKc+0x14f) [0x4bcfcf]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZNK5mongo9IndexSpec8_getKeysESt6vectorIPKcSaIS3_EES1_INS_11BSONElementESaIS6_EERKNS_7BSONObjERSt3setIS9_NS_22BSONObjCmpDefaultOrderESaIS9_EE+0x530) [0x5834f0]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZNK5mongo9IndexSpec7getKeysERKNS_7BSONObjERSt3setIS1_NS_22BSONObjCmpDefaultOrderESaIS1_EE+0x2ea) [0x58402a]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZN5mongo14fastBuildIndexEPKcPNS_16NamespaceDetailsERNS_12IndexDetailsEi+0x62c) [0x653ccc]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod [0x655b1a]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibRKNS_11BSONElementEb+0xa4c) [0x65703c]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZN5mongo11DataFileMgr16insertWithObjModEPKcRNS_7BSONObjEb+0x55) [0x658c25]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZN5mongo14receivedInsertERNS_7MessageERNS_5CurOpE+0x21e) [0x63a79e]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0x179c) [0x63fb1c]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(_ZN5mongo10connThreadEv+0x232) [0x6ea3b2]
/usr/local/mongodb-linux-x86_64-2010-05-04/bin/mongod(thread_proxy+0x80) [0x6fd800]
/lib64/libpthread.so.0 [0x2aaaaacce617]
/lib64/libc.so.6(clone+0x6d) [0x2aaaab748c2d]
Tue May 4 18:42:26 Caught Assertion in insert , continuing
Tue May 4 18:42:26 insert examiner_migration.system.indexes exception assertion db/../bson/bsonobjbuilder.h:65 665ms
Tue May 4 18:12:25 Mongo DB : starting : pid = 11865 port = 29019 dbpath = data2 master = 0 slave = 0 64-bit
****
WARNING: This is development version of MongoDB. Not recommended for production.
****
Tue May 4 18:12:25 db version v1.5.2-pre-, pdfile version 4.5
Tue May 4 18:12:25 git version: ffc097e429cd58aea0063be168b91e229b4f9942
Tue May 4 18:12:25 sys info: Linux domU-12-31-39-06-79-A1 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Tue May 4 18:12:25 waiting for connections on port 29019
|
|
Attached is the section of our log containing this entire session, as some of the indexes built correctly and some didn't depending on the index. Rather than looking for the pattern myself and missing something, I figured I'd just give you the entire session.
-N
|
|
After I updated to the nightly I started reindexing because my testing have shown that the upgrade did not help but reindexing did. I then left for the night with a little script that ran reindex on every collection. That apparently backfired.
|
|
I'm afraid we maybe having some issues with this fix, after upgrading to the nightly release with these commits, we are seeing assertions on index build. The following is an except of this behavior:
Tue May 4 17:10:59 insert examiner_0428.system.indexes exception assertion db/jsobj.h:1196 247ms
Tue May 4 17:10:59 building new index on
{ status: 1, @fields.ex_node_group.value: 1, @fields.current.value: 1, @fields.promoted_media.fid: 1 }
for examiner_0428.fields_current.node
Tue May 4 17:10:59 Buildindex examiner_0428.fields_current.node idxNo:9 { _id: ObjId(4be08d638ead0ecf33210000), ns: "examiner_0428.fields_current.node", key:
{ status: 1, @fields.ex_node_group.value: 1, @fields.current.value: 1, @fields.promoted_media.fid: 1 }
, name: "ex_skybox_pageviews", background: false }
Tue May 4 17:10:59 examiner_0428.system.indexes Assertion failure !e.eoo() db/jsobj.h 1196
0x4f6696 0x4feda4 0x4b837b 0x658a91 0x659cda 0x607ffb 0x609523 0x60a8c2 0x60c4e6 0x5f5f9c 0x5f8887 0x6aa034 0x6bc730 0x2aaaaacce617 0x2aaaab748c2d
mongod(_ZN5mongo12sayDbContextEPKc+0xe6) [0x4f6696]
mongod(_ZN5mongo8assertedEPKcS1_j+0x154) [0x4feda4]
mongod(_ZN5mongo14BSONObjBuilder8appendAsERKNS_11BSONElementEPKc+0x14b) [0x4b837b]
mongod(_ZNK5mongo9IndexSpec8_getKeysESt6vectorIPKcSaIS3_EES1_INS_11BSONElementESaIS6_EERKNS_7BSONObjERSt3setIS9_NS_22BSONObjCmpDefaultOrderESaIS9_EE+0x411) [0x658a91]
mongod(_ZNK5mongo9IndexSpec7getKeysERKNS_7BSONObjERSt3setIS1_NS_22BSONObjCmpDefaultOrderESaIS1_EE+0x2ea) [0x659cda]
mongod(_ZN5mongo14fastBuildIndexEPKcPNS_16NamespaceDetailsERNS_12IndexDetailsEi+0x56b) [0x607ffb]
mongod [0x609523]
mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibRKNS_11BSONElementEb+0x8a2) [0x60a8c2]
mongod(_ZN5mongo11DataFileMgr6insertEPKcRNS_7BSONObjEb+0x66) [0x60c4e6]
mongod(_ZN5mongo14receivedInsertERNS_7MessageERNS_5CurOpE+0x26c) [0x5f5f9c]
mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERK11sockaddr_in+0x14e7) [0x5f8887]
mongod(_ZN5mongo10connThreadEv+0x244) [0x6aa034]
mongod(thread_proxy+0x80) [0x6bc730]
/lib64/libpthread.so.0 [0x2aaaaacce617]
/lib64/libc.so.6(clone+0x6d) [0x2aaaab748c2d]
Tue May 4 17:10:59 Caught Assertion in insert , continuing
Tue May 4 17:10:59 insert examiner_0428.system.indexes exception assertion db/jsobj.h:1196 184ms
Tue May 4 17:10:59 end connection 10.192.198.15:52586
Tue May 4 17:20:31 connection accepted from 10.192.198.15:33785 #2330
Here is the banner from this version so you can see the git string:
Tue May 4 02:43:26 Mongo DB : starting : pid = 8319 port = 27017 dbpath = /ebs/lib/mongodb/data/ master = 0 slave = 0 64-bit
Tue May 4 02:43:26 db version v1.4.3-pre-, pdfile version 4.5
Tue May 4 02:43:26 git version: fe2d4d93f2599cf2b955e880be2fb64d8dca9097
Tue May 4 02:43:26 sys info: Linux domU-12-31-39-06-79-A1 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Tue May 4 02:43:26 waiting for connections on port 27017
Tue May 4 02:43:26 web admin interface listening on port 28017
Downgrading to 1.4.2 removed these assertions on IndexBuild.
|
|
We ran this for a day, and it broke our index creations. I think that Karoly removed all indexes, so that this nightly build could recreate them. But it would only create some indexes before seeing these errors in the log. We backed out the nightly and go back to the previous version, and index creation is working fine.
Tue May 4 17:10:58 building new index on
{ uid: 1, @fields.ex_node_group.value: 1, @fields.examiner_title.tid: 1, @fields.tag_summary.tid: 1, status: 1, created: -1 }
for examiner_0428.fields_current.node
Tue May 4 17:10:58 Buildindex examiner_0428.fields_current.node idxNo:9 { _id: ObjId(4be08d628ead0ecf33200000), ns: "examiner_0428.fields_current.node", key:
{ uid: 1, @fields.ex_node_group.value: 1, @fields.examiner_title.tid: 1, @fields.tag_summary.tid: 1, status: 1, created: -1 }
, name: "author_nodes_per_examiner_title_per_tag", background: false }
Tue May 4 17:10:59 examiner_0428.system.indexes Assertion failure !e.eoo() db/jsobj.h 1196
0x4f6696 0x4feda4 0x4b837b 0x658a91 0x659cda 0x607ffb 0x609523 0x60a8c2 0x60c4e6 0x5f5f9c 0x5f8887 0x6aa034 0x6bc730 0x2aaaaacce617 0x2aaaab748c2d
mongod(_ZN5mongo12sayDbContextEPKc+0xe6) [0x4f6696]
mongod(_ZN5mongo8assertedEPKcS1_j+0x154) [0x4feda4]
mongod(_ZN5mongo14BSONObjBuilder8appendAsERKNS_11BSONElementEPKc+0x14b) [0x4b837b]
mongod(_ZNK5mongo9IndexSpec8_getKeysESt6vectorIPKcSaIS3_EES1_INS_11BSONElementESaIS6_EERKNS_7BSONObjERSt3setIS9_NS_22BSONObjCmpDefaultOrderESaIS9_EE+0x411) [0x658a91]
mongod(_ZNK5mongo9IndexSpec7getKeysERKNS_7BSONObjERSt3setIS1_NS_22BSONObjCmpDefaultOrderESaIS1_EE+0x2ea) [0x659cda]
mongod(_ZN5mongo14fastBuildIndexEPKcPNS_16NamespaceDetailsERNS_12IndexDetailsEi+0x56b) [0x607ffb]
mongod [0x609523]
mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibRKNS_11BSONElementEb+0x8a2) [0x60a8c2]
mongod(_ZN5mongo11DataFileMgr6insertEPKcRNS_7BSONObjEb+0x66) [0x60c4e6]
mongod(_ZN5mongo14receivedInsertERNS_7MessageERNS_5CurOpE+0x26c) [0x5f5f9c]
mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERK11sockaddr_in+0x14e7) [0x5f8887]
mongod(_ZN5mongo10connThreadEv+0x244) [0x6aa034]
mongod(thread_proxy+0x80) [0x6bc730]
/lib64/libpthread.so.0 [0x2aaaaacce617]
/lib64/libc.so.6(clone+0x6d) [0x2aaaab748c2d]
Tue May 4 17:10:59 Caught Assertion in insert , continuing
Tue May 4 17:10:59 insert examiner_0428.system.indexes exception assertion db/jsobj.h:1196 247ms
|
|
Thank you SO MUCH for prompt discovery and fixing.
|
|
Author:
{'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}
Message: fix array index when empty array SERVER-1082
http://github.com/mongodb/mongo/commit/fe2d4d93f2599cf2b955e880be2fb64d8dca9097
|
|
Author:
{'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}
Message: fix array index when empty array SERVER-1082
Note: this only increase index keys, not decrease or change, so old indexes won't cause corruption
http://github.com/mongodb/mongo/commit/c540e566de6c1e2a478f62beb66efbf65a5fba56
|
|
yeah - that's wrong - but doesn't impact the case.
|
|
please fix in 1.4 if possible too.
Also... that commit might be broken, you are inserting two _id: 1 items?
|
|
Author:
{'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}
Message: broken test for SERVER-1082
http://github.com/mongodb/mongo/commit/98e28e523c5110c48e4e6e861780f6d20a669c86
|
Generated at Thu Feb 08 02:56:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.