|
Hi Roy,
sorry for the delay on this issue, but we are still not able to reproduce the segmentation fault.
I've modified the data and made the fields arrays like your example documents. I created all the indices you provided, then executed both commands simultaneously in two mongo shells. Here is a snippet of the log file that shows that the commands are running successfully and are updating results and aggregating data.
2014-09-23T14:29:47.569+0200 [conn15] update test.files query: { appId: 11114.0, fileStatus: { $nin: [ 2.0, "DELETED" ] }, snapshotLastScanned: { $lt: 1409316400697.0 } } update: { $inc: { unseenScans: 1.0 } } nscanned:17515 nscannedObjects:6400 nMatched:5147 nModified:5147 keyUpdates:2 numYields:148 locks(micros) w:2880786 1527ms
|
2014-09-23T14:29:47.570+0200 [conn15] command test.$cmd command: update { update: "files", ordered: true, updates: [ { q: { appId: 11114.0, fileStatus: { $nin: [ 2.0, "DELETED" ] }, snapshotLastScanned: { $lt: 1409316400697.0 } }, u: { $inc: { unseenScans: 1.0 } }, multi: true } ] } keyUpdates:0 numYields:0 reslen:55 1528ms
|
2014-09-23T14:29:47.572+0200 [conn18] command test.$cmd command: aggregate { aggregate: "files", pipeline: [ { $match: { isForeign: false, fileStatus: { $in: [ 0, 1 ] }, isFolder: false, appId: 11114 } }, { $group: { count: { $sum: 1 }, _id: "$fileAccessLevel" } } ], cursor: {} } keyUpdates:0 numYields:19 locks(micros) r:13261 reslen:297 13ms
|
2014-09-23T14:29:47.814+0200 [conn18] command test.$cmd command: aggregate { aggregate: "files", pipeline: [ { $match: { isForeign: false, fileStatus: { $in: [ 0, 1 ] }, isFolder: false, appId: 11114 } }, { $group: { count: { $sum: 1 }, _id: "$fileAccessLevel" } } ], cursor: {} } keyUpdates:0 numYields:19 locks(micros) r:15756 reslen:297 241ms
|
2014-09-23T14:29:48.711+0200 [conn18] command test.$cmd command: aggregate { aggregate: "files", pipeline: [ { $match: { isForeign: false, fileStatus: { $in: [ 0, 1 ] }, isFolder: false, appId: 11114 } }, { $group: { count: { $sum: 1 }, _id: "$fileAccessLevel" } } ], cursor: {} } keyUpdates:0 numYields:50 locks(micros) r:49784 reslen:297 884ms
|
2014-09-23T14:29:49.155+0200 [conn18] command test.$cmd command: aggregate { aggregate: "files", pipeline: [ { $match: { isForeign: false, fileStatus: { $in: [ 0, 1 ] }, isFolder: false, appId: 11114 } }, { $group: { count: { $sum: 1 }, _id: "$fileAccessLevel" } } ], cursor: {} } keyUpdates:0 numYields:49 locks(micros) r:37877 reslen:297 428ms
|
2014-09-23T14:29:49.169+0200 [conn18] command test.$cmd command: aggregate { aggregate: "files", pipeline: [ { $match: { isForeign: false, fileStatus: { $in: [ 0, 1 ] }, isFolder: false, appId: 11114 } }, { $group: { count: { $sum: 1 }, _id: "$fileAccessLevel" } } ], cursor: {} } keyUpdates:0 numYields:19 locks(micros) r:13253 reslen:297 12ms
|
2014-09-23T14:29:49.170+0200 [conn15] update test.files query: { appId: 11114.0, fileStatus: { $nin: [ 2.0, "DELETED" ] }, snapshotLastScanned: { $lt: 1409316400697.0 } } update: { $inc: { unseenScans: 1.0 } } nscanned:17515 nscannedObjects:6400 nMatched:5147 nModified:5147 keyUpdates:2 numYields:147 locks(micros) w:3019634 1599ms
|
2014-09-23T14:29:49.170+0200 [conn15] command test.$cmd command: update { update: "files", ordered: true, updates: [ { q: { appId: 11114.0, fileStatus: { $nin: [ 2.0, "DELETED" ] }, snapshotLastScanned: { $lt: 1409316400697.0 } }, u: { $inc: { unseenScans: 1.0 } }, multi: true } ] } keyUpdates:0 numYields:0 reslen:55 1600ms
|
2014-09-23T14:29:49.367+0200 [conn18] command test.$cmd command: aggregate { aggregate: "files", pipeline: [ { $match: { isForeign: false, fileStatus: { $in: [ 0, 1 ] }, isFolder: false, appId: 11114 } }, { $group: { count: { $sum: 1 }, _id: "$fileAccessLevel" } } ], cursor: {} } keyUpdates:0 numYields:19 locks(micros) r:15569 reslen:297 195ms
|
Can you run a validate command on this collection with the "full" option to see if there are any corruption issues? Notice that validate is a resource-intensive command and should best be run during off-peak hours or a maintenance window:
Please let us know what the validate command returns.
If there is a corrupt index, you may be able to drop the particular index and re-create it. A verbose log file (verbosity level 2 or higher ideally) during such a segmentation fault would also be helpful.
Regards,
Thomas
|
|
Hi,
Actually I converted my aggregation into a map/reduce, and it still happened. So what I did was to convert that map/reduce's query into a query that does not use the same index as the update and filters out some of the parameters in the map function - and it didn't crash anymore. So I think that indexes are indeed relevant here.
Just a few changes (not sure if they actually have any effect):
fileAccessLevel is an array (example: "fileAccessLevel" : [ 0, "PRIVATE" ])
same goes for fileStatus (example: "fileStatus" : [ 1, "TRASHED" ])
My indexes look like this (sorry for the long list..):
[
|
{
|
"v" : 1,
|
"key" : {
|
"_id" : 1
|
},
|
"name" : "_id_",
|
"ns" : "Tenant_2.files"
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"appId" : 1,
|
"domains" : 1
|
},
|
"name" : "appId_1_domains_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"fileAccessLevel" : 1
|
},
|
"name" : "fileAccessLevel_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"emails" : 1
|
},
|
"name" : "emails_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"groupIds" : 1
|
},
|
"name" : "groupIds_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"_fts" : "text",
|
"_ftsx" : 1
|
},
|
"name" : "name_text",
|
"ns" : "Tenant_2.files",
|
"background" : true,
|
"weights" : {
|
"name" : 1
|
},
|
"default_language" : "english",
|
"language_override" : "language",
|
"textIndexVersion" : 2
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"appId" : 1,
|
"parentKeyPrefix" : 1,
|
"parentType" : 1
|
},
|
"name" : "appId_1_parentKeyPrefix_1_parentType_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"modifiedDate" : -1
|
},
|
"name" : "modifiedDate_-1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"id" : "hashed"
|
},
|
"name" : "id_hashed",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"appId" : "hashed"
|
},
|
"name" : "appId_hashed",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"ownerAddress" : "hashed"
|
},
|
"name" : "ownerAddress_hashed",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isFolder" : "hashed"
|
},
|
"name" : "isFolder_hashed",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isForeign" : "hashed"
|
},
|
"name" : "isForeign_hashed",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"objectType" : "hashed"
|
},
|
"name" : "objectType_hashed",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"labels.trashed" : "hashed"
|
},
|
"name" : "labels.trashed_hashed",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isTrashed" : "hashed"
|
},
|
"name" : "isTrashed_hashed",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"fileSize" : -1,
|
"appId" : 1
|
},
|
"name" : "fileSize_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"ownerAddress" : -1,
|
"appId" : 1
|
},
|
"name" : "ownerAddress_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"fileAccessLevel" : -1,
|
"appId" : 1
|
},
|
"name" : "fileAccessLevel_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"name" : -1,
|
"appId" : 1
|
},
|
"name" : "name_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"modifiedDate" : -1,
|
"appId" : 1
|
},
|
"name" : "modifiedDate_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isForeign" : 1,
|
"fileStatus" : 1,
|
"fileSize" : -1,
|
"appId" : 1
|
},
|
"name" : "isForeign_1_fileStatus_1_fileSize_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isForeign" : 1,
|
"fileStatus" : 1,
|
"ownerAddress" : -1,
|
"appId" : 1
|
},
|
"name" : "isForeign_1_fileStatus_1_ownerAddress_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isForeign" : 1,
|
"fileAccessLevel" : -1,
|
"appId" : 1
|
},
|
"name" : "isForeign_1_fileAccessLevel_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isForeign" : 1,
|
"fileStatus" : 1,
|
"name" : -1,
|
"appId" : 1
|
},
|
"name" : "isForeign_1_fileStatus_1_name_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isForeign" : 1,
|
"fileStatus" : 1,
|
"modifiedDate" : -1,
|
"appId" : 1
|
},
|
"name" : "isForeign_1_fileStatus_1_modifiedDate_-1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isForeign" : 1,
|
"fileStatus" : 1,
|
"appId" : 1
|
},
|
"name" : "isForeign_1_fileStatus_1_appId_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"name" : 1
|
},
|
"name" : "name_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"fileStatus" : 1
|
},
|
"name" : "fileStatus_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"isForeign" : 1,
|
"fileStatus" : 1
|
},
|
"name" : "isForeign_1_fileStatus_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"parentIds" : 1
|
},
|
"name" : "parentIds_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"cabinetState" : 1
|
},
|
"name" : "cabinetState_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"appId" : 1,
|
"fileStatus" : 1,
|
"ownerAddress" : 1
|
},
|
"name" : "appId_1_fileStatus_1_ownerAddress_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"parents.id" : 1
|
},
|
"name" : "parents.id_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"parentId" : "hashed"
|
},
|
"name" : "parentId_hashed",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
},
|
{
|
"v" : 1,
|
"key" : {
|
"appId" : 1,
|
"fileStatus" : 1,
|
"unseenScans" : 1,
|
"snapshotLastScanned" : 1
|
},
|
"name" : "appId_1_fileStatus_1_unseenScans_1_snapshotLastScanned_1",
|
"ns" : "Tenant_2.files",
|
"background" : true
|
}
|
]
|
|