[SERVER-19265] Aggregation running only on 8 shards at a time Created: 02/Jul/15  Updated: 03/Jul/15  Resolved: 02/Jul/15

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: 3.0.2
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: John Page Assignee: Mathias Stearn
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

I'm running aggregations against 12 shards. This is with wired tiger.
The aggregation is a complex project then a group with sum.

I see first the aggregation run on 8 shards
then run on the remaining 4 shards
then a brief flurry on the primary shard.

Why is this - should all 12 not run in parallel.



 Comments   
Comment by John Page [ 02/Jul/15 ]

PEBCAK

Comment by John Page [ 02/Jul/15 ]

OK PrintShardingStatus is the clue here

			shard0000	339
				shard0001	343
				shard0002	341
				shard0003	342
				shard0004	336
				shard0005	333
				shard0006	335
				shard0007	339

For whatever reason I only have 8 shards !!!
Doh Let me go fix that

Comment by John Page [ 02/Jul/15 ]

mongos> db.mot_results.count()
580650947
mongos> db.mot_results.aggregate({$group:{_id:1,count:{$sum:1}}})
{ "_id" : 1, "count" : 580650947 }

Comment by Mathias Stearn [ 02/Jul/15 ]

Can you post printShardingStatus()?

Comment by John Page [ 02/Jul/15 ]

Observing same behaviour on a simple {$group:{_id:1,count:{$sum:1}}}

Comment by John Page [ 02/Jul/15 ]

So The server is 30GB/16CPU/1 SLOW Spinning Disk EBS Drive (150 IOPS & 60MB/s throughput)

12 Wired Tiger Microshards with WT cache set to 2MB (Yes Megabytes) - all Data fits in O/S Disk Cache
So Aggregation after first run is out of RAM (This is the whole point )

Aggregation seems to run OK - although I've not done a simple count - BUT it runs for 2 minutes with 8 mongoD's at 100% and the rest at 0% then another 2 minutes with a different 4 at 100%
why not 12 at the same time?

Have sent you login details

Comment by Mathias Stearn [ 02/Jul/15 ]

Yeah. I'm actually more interested in the empty pipeline explain and the printShardingStatus output to be honest.

Comment by John Page [ 02/Jul/15 ]

Just a group on an existing field? It's on aws would you like login access? Or I have a single script to build it.

Comment by Mathias Stearn [ 02/Jul/15 ]

It seems to have decided to only dispatch to shards 0-7. Could you include explains with just a $group stage, an empty pipeline and the output of printShardingStatus()?

Comment by John Page [ 02/Jul/15 ]

Let me know if you want access to the data to observe this

Comment by John Page [ 02/Jul/15 ]

db.mot_results.aggregate([banding,group,sort],{explain:true})
{
	"splitPipeline" : {
		"shardsPart" : [
			{
				"$project" : {
					"b" : {
						"$multiply" : [
							{
								"$cond" : [
									{
										"$gt" : [
											"$e",
											{
												"$const" : 1000
											}
										]
									},
									{
										"$cond" : [
											{
												"$gt" : [
													"$e",
													{
														"$const" : 1200
													}
												]
											},
											{
												"$cond" : [
													{
														"$gt" : [
															"$e",
															{
																"$const" : 1600
															}
														]
													},
													{
														"$cond" : [
															{
																"$gt" : [
																	"$e",
																	{
																		"$const" : 2000
																	}
																]
															},
															{
																"$cond" : [
																	{
																		"$gt" : [
																			"$e",
																			{
																				"$const" : 2500
																			}
																		]
																	},
																	{
																		"$cond" : [
																			{
																				"$gt" : [
																					"$e",
																					{
																						"$const" : 3500
																					}
																				]
																			},
																			{
																				"$const" : 7
																			},
																			{
																				"$const" : 6
																			}
																		]
																	},
																	{
																		"$const" : 5
																	}
																]
															},
															{
																"$const" : 4
															}
														]
													},
													{
														"$const" : 3
													}
												]
											},
											{
												"$const" : 2
											}
										]
									},
									{
										"$const" : 1
									}
								]
							},
							{
								"$cond" : [
									{
										"$eq" : [
											"$f",
											{
												"$const" : "D"
											}
										]
									},
									{
										"$const" : 1.5
									},
									{
										"$const" : 1
									}
								]
							}
						]
					},
					"m" : true
				}
			},
			{
				"$group" : {
					"_id" : "$b",
					"count" : {
						"$sum" : {
							"$const" : 1
						}
					},
					"miles" : {
						"$sum" : "$m"
					}
				}
			}
		],
		"mergerPart" : [
			{
				"$group" : {
					"_id" : "$$ROOT._id",
					"count" : {
						"$sum" : "$$ROOT.count"
					},
					"miles" : {
						"$sum" : "$$ROOT.miles"
					},
					"$doingMerge" : true
				}
			},
			{
				"$sort" : {
					"sortKey" : {
						"_id" : 1
					}
				}
			}
		]
	},
	"shards" : {
		"shard0000" : {
			"host" : "localhost:27001",
			"stages" : [
				{
					"$cursor" : {
						"query" : {
							
						},
						"fields" : {
							"e" : 1,
							"f" : 1,
							"m" : 1,
							"_id" : 1
						},
						"queryPlanner" : {
							"plannerVersion" : 1,
							"namespace" : "vosa.mot_results",
							"indexFilterSet" : false,
							"parsedQuery" : {
								"$and" : [ ]
							},
							"winningPlan" : {
								"stage" : "SHARDING_FILTER",
								"inputStage" : {
									"stage" : "COLLSCAN",
									"filter" : {
										"$and" : [ ]
									},
									"direction" : "forward"
								}
							},
							"rejectedPlans" : [ ]
						}
					}
				},
				{
					"$project" : {
						"b" : {
							"$multiply" : [
								{
									"$cond" : [
										{
											"$gt" : [
												"$e",
												{
													"$const" : 1000
												}
											]
										},
										{
											"$cond" : [
												{
													"$gt" : [
														"$e",
														{
															"$const" : 1200
														}
													]
												},
												{
													"$cond" : [
														{
															"$gt" : [
																"$e",
																{
																	"$const" : 1600
																}
															]
														},
														{
															"$cond" : [
																{
																	"$gt" : [
																		"$e",
																		{
																			"$const" : 2000
																		}
																	]
																},
																{
																	"$cond" : [
																		{
																			"$gt" : [
																				"$e",
																				{
																					"$const" : 2500
																				}
																			]
																		},
																		{
																			"$cond" : [
																				{
																					"$gt" : [
																						"$e",
																						{
																							"$const" : 3500
																						}
																					]
																				},
																				{
																					"$const" : 7
																				},
																				{
																					"$const" : 6
																				}
																			]
																		},
																		{
																			"$const" : 5
																		}
																	]
																},
																{
																	"$const" : 4
																}
															]
														},
														{
															"$const" : 3
														}
													]
												},
												{
													"$const" : 2
												}
											]
										},
										{
											"$const" : 1
										}
									]
								},
								{
									"$cond" : [
										{
											"$eq" : [
												"$f",
												{
													"$const" : "D"
												}
											]
										},
										{
											"$const" : 1.5
										},
										{
											"$const" : 1
										}
									]
								}
							]
						},
						"m" : true
					}
				},
				{
					"$group" : {
						"_id" : "$b",
						"count" : {
							"$sum" : {
								"$const" : 1
							}
						},
						"miles" : {
							"$sum" : "$m"
						}
					}
				}
			]
		},
		"shard0001" : {
			"host" : "localhost:27002",
			"stages" : [
				{
					"$cursor" : {
						"query" : {
							
						},
						"fields" : {
							"e" : 1,
							"f" : 1,
							"m" : 1,
							"_id" : 1
						},
						"queryPlanner" : {
							"plannerVersion" : 1,
							"namespace" : "vosa.mot_results",
							"indexFilterSet" : false,
							"parsedQuery" : {
								"$and" : [ ]
							},
							"winningPlan" : {
								"stage" : "SHARDING_FILTER",
								"inputStage" : {
									"stage" : "COLLSCAN",
									"filter" : {
										"$and" : [ ]
									},
									"direction" : "forward"
								}
							},
							"rejectedPlans" : [ ]
						}
					}
				},
				{
					"$project" : {
						"b" : {
							"$multiply" : [
								{
									"$cond" : [
										{
											"$gt" : [
												"$e",
												{
													"$const" : 1000
												}
											]
										},
										{
											"$cond" : [
												{
													"$gt" : [
														"$e",
														{
															"$const" : 1200
														}
													]
												},
												{
													"$cond" : [
														{
															"$gt" : [
																"$e",
																{
																	"$const" : 1600
																}
															]
														},
														{
															"$cond" : [
																{
																	"$gt" : [
																		"$e",
																		{
																			"$const" : 2000
																		}
																	]
																},
																{
																	"$cond" : [
																		{
																			"$gt" : [
																				"$e",
																				{
																					"$const" : 2500
																				}
																			]
																		},
																		{
																			"$cond" : [
																				{
																					"$gt" : [
																						"$e",
																						{
																							"$const" : 3500
																						}
																					]
																				},
																				{
																					"$const" : 7
																				},
																				{
																					"$const" : 6
																				}
																			]
																		},
																		{
																			"$const" : 5
																		}
																	]
																},
																{
																	"$const" : 4
																}
															]
														},
														{
															"$const" : 3
														}
													]
												},
												{
													"$const" : 2
												}
											]
										},
										{
											"$const" : 1
										}
									]
								},
								{
									"$cond" : [
										{
											"$eq" : [
												"$f",
												{
													"$const" : "D"
												}
											]
										},
										{
											"$const" : 1.5
										},
										{
											"$const" : 1
										}
									]
								}
							]
						},
						"m" : true
					}
				},
				{
					"$group" : {
						"_id" : "$b",
						"count" : {
							"$sum" : {
								"$const" : 1
							}
						},
						"miles" : {
							"$sum" : "$m"
						}
					}
				}
			]
		},
		"shard0002" : {
			"host" : "localhost:27006",
			"stages" : [
				{
					"$cursor" : {
						"query" : {
							
						},
						"fields" : {
							"e" : 1,
							"f" : 1,
							"m" : 1,
							"_id" : 1
						},
						"queryPlanner" : {
							"plannerVersion" : 1,
							"namespace" : "vosa.mot_results",
							"indexFilterSet" : false,
							"parsedQuery" : {
								"$and" : [ ]
							},
							"winningPlan" : {
								"stage" : "SHARDING_FILTER",
								"inputStage" : {
									"stage" : "COLLSCAN",
									"filter" : {
										"$and" : [ ]
									},
									"direction" : "forward"
								}
							},
							"rejectedPlans" : [ ]
						}
					}
				},
				{
					"$project" : {
						"b" : {
							"$multiply" : [
								{
									"$cond" : [
										{
											"$gt" : [
												"$e",
												{
													"$const" : 1000
												}
											]
										},
										{
											"$cond" : [
												{
													"$gt" : [
														"$e",
														{
															"$const" : 1200
														}
													]
												},
												{
													"$cond" : [
														{
															"$gt" : [
																"$e",
																{
																	"$const" : 1600
																}
															]
														},
														{
															"$cond" : [
																{
																	"$gt" : [
																		"$e",
																		{
																			"$const" : 2000
																		}
																	]
																},
																{
																	"$cond" : [
																		{
																			"$gt" : [
																				"$e",
																				{
																					"$const" : 2500
																				}
																			]
																		},
																		{
																			"$cond" : [
																				{
																					"$gt" : [
																						"$e",
																						{
																							"$const" : 3500
																						}
																					]
																				},
																				{
																					"$const" : 7
																				},
																				{
																					"$const" : 6
																				}
																			]
																		},
																		{
																			"$const" : 5
																		}
																	]
																},
																{
																	"$const" : 4
																}
															]
														},
														{
															"$const" : 3
														}
													]
												},
												{
													"$const" : 2
												}
											]
										},
										{
											"$const" : 1
										}
									]
								},
								{
									"$cond" : [
										{
											"$eq" : [
												"$f",
												{
													"$const" : "D"
												}
											]
										},
										{
											"$const" : 1.5
										},
										{
											"$const" : 1
										}
									]
								}
							]
						},
						"m" : true
					}
				},
				{
					"$group" : {
						"_id" : "$b",
						"count" : {
							"$sum" : {
								"$const" : 1
							}
						},
						"miles" : {
							"$sum" : "$m"
						}
					}
				}
			]
		},
		"shard0003" : {
			"host" : "localhost:27008",
			"stages" : [
				{
					"$cursor" : {
						"query" : {
							
						},
						"fields" : {
							"e" : 1,
							"f" : 1,
							"m" : 1,
							"_id" : 1
						},
						"queryPlanner" : {
							"plannerVersion" : 1,
							"namespace" : "vosa.mot_results",
							"indexFilterSet" : false,
							"parsedQuery" : {
								"$and" : [ ]
							},
							"winningPlan" : {
								"stage" : "SHARDING_FILTER",
								"inputStage" : {
									"stage" : "COLLSCAN",
									"filter" : {
										"$and" : [ ]
									},
									"direction" : "forward"
								}
							},
							"rejectedPlans" : [ ]
						}
					}
				},
				{
					"$project" : {
						"b" : {
							"$multiply" : [
								{
									"$cond" : [
										{
											"$gt" : [
												"$e",
												{
													"$const" : 1000
												}
											]
										},
										{
											"$cond" : [
												{
													"$gt" : [
														"$e",
														{
															"$const" : 1200
														}
													]
												},
												{
													"$cond" : [
														{
															"$gt" : [
																"$e",
																{
																	"$const" : 1600
																}
															]
														},
														{
															"$cond" : [
																{
																	"$gt" : [
																		"$e",
																		{
																			"$const" : 2000
																		}
																	]
																},
																{
																	"$cond" : [
																		{
																			"$gt" : [
																				"$e",
																				{
																					"$const" : 2500
																				}
																			]
																		},
																		{
																			"$cond" : [
																				{
																					"$gt" : [
																						"$e",
																						{
																							"$const" : 3500
																						}
																					]
																				},
																				{
																					"$const" : 7
																				},
																				{
																					"$const" : 6
																				}
																			]
																		},
																		{
																			"$const" : 5
																		}
																	]
																},
																{
																	"$const" : 4
																}
															]
														},
														{
															"$const" : 3
														}
													]
												},
												{
													"$const" : 2
												}
											]
										},
										{
											"$const" : 1
										}
									]
								},
								{
									"$cond" : [
										{
											"$eq" : [
												"$f",
												{
													"$const" : "D"
												}
											]
										},
										{
											"$const" : 1.5
										},
										{
											"$const" : 1
										}
									]
								}
							]
						},
						"m" : true
					}
				},
				{
					"$group" : {
						"_id" : "$b",
						"count" : {
							"$sum" : {
								"$const" : 1
							}
						},
						"miles" : {
							"$sum" : "$m"
						}
					}
				}
			]
		},
		"shard0004" : {
			"host" : "localhost:27009",
			"stages" : [
				{
					"$cursor" : {
						"query" : {
							
						},
						"fields" : {
							"e" : 1,
							"f" : 1,
							"m" : 1,
							"_id" : 1
						},
						"queryPlanner" : {
							"plannerVersion" : 1,
							"namespace" : "vosa.mot_results",
							"indexFilterSet" : false,
							"parsedQuery" : {
								"$and" : [ ]
							},
							"winningPlan" : {
								"stage" : "SHARDING_FILTER",
								"inputStage" : {
									"stage" : "COLLSCAN",
									"filter" : {
										"$and" : [ ]
									},
									"direction" : "forward"
								}
							},
							"rejectedPlans" : [ ]
						}
					}
				},
				{
					"$project" : {
						"b" : {
							"$multiply" : [
								{
									"$cond" : [
										{
											"$gt" : [
												"$e",
												{
													"$const" : 1000
												}
											]
										},
										{
											"$cond" : [
												{
													"$gt" : [
														"$e",
														{
															"$const" : 1200
														}
													]
												},
												{
													"$cond" : [
														{
															"$gt" : [
																"$e",
																{
																	"$const" : 1600
																}
															]
														},
														{
															"$cond" : [
																{
																	"$gt" : [
																		"$e",
																		{
																			"$const" : 2000
																		}
																	]
																},
																{
																	"$cond" : [
																		{
																			"$gt" : [
																				"$e",
																				{
																					"$const" : 2500
																				}
																			]
																		},
																		{
																			"$cond" : [
																				{
																					"$gt" : [
																						"$e",
																						{
																							"$const" : 3500
																						}
																					]
																				},
																				{
																					"$const" : 7
																				},
																				{
																					"$const" : 6
																				}
																			]
																		},
																		{
																			"$const" : 5
																		}
																	]
																},
																{
																	"$const" : 4
																}
															]
														},
														{
															"$const" : 3
														}
													]
												},
												{
													"$const" : 2
												}
											]
										},
										{
											"$const" : 1
										}
									]
								},
								{
									"$cond" : [
										{
											"$eq" : [
												"$f",
												{
													"$const" : "D"
												}
											]
										},
										{
											"$const" : 1.5
										},
										{
											"$const" : 1
										}
									]
								}
							]
						},
						"m" : true
					}
				},
				{
					"$group" : {
						"_id" : "$b",
						"count" : {
							"$sum" : {
								"$const" : 1
							}
						},
						"miles" : {
							"$sum" : "$m"
						}
					}
				}
			]
		},
		"shard0005" : {
			"host" : "localhost:27010",
			"stages" : [
				{
					"$cursor" : {
						"query" : {
							
						},
						"fields" : {
							"e" : 1,
							"f" : 1,
							"m" : 1,
							"_id" : 1
						},
						"queryPlanner" : {
							"plannerVersion" : 1,
							"namespace" : "vosa.mot_results",
							"indexFilterSet" : false,
							"parsedQuery" : {
								"$and" : [ ]
							},
							"winningPlan" : {
								"stage" : "SHARDING_FILTER",
								"inputStage" : {
									"stage" : "COLLSCAN",
									"filter" : {
										"$and" : [ ]
									},
									"direction" : "forward"
								}
							},
							"rejectedPlans" : [ ]
						}
					}
				},
				{
					"$project" : {
						"b" : {
							"$multiply" : [
								{
									"$cond" : [
										{
											"$gt" : [
												"$e",
												{
													"$const" : 1000
												}
											]
										},
										{
											"$cond" : [
												{
													"$gt" : [
														"$e",
														{
															"$const" : 1200
														}
													]
												},
												{
													"$cond" : [
														{
															"$gt" : [
																"$e",
																{
																	"$const" : 1600
																}
															]
														},
														{
															"$cond" : [
																{
																	"$gt" : [
																		"$e",
																		{
																			"$const" : 2000
																		}
																	]
																},
																{
																	"$cond" : [
																		{
																			"$gt" : [
																				"$e",
																				{
																					"$const" : 2500
																				}
																			]
																		},
																		{
																			"$cond" : [
																				{
																					"$gt" : [
																						"$e",
																						{
																							"$const" : 3500
																						}
																					]
																				},
																				{
																					"$const" : 7
																				},
																				{
																					"$const" : 6
																				}
																			]
																		},
																		{
																			"$const" : 5
																		}
																	]
																},
																{
																	"$const" : 4
																}
															]
														},
														{
															"$const" : 3
														}
													]
												},
												{
													"$const" : 2
												}
											]
										},
										{
											"$const" : 1
										}
									]
								},
								{
									"$cond" : [
										{
											"$eq" : [
												"$f",
												{
													"$const" : "D"
												}
											]
										},
										{
											"$const" : 1.5
										},
										{
											"$const" : 1
										}
									]
								}
							]
						},
						"m" : true
					}
				},
				{
					"$group" : {
						"_id" : "$b",
						"count" : {
							"$sum" : {
								"$const" : 1
							}
						},
						"miles" : {
							"$sum" : "$m"
						}
					}
				}
			]
		},
		"shard0006" : {
			"host" : "localhost:27011",
			"stages" : [
				{
					"$cursor" : {
						"query" : {
							
						},
						"fields" : {
							"e" : 1,
							"f" : 1,
							"m" : 1,
							"_id" : 1
						},
						"queryPlanner" : {
							"plannerVersion" : 1,
							"namespace" : "vosa.mot_results",
							"indexFilterSet" : false,
							"parsedQuery" : {
								"$and" : [ ]
							},
							"winningPlan" : {
								"stage" : "SHARDING_FILTER",
								"inputStage" : {
									"stage" : "COLLSCAN",
									"filter" : {
										"$and" : [ ]
									},
									"direction" : "forward"
								}
							},
							"rejectedPlans" : [ ]
						}
					}
				},
				{
					"$project" : {
						"b" : {
							"$multiply" : [
								{
									"$cond" : [
										{
											"$gt" : [
												"$e",
												{
													"$const" : 1000
												}
											]
										},
										{
											"$cond" : [
												{
													"$gt" : [
														"$e",
														{
															"$const" : 1200
														}
													]
												},
												{
													"$cond" : [
														{
															"$gt" : [
																"$e",
																{
																	"$const" : 1600
																}
															]
														},
														{
															"$cond" : [
																{
																	"$gt" : [
																		"$e",
																		{
																			"$const" : 2000
																		}
																	]
																},
																{
																	"$cond" : [
																		{
																			"$gt" : [
																				"$e",
																				{
																					"$const" : 2500
																				}
																			]
																		},
																		{
																			"$cond" : [
																				{
																					"$gt" : [
																						"$e",
																						{
																							"$const" : 3500
																						}
																					]
																				},
																				{
																					"$const" : 7
																				},
																				{
																					"$const" : 6
																				}
																			]
																		},
																		{
																			"$const" : 5
																		}
																	]
																},
																{
																	"$const" : 4
																}
															]
														},
														{
															"$const" : 3
														}
													]
												},
												{
													"$const" : 2
												}
											]
										},
										{
											"$const" : 1
										}
									]
								},
								{
									"$cond" : [
										{
											"$eq" : [
												"$f",
												{
													"$const" : "D"
												}
											]
										},
										{
											"$const" : 1.5
										},
										{
											"$const" : 1
										}
									]
								}
							]
						},
						"m" : true
					}
				},
				{
					"$group" : {
						"_id" : "$b",
						"count" : {
							"$sum" : {
								"$const" : 1
							}
						},
						"miles" : {
							"$sum" : "$m"
						}
					}
				}
			]
		},
		"shard0007" : {
			"host" : "localhost:27012",
			"stages" : [
				{
					"$cursor" : {
						"query" : {
							
						},
						"fields" : {
							"e" : 1,
							"f" : 1,
							"m" : 1,
							"_id" : 1
						},
						"queryPlanner" : {
							"plannerVersion" : 1,
							"namespace" : "vosa.mot_results",
							"indexFilterSet" : false,
							"parsedQuery" : {
								"$and" : [ ]
							},
							"winningPlan" : {
								"stage" : "SHARDING_FILTER",
								"inputStage" : {
									"stage" : "COLLSCAN",
									"filter" : {
										"$and" : [ ]
									},
									"direction" : "forward"
								}
							},
							"rejectedPlans" : [ ]
						}
					}
				},
				{
					"$project" : {
						"b" : {
							"$multiply" : [
								{
									"$cond" : [
										{
											"$gt" : [
												"$e",
												{
													"$const" : 1000
												}
											]
										},
										{
											"$cond" : [
												{
													"$gt" : [
														"$e",
														{
															"$const" : 1200
														}
													]
												},
												{
													"$cond" : [
														{
															"$gt" : [
																"$e",
																{
																	"$const" : 1600
																}
															]
														},
														{
															"$cond" : [
																{
																	"$gt" : [
																		"$e",
																		{
																			"$const" : 2000
																		}
																	]
																},
																{
																	"$cond" : [
																		{
																			"$gt" : [
																				"$e",
																				{
																					"$const" : 2500
																				}
																			]
																		},
																		{
																			"$cond" : [
																				{
																					"$gt" : [
																						"$e",
																						{
																							"$const" : 3500
																						}
																					]
																				},
																				{
																					"$const" : 7
																				},
																				{
																					"$const" : 6
																				}
																			]
																		},
																		{
																			"$const" : 5
																		}
																	]
																},
																{
																	"$const" : 4
																}
															]
														},
														{
															"$const" : 3
														}
													]
												},
												{
													"$const" : 2
												}
											]
										},
										{
											"$const" : 1
										}
									]
								},
								{
									"$cond" : [
										{
											"$eq" : [
												"$f",
												{
													"$const" : "D"
												}
											]
										},
										{
											"$const" : 1.5
										},
										{
											"$const" : 1
										}
									]
								}
							]
						},
						"m" : true
					}
				},
				{
					"$group" : {
						"_id" : "$b",
						"count" : {
							"$sum" : {
								"$const" : 1
							}
						},
						"miles" : {
							"$sum" : "$m"
						}
					}
				}
			]
		}
	},
	"ok" : 1
}

Comment by John Page [ 02/Jul/15 ]

isover3500 = { $cond : [ { $gt : [ '$e',3500 ] }, 7, 6 ] }
isover2500 = { $cond : [ { $gt : [ '$e',2500 ] }, isover3500, 5 ] }
isover2000 = { $cond : [ { $gt : [ '$e',2000 ] }, isover2500, 4 ] }
isover1600 = { $cond : [ { $gt : [ '$e',1600 ] }, isover2000, 3 ] }
isover1200 = { $cond : [ { $gt : [ '$e',1200 ] }, isover1600, 2 ] }
getband = { $cond : [ { $gt : [ '$e',1000 ] }, isover1200, 1 ] }
 
isdirty = { $cond : [ { $eq : [ '$f' ,'D']}, 1.5 , 1 ]}
dirtypenalty = { $multiply : [ getband, isdirty ]}
banding = { $project : { b : dirtypenalty , m : 1}}
group = { $group : { _id : '$b' , count : { $sum : 1} , miles : { $sum : '$m' } } }
sort = {$sort:{_id:1}}
db.mot_results.aggregate([banding,group,sort])

Comment by John Page [ 02/Jul/15 ]

strace on mongod that isnt busy shows.

select(8, [6 7], NULL, NULL, {0, 10000}) = 0 (Timeout)

repeating

lsof shows

mongod  58600 ec2-user    6u  IPv4             114730       0t0      TCP *:27005 (LISTEN)
mongod  58600 ec2-user    7u  unix 0xffff8800375e5e80       0t0   114731 /tmp/mongodb-27005.sock
mongod  58600 ec2-user    8uW  REG             202,16         6  9437186 /data/shard-27005/mongod.lock

So it looks like the primary shard (or mongos) just doesn't want to talk to it.

Generated at Thu Feb 08 03:50:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.