[SERVER-79282] Cannot group using searchMeta Created: 25/Jul/23  Updated: 09/Oct/23  Resolved: 09/Oct/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: ahmad al sharbaji Assignee: Noopur Gupta
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Server Triage
Participants:

 Description   

My documents are like this below:

 

{
    "_id": ObjectId("63bd85a644552d73f36c366e"),
    "ts": datetime.datetime(2023, 1, 10, 15, 35, 2, 374000),
    "dt": datetime.datetime(2023, 1, 10, 15, 34, 10),
    "aff": 2,
    "src": 2,
    "st": 2
}

 

I have more than 50M documents, and $searchMeta saved my life with its faceting speed! But, it's not complete. Please help me to achieve it.

I wanna count documents based on aff and src grouping. And ts and st as filters. And getting the maximum date for every group.

I developed this aggregation pipeline below:

 

[
    {
        "$searchMeta": {
            "index": "MsgAtlasIndex",
            "count": {"type": "total"},
            "facet": {
                "operator": {
                    "compound": {
                        "must": [
                            {
                                "range": {
                                    "path": "ts",
                                    "gte": datetime.now() - timedelta(hours=24)
                                }
                            },
                            {"equals": {"path": "st", "value": 2}},
                        ]
                    }
                },
                "facets": {
                    "src": {
                        "type": "number",
                        "path": "src",
                        "boundaries": [1, 2, 3, 4, 5, 6, 7, 10, 11]
                    },
                    "aff": {
                        "type": "number",
                        "path": "aff",
                        "boundaries": [1,2,3,4,5,7,8,9,10,11,12,14,17,18,19,20,21]
                    },
                },
            },
        }
    },
]

 

The output of this is like below:

[
    {
        "count": {"total": 35},
        "facet": {
            "aff": {
                "buckets": [
                    {"_id": 1, "count": 1},
                    {"_id": 2, "count": 34},
                    {"_id": 3, "count": 0},
                    {"_id": 4, "count": 0},
                    {"_id": 5, "count": 0},
                    {"_id": 7, "count": 0},
                    {"_id": 8, "count": 0},
                    {"_id": 9, "count": 0},
                    {"_id": 10, "count": 0},
                    {"_id": 11, "count": 0},
                    {"_id": 12, "count": 0},
                    {"_id": 14, "count": 0},
                    {"_id": 17, "count": 0},
                    {"_id": 18, "count": 0},
                    {"_id": 19, "count": 0},
                    {"_id": 20, "count": 0},
                ]
            },
            "src": {
                "buckets": [
                    {"_id": 1, "count": 0},
                    {"_id": 2, "count": 34},
                    {"_id": 3, "count": 0},
                    {"_id": 4, "count": 0},
                    {"_id": 5, "count": 0},
                    {"_id": 6, "count": 0},
                    {"_id": 7, "count": 1},
                    {"_id": 10, "count": 0},
                ]
            },
        },
    }
]

I don't want aff and src separated. I want them to be grouped.

Wanted result:

[
    {
        "_id": {"aff": 1, "src": 7},
        "count": 1,
        "last_message_datetime": datetime.datetime(2023, 7, 24, 13, 8, 3, 488000),
    },
    {
        "_id": {"aff": 2, "src": 2},
        "count": 34,
        "last_message_datetime": datetime.datetime(2023, 7, 24, 22, 25, 23),
    },
]

Please help me!



 Comments   
Comment by Noopur Gupta [ 09/Oct/23 ]

Hi Ahmad,

I see you have posted this question on the Developer Forum. I request you to follow up on the forum if you still need help with aggregation pipeline to achieve your desired results.

However, if there is a bug with the aggregation, feel free to re-open this ticket pointing us to the bug.

 

Thanks.

Generated at Thu Feb 08 06:40:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.