[SERVER-14255] Allow storing aggregation pipelines as documents in MongoDB Created: 14/Jun/14  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Aggregation Framework, Usability
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Thomas Rueckstiess Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-30575 Please add escaping convention for do... Backlog
is related to SERVER-8436 Aggregation pipeline cannot process s... Backlog
Assigned Teams:
Query Optimization
Participants:
Case:

 Description   

Currently it's not possible to store MongoDB's own aggregation pipelines due to the limitation that field names cannot start with a "$" symbol, and cannot contain a "." symbol either.

For MongoDB to become the one-stop-shop for data storage, aggregation, massaging this is an unnecessary limitation that we should fix.

Usage example: I want to build an app using custom / reusable aggregation pipelines that I can mix and match to get different views into my dataset.

My current workaround is to replace all "$" symbols with "#" in MongoDB and recursively replace them back to "$" on the client. This is clumsy and expensive. I have not yet needed dot-notation fields.

Two suggestions:
a) either lift the limitations of storing field names with $-prefix and dots (could be escaped to not confuse with operators ($set, $push, etc), like

db.foo.insert({
    "name": "my custom aggregation", 
    pipeline: [ {"\$match": {foo\.bar: 1} }, {"\$group": {_id: "bar", ...} } ],
});

b) or offer an alternative aggregation reserved symbol (# instead of $). For this alternative, we could allow both for the time being to not break backwards compatibility. This doesn't solve the dotted field name problem however.



 Comments   
Comment by Thomas Rueckstiess [ 14/Jun/14 ]

Also notice that the profiler is allowed to store $-prefixed keys in system.profile from aggregation commands. See SERVER-8436.

Generated at Thu Feb 08 03:34:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.