[SERVER-27440] Add allowDiskUse option to createView Created: 15/Dec/16  Updated: 01/Jun/23  Resolved: 10/Apr/19

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 3.4.0
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Rafael Caballero [X] Assignee: Backlog - Query Team (Inactive)
Resolution: Won't Fix Votes: 2
Labels: createView, read-only-views, storch
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-40456 Add allowDiskUse option to createView Closed
Related
related to SERVER-36410 View definitions should permit specif... Closed
Assigned Teams:
Query
Participants:
Case:

 Description   

In the mongo aggregation framework is possible to use the option

{allowDiskUse:true}

. This is really useful when some heavy operations such as sorting, which cannot be performed in memory.

I'm trying to do the same with createView (available in Mongo 3.4), but it seems that this possibility is currently unavailable. For instance,

db.mydb.aggregate([....,{$sort:{"a":-1}}],{allowDiskUse:true})

works, but:

db.createView("newview","mydb",[....,{$sort:{"a":-1}}],{allowDiskUse:true})

produces the error

The field 'allowDiskUse' is not a valid collection option.

Of course, I can just remove

{allowDiskUse:true}

. Then the view is created, but when I try:

> db.newview.find()
Error: error: {
    "ok" : 0,
    "errmsg" : "Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.",
    "code" : 16819,
    "codeName" : "Location16819"
}

Thus I would suggest to add the option allowDiskUse to createView.



 Comments   
Comment by Yasmany Cubela Medina [ 27/Mar/19 ]

I recently cross the same limitation and besides i understand above comment i also think it will bring more benefits allowing such option. Answering to @kyle on above comment, yes with current behaviour it's unclear, but if the option is allowed, then any of the following could happen:

  • disable disk usage per first view restriction
  • allow it using always latest specification

given that you guys develop mongodb im pretty sure you can find better solutions, the same way allowing to create views which process pipelines more efficiently than any driver from any other language will increase performance.

I really hope you guys reconsider this

Comment by Kyle Suarez [ 20/Dec/16 ]

Hello RafaelCaballero, I see your point that it's strange that find, count and distinct queries on a view can fail if that particular operation happens to exceed the memory limit. However, I think adding an allowDiskUse option would lead to more confusing behavior. For example, consider views built on views with conflicting disk use options:

> db.runCommand({ create: "viewForbidsDiskUse", viewOn: "other", allowDiskUse: false });
> db.runCommand({ create: "viewAllowsDiskUse", viewOn: "viewForbidsDiskUse", allowDiskUse: true });

In this case, it's unclear how the server should act. In general, it's undesirable to have operations that can implicitly spill to disk. Instead, such an option should be explicitly specified on a per-operation basis. As such, I'm closing this ticket as Won't Fix.

Please note that you can work around this limitation by expressing the operation against the view as an aggregation and specifying the allowDiskUse option; for example, instead of

> db.view.find();

you can instead try

> db.view.aggregate([], { allowDiskUse: true });

Regards,
Kyle

Generated at Thu Feb 08 04:15:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.