[SERVER-20783] aggregate $out + validator error: wrong error code Created: 06/Oct/15  Updated: 06/Dec/22  Resolved: 02/Aug/19

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: 3.1.9
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Hannes Magnusson Assignee: Backlog - Query Team (Inactive)
Resolution: Done Votes: 0
Labels: query-44-grooming, usability
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to DRIVERS-250 Support bypassDocumentValidation Closed
is related to SERVER-18227 Document Validation Closed
Assigned Teams:
Query
Operating System: ALL
Steps To Reproduce:

> db.runCommand({"create": "collectionName", "validator": {"fieldName": { "$gte": 1024 }}})
{ "ok" : 1 }
> db.runCommand({"create": "outCollectionName", "validator": {"fieldName": { "$gte": 8192 }}})
{ "ok" : 1 }
> db.runCommand({insert: "collectionName", "documents": [{"fieldName": 128}]})
{
	"ok" : 1,
	"n" : 0,
	"writeErrors" : [
		{
			"index" : 0,
			"code" : 121,
			"errmsg" : "Document failed validation"
		}
	]
}
> db.runCommand({insert: "collectionName", "documents": [{"fieldName": 2048}]})
{ "ok" : 1, "n" : 1 }
> db.runCommand({"aggregate": "collectionName", pipeline: [ {"$out": "outCollectionName"} ]})
{
	"ok" : 0,
	"errmsg" : "insert for $out failed: { connectionId: 1, err: \"Document failed validation\", code: 121, n: 0, ok: 1.0 }",
	"code" : 16996
}

> db.version()
3.1.9-pre-
> db.runCommand({buildinfo: 1})
{
	"version" : "3.1.9-pre-",
	"gitVersion" : "849505ebc395d15c6f777d96436447f1f98f7285",
	"modules" : [
		"enterprise"
	],
	"allocator" : "tcmalloc",
	"javascriptEngine" : "mozjs",
	"sysInfo" : "deprecated",
	"versionArray" : [
		3,
		1,
		9,
		-100
	],
	"openssl" : {
		"running" : "OpenSSL 1.0.1f 6 Jan 2014",
		"compiled" : "OpenSSL 1.0.1f 6 Jan 2014"
	},
	"buildEnvironment" : {
		"distmod" : "ubuntu1404",
		"distarch" : "x86_64",
		"cc" : "/opt/mongodbtoolchain/bin/gcc: gcc (GCC) 4.8.2",
		"ccflags" : "-fno-omit-frame-pointer -fPIC -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -Werror -O2 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-but-set-variable -Wno-missing-braces -fno-builtin-memcmp",
		"cxx" : "/opt/mongodbtoolchain/bin/g++: g++ (GCC) 4.8.2",
		"cxxflags" : "-Wnon-virtual-dtor -Woverloaded-virtual -std=c++11",
		"linkflags" : "-fPIC -pthread -Wl,-z,now -rdynamic -fuse-ld=gold -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,-E",
		"target_arch" : "x86_64",
		"target_os" : "linux"
	},
	"bits" : 64,
	"debug" : false,
	"maxBsonObjectSize" : 16777216,
	"storageEngines" : [
		"devnull",
		"inMemoryExperiment",
		"mmapv1",
		"wiredTiger"
	],
	"ok" : 1
}

Participants:

 Description   

The errorcode returned from validation failures is generally `121`.

In the case of write failure by `aggregate $out` however it returns a generic failure code, and the error message contains a long bson-loooking string, with the correct error code – but not something drivers can be expected to programmatically deal with.



 Comments   
Comment by David Storch [ 02/Aug/19 ]

It appears that this problem has been fixed:

MongoDB Enterprise > db.runCommand({"aggregate": "collectionName", pipeline: [ {"$out": "outCollectionName"} ], cursor: {}})
{
	"ok" : 0,
	"errmsg" : "Document failed validation",
	"code" : 121,
	"codeName" : "DocumentValidationFailure"
}

Closing as Gone Away.

Comment by Andy Schwerin [ 06/Oct/15 ]

Hannes has a good point, though the write commands error format as it exists today may not be appropriate for $out, where there might be millions of inserts and so, many errors to report.

Comment by Hannes Magnusson [ 06/Oct/15 ]

Ideally $out errors would be the same as normal write-command errors, not custom invention.
Then any existing programmatic approach to resolving these errors can be used (such as DuplicateKeyErrors, writeConcern, validators, ...).

Comment by Charlie Swanson [ 06/Oct/15 ]

So you are proposing that if there is an error inserting into the $out collection, to always propagate that error code, getting rid of error code 16996? This seems reasonable to me, though I'm not sure what all the consequences would be for doing so.

If I understand correctly, it is not possible to have continue-on-failure semantics during $out's insertions, so there will be at most one error/error code. However, I am not sure if there will be at least one error code... That comes from the getLastError.

cc redbeard0531

Generated at Thu Feb 08 03:55:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.