[SERVER-45233] Indexed inequalities to arrays return incorrect results Created: 18/Dec/19  Updated: 29/Oct/23  Resolved: 19/Feb/20

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 4.2.2, 4.0.14
Fix Version/s: 4.3.4, 4.2.9, 4.0.21

Type: Bug Priority: Major - P3
Reporter: Ian Boros Assignee: Ted Tuckman
Resolution: Fixed Votes: 0
Labels: afz, qopt-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Duplicate
is duplicated by SERVER-26655 $gt operation on array with index Closed
Related
related to SERVER-49766 Indexed and non-indexed collections r... Closed
related to SERVER-47382 Enable $not with arrays Closed
Backwards Compatibility: Major Change
Operating System: ALL
Backport Requested:
v4.2, v4.0, v3.6
Sprint: Query 2020-01-27, Query 2020-02-10, Query 2020-02-24
Participants:
Linked BF Score: 17

 Description   

> db.c.insert({a: [1, 2]})
WriteResult({ "nInserted" : 1 })
> db.c.find({a: {$lte: [0, 0]}})
> db.c.find({a: {$lte: [2, 2]}})
{ "_id" : ObjectId("5dfa766b191cd168059aaf54"), "a" : [ 1, 2 ] }
> db.c.createIndex({a: 1})
{
	"createdCollectionAutomatically" : false,
	"numIndexesBefore" : 1,
	"numIndexesAfter" : 2,
	"ok" : 1
}
> db.c.find({a: {$lte: [2, 2]}})
> 

The bounds we build for this look wrong:

{
	"queryPlanner" : {
		"plannerVersion" : 1,
		"namespace" : "test.c",
		"indexFilterSet" : false,
		"parsedQuery" : {
			"a" : {
				"$lte" : [
					2,
					2
				]
			}
		},
		"queryHash" : "811451DD",
		"planCacheKey" : "B859B02A",
		"winningPlan" : {
			"stage" : "FETCH",
			"filter" : {
				"a" : {
					"$lte" : [
						2,
						2
					]
				}
			},
			"inputStage" : {
				"stage" : "IXSCAN",
				"keyPattern" : {
					"a" : 1
				},
				"indexName" : "a_1",
				"isMultiKey" : true,
				"multiKeyPaths" : {
					"a" : [
						"a"
					]
				},
				"isUnique" : false,
				"isSparse" : false,
				"isPartial" : false,
				"indexVersion" : 2,
				"direction" : "forward",
				"indexBounds" : {
					"a" : [
						"[[], [ 2.0, 2.0 ]]"
					]
				}
			}
		},
		"rejectedPlans" : [ ]
	},
	"serverInfo" : {
		"host" : "borosaurus",
		"port" : 30000,
		"version" : "4.2.1",
		"gitVersion" : "edf6d45851c0b9ee15548f0f847df141764a317e"
	},
	"ok" : 1
}

It only includes arrays! Since we index arrays element-wise, this will miss results.



 Comments   
Comment by Githook User [ 14/Aug/20 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-45233 Include the empty array in the comparison with index test

(cherry picked from commit 0630338fb592b3283240ba72ac4b0c7111f6193c)
(cherry picked from commit 0d8d980ae814f5b9f55cd9bea4d4bef84a9dca6a)
Branch: v4.0
https://github.com/mongodb/mongo/commit/11981cbf54adad061d688b44689e65ea4e814c86

Comment by Githook User [ 14/Aug/20 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-45233 Indexed array inequality fixed to use correct bounds

create mode 100644 jstests/core/array_comparison_correctness.js
create mode 100644 jstests/core/array_index_and_nonIndex_consistent.js

(cherry picked from commit 52e7950cae37ef3783b05840c04abbbe383fa1ff)
(cherry picked from commit e3fba401e6e75b26cc2e5d363ad331dff749cfbf)
Branch: v4.0
https://github.com/mongodb/mongo/commit/65864da5eb9885dae432fa8a1b9b46f31be04b07

Comment by Githook User [ 15/Jul/20 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-45233 Include the empty array in the comparison with index test

(cherry picked from commit 0630338fb592b3283240ba72ac4b0c7111f6193c)
Branch: v4.2
https://github.com/mongodb/mongo/commit/e9bdaa0807b6aa6e3054efc8598030e28fb9c880

Comment by Githook User [ 15/Jul/20 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-45233 Indexed array inequality fixed to use correct bounds

create mode 100644 jstests/core/array_comparison_correctness.js
create mode 100644 jstests/core/array_index_and_nonIndex_consistent.js

(cherry picked from commit 52e7950cae37ef3783b05840c04abbbe383fa1ff)
Branch: v4.2
https://github.com/mongodb/mongo/commit/1f89ab198058cd5ed012acd2b081f0e8c6949927

Comment by Ted Tuckman [ 19/Feb/20 ]

This changes query results on a multikey index. Where before query results would be wrong and missing most documents from the query without the index, the index should now report the same documents.

Comment by Githook User [ 19/Feb/20 ]

Author:

{'name': 'Ted Tuckman', 'username': 'TedTuckman', 'email': 'ted.tuckman@mongodb.com'}

Message: SERVER-45233 Include the empty array in the comparison with index test
Branch: master
https://github.com/mongodb/mongo/commit/0630338fb592b3283240ba72ac4b0c7111f6193c

Comment by Githook User [ 04/Feb/20 ]

Author:

{'username': 'TedTuckman', 'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com'}

Message: SERVER-45233 Indexed array inequality fixed to use correct bounds

create mode 100644 jstests/core/array_comparison_correctness.js
create mode 100644 jstests/core/array_index_and_nonIndex_consistent.js
Branch: master
https://github.com/mongodb/mongo/commit/52e7950cae37ef3783b05840c04abbbe383fa1ff

Comment by Githook User [ 04/Feb/20 ]

Author:

{'name': 'Ted Tuckman', 'username': 'TedTuckman', 'email': 'TedTuckman@users.noreply.github.com'}

Message: SERVER-45233 Blacklist GT/LT/GTE/LTE comparisons against arrays (#303)
Branch: master
https://github.com/10gen/jstestfuzz/commit/6ca51f50a2ec8c0acbc62813b5e91601fb3ae191

Comment by Ian Boros [ 18/Dec/19 ]

charlie.swanson Jacob and I were discussing SERVER-45152 which lead me to read the bounds building code for inequalities to arrays.

Comment by Charlie Swanson [ 18/Dec/19 ]

ian.boros how did you notice this? jacob.evans this seems related to or at least in the same area as SERVER-45152

Generated at Thu Feb 08 05:08:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.