[SERVER-1014] A modifier to delete a single value from an array Created: 13/Apr/10  Updated: 06/Apr/23  Resolved: 29/Jun/19

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor - P4
Reporter: Wouter Assignee: Backlog - Query Team (Inactive)
Resolution: Won't Do Votes: 119
Labels: pull-request, storch, syntax
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File update.patch    
Issue Links:
Duplicate
is duplicated by SERVER-4708 "{ $pull : { field : _value } }" supp... Closed
is duplicated by SERVER-2036 $insert and $remove for arrays (curre... Closed
Related
related to SERVER-6566 Support conditional updates: $updates Closed
is related to SERVER-1824 Support for inserting into specific a... Closed
is related to SERVER-6399 Refactor update() code Closed
Assigned Teams:
Query
Participants:

 Description   

When dealing with arrays with non unique values, it is impossible to remove a single value from that array in 1 update.

$pop doesn't work, because the value can be anywhere in the array.
$pull doesn't work, because that will remove all occurences of the value (which can happen in a non unique array)
$unset doesn't work, because that sets the value to NULL, instead of removing it.

EG:

> db.example.remove()
> db.example.insert({_id:1, sequence : [1,2,3,4,3,2,3,4]})
> db.example.update({_id:1}, { $unset : { 'sequence.1' : 1 }})
> db.example.find()

{ "_id" : 1, "sequence" : [ 1, null, 3, 4, 3, 2, 3, 4 ] }

I'm looking for a modifier that leaves me with

{ "_id" : 1, "sequence" : [ 1, 3, 4, 3, 2, 3, 4 ] }

Maybe something like

> db.example.update({_id:1}, { $remove : { 'sequence.1' : 1 }})

I'm curious what you think of it. Currently, with $addToSet and $pull it's very easy to manage arrays with unique values. For non unique arrays, there's $push to add values, but no proper way to remove values.



 Comments   
Comment by Ivan Cherviakov [ 15/Feb/20 ]

Well, would be there example how to use microscope to bash nails there it is. But since it is really working, is it possible to add some alias operator, which under the hood runs this monstrosity? As essentially we only need to pass name of array field and index. Maybe something like
db.example.update({}, { $removeArrayElByIndex: ['sequence', index] });

Comment by Asya Kamsky [ 29/Jun/19 ]

SERVER-40381 implemented support for aggregation expressions to specify update for 4.2. 

 You can see some examples here.

This can be handled a couple of different ways in aggregation.  One way would be to determine position of the array element that you want to remove and use $slice and $concatArrays.   Note that if the position is already known then it's a simpler version of the same thing.

Position known (P)

db.example.update({}, [
     {$set:{ sequence: {
                   $concatArrays:[ 
                       {$slice:[ "$sequence", P ]}, 
                       {$slice:[ "$sequence", {$add:[1,P]}, {$size:"$sequence"}]}
                    ]
     }}}
]);
 

Position not known (first occurrence that matches a condition:

db.example.update({}, [
     {$set:{ sequence: {
         $let:{ vars: {p: {$indexOfArray:["$sequence", 2] } },
                    in:{$concatArrays:[ 
                       {$slice:[ "$sequence", "$$p" ]}, 
                       {$slice:[ "$sequence", {$add:[1,"$$p"]}, {$size:"$sequence"}]}
                    ]
     }}}
]);
 

I've simplified slightly you would need to check that p is not -1 to avoid getting the wrong result (when there is no match).

Comment by Ethan Farbiash [ 09/Jun/19 ]

Hi, is there any update on this issue ? It's been 9 years since it was opened, and almost 6 months since the last update. The workarounds you mention are not only less efficient, but guaranteeing their correctness is cumbersome and leads to a lot of code no one can later figure out. 

This seems really intuitive and I would think that 9 years were quite enough to make a decision regarding syntax. As you can see by the comments, this is a burning issue for a lot of people. Could you please consider raising its priority ?

Thanks in advance,

Ethan

Comment by Asya Kamsky [ 04/Jan/19 ]

Note that there are a couple of (less efficient) workarounds for this in current versions. They involve more than one operation but correctness can be guaranteed, just not as efficient as a single update.

Comment by Asya Kamsky [ 04/Jan/19 ]

nachiketg see my comment above - we are still working on possible solution to make this possible for the next major release.

Comment by Nachiket Goswami [ 03/Jan/19 ]

Can we please get a timeline for this? It will be very much appreciated!

Comment by Desmond [ 22/Oct/18 ]

This messes up any effort to work with embedded documents, you should guarantee if you care about your product.

Comment by Asya Kamsky [ 09/Oct/18 ]

We are considering possible syntax to alleviate this issue for the next major release. No guarantees, of course, but we are working on it.

Comment by Nachiket Goswami [ 17/Sep/18 ]

It is highly desirable to have this implemented ($remove) which is indicative from the comments above as well.

Can we know if this is in the plan to be implemented? If yes, I am interested in knowing if we have a timeline.

Comment by Chris Fischer [ 01/Jul/18 ]

Since $unset almost works (leaves a null in the array element), how about something like:

 { $unset: { "sequence.3": { $remove: 1 } } }

Comment by Riccardo Cardin [ 31/Oct/17 ]

Until a native solution will be implemented in Mongo, you can evaluate a workaround that I developed: https://stackoverflow.com/a/47036857/1173755

Comment by Gonzalo J Sambucaro [ 12/Sep/17 ]

$pop with $position is needed!

Comment by Zoe Carver [ 05/Jul/17 ]

Is there any way to post this on something like Bountysource? Because I would be willing to pay money to have this fixed.

Comment by Scott Lowe [ 02/Dec/16 ]

Very frustrating that after more than SIX YEARS this has still not been addressed. There is no way to atomically remove data from an array at a particular position. How is this considered minor?

Comment by david zhang [ 26/Aug/16 ]

guys please, we need this very much, this issue won't take you more than three minutes.

need this.

Comment by 4F2E4A2E [X] [ 24/Aug/16 ]

Just got informed on this issues, which goes back to 2010!? by reading "Meteor in Action".
I hope it gets fixed soon. Meteor ftwwWWWwwwwwWWWWWwwwww!

Comment by Jean-Philippe Pellet [ 04/Apr/16 ]

I'd vote for $pop with $position, too, would make a lot of sense. It would also give drivers a greater flexibility to provide higher-level functions—e.g. given two arrays of objects, generate an update description which would convert one into the other.

Comment by Ioannis Chouklis [ 29/Mar/16 ]

It's been almost 6 years. C'mon devs, we need this

Comment by Carl Banbury [ 09/Sep/15 ]

Woah! I thought that comment was aimed at me. Had totally forgotten about the 'shut the fuck up carl!' thing.

Fair point though.

Comment by hello kitty [ 08/Jun/15 ]

$pop with $position is good. could someone please implement it?

Comment by Ranno [ 18/Apr/15 ]

Would really like to see this implemented in a near release.

Comment by Joshua Sullivan [ 02/Sep/14 ]

I really want a $pullOne operator. Just pass it an object to match, like $pull, but it will only remove one match and stop. Simple, elegant, and does what I need without tracking indexes which isn't going to be autonomous.

Comment by Meruem Pitou [X] [ 12/Jul/14 ]

I just don't understand how this feature is not implemented. Ex: It is not necessary (or recommended) to use "Id" fields on arrays, but how else can you uniquely identify an array element if not by it's index.

Comment by Albert Engelbrecht [ 11/Jun/14 ]

+1

TIL it's not possible to remove an array based on its index.

Yes, there's a possible race condition by doing it like that. But it's a pretty basic use-case for MongoDB (especially in use-cases where a race condition would have no impact).

Comment by Stephane [ 05/Jan/14 ]

hello, for me too this is a big hole !

Comment by James Vincent [ 09/Dec/13 ]

@Eliot and update on @Jiao's pull request (if he ever made one?). Would love to see this implemented, this is a big hole in MongoDB for me.

Comment by Joao Paulo Farias [ 02/May/12 ]

A sample usage would be like:

$ mongo localhost/test
MongoDB shell version: 2.0.4
connecting to: localhost/test
> db.test.insert(

{x: 1, y: [1, 2, 3, 4, 5, 6]}

)
> db.test.update(

{x: 1}

, {$pop: {y: -2}})
> db.test.findOne()
{
"_id" : ObjectId("4fa0a66b6b27d2c6bac971a2"),
"x" : 1,
"y" : [
1,
3,
4,
5,
6
]
}
> db.test.update(

{x: 1}

, {$pop: {y: 2}})
> db.test.findOne()
{
"_id" : ObjectId("4fa0a66b6b27d2c6bac971a2"),
"x" : 1,
"y" : [
1,
3,
4,
6
]
}

As Wouter suggested on a previous comment. It was quite easy to implement this, perhaps it was not the best way but sounds like a good idea to have a "remove from array" operation that can remove with an index based parameter. Actually, I was surprised that such a feature didn't exist already.

Comment by Eliot Horowitz (Inactive) [ 02/May/12 ]

@joao - not sure this makes sense as is.
Can you add some tests and a spec so we can see exactly what you intend?

Comment by Joao Paulo Farias [ 01/May/12 ]

I just sent a pull request on github and accepted the contributor terms. Thanks, --JP

Comment by Ian Whalen (Inactive) [ 01/May/12 ]

@Joao, when possible we handle all patch contributions via Github pull requests. It would be great if you could open up a pull request against the primary repo at https://github.com/mongodb/mongo, and also read + sign the contributor agreement at http://www.10gen.com/contributor. Once you've done that I'll be able to get an engineer to start reviewing your patch.

Comment by Joao Paulo Farias [ 01/May/12 ]

This patch was written agains r2.0.4 and allows pop to remove an element by its index.

To keep backward compatibility, I made it so -N removes from beginning and N removes from the end.

In case the absolute value of N is greater than the array size, no elements are removed.

For instance, $pop with -1 removes the last element, unless the array is empty, in which case nothing is changed.

This is a pretty simple patch and I hope to see that in the next realeases of mongo.

Comment by Wouter [ 15/Apr/10 ]

> If $unset did not leave nulls then the array index's would change

Therefore a separate modifier?!

> db.example.update({_id:1}, { $remove : { 'sequence.1' : 1 }})

Or modify $pop, so that it takes an index

> db.example.update({_id:1}, { $pop : { 'sequence' : 4 }})

Or to remain backwards compatibility

> db.example.update({_id:1}, { $pop : { 'sequence' : N }})
with:
N > 0 --> N'th value starting from end of set
N < 0 --> N'th value starting from beginning of set

Comment by Coen Hyde [ 15/Apr/10 ]

Maybe just engineer your application to gracefully deal with null's. I initially didn't like the fact that $unset left nulls but there is a reason ... If $unset did not leave nulls then the array index's would change. This may sound good, but if you have a lot of concurrent update queries going on, an update request may accidentally update the wrong array element. Well i assume this is the reason why nulls are left. Plus it's probably quicker to just leave null's in place,.

Generated at Thu Feb 08 02:55:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.