[SERVER-14912] Need alternative upsert for time series collection Created: 15/Aug/14  Updated: 15/Aug/14  Resolved: 15/Aug/14

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 2.4.10
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Lucille Wilson Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-10711 $setOnInsert mods should not conflict... Closed
Operating System: ALL
Participants:

 Description   

I would like to either insert or update a record in a time series collection and do this in one pass with a update() command with the $setOnInsert and $inc commands.

I would like to avoid querying to find out if the record exists and then doing an insert if it does not exist or an update increment if the record does exist.

For example: This does not work:

db.EventRecord.update(
{ "appKey" : "apRiBsgUCm" , "name" : "ApplicationOpened" , "type" : "application" },
{ 
  "$setOnInsert" : { "appKey" : "apRiBsgUCm" ,  "type" : "application" , "name" : "ApplicationOpened" , "totalEvents" : 0 ,
                   "hours" : {"18" : { "totalHr" : 0 , "0" : 0 , "5" : 0 , "10" : 0 , "15" : 0 , "20" : 0 , "25" : 0 , "30" : 0 , "35" : 0 , "40" : 0 , "45" : 0 , "50" : 0 , "55" : 0 } } },
  "$inc" : { "totalEvents" : 1 , "hours.18.0" : 1 ,    "hours.18.totalHr" : 1}
},
{"upsert":true, "multi":false}
)
// get error: Field name duplication not allowed with modifiers

But this does.

db.find({ "appKey" : "apRiBsgUCm" , "name" : "ApplicationOpened" , "type" : "application" }) 

Then if record exists:

 
db.EventRecord.update(
{ "appKey" : "apRiBsgUCm" , "name" : "ApplicationOpened" , "type" : "application" },
{ 
 
  "$inc" : { "totalEvents" : 1 , "hours.18.0" : 1 ,    "hours.18.totalHr" : 1}
},
{"upsert":true, "multi":false}
)

Else if the record doesn't exist, then do an insert.

If I do the query and then the if test, for every single event, I have to access the database twice - one to query the document to see if it exists and then again to insert or update the record.

What is a better strategy? How can I access the database once to upsert the document in this time series collection. This collection is being updated millions of times a day. I'd like to increase my performance.



 Comments   
Comment by Thomas Rueckstiess [ 15/Aug/14 ]

Hi Lucille,

For these kinds of questions I recommend using the mongodb-user group (http://groups.google.com/group/mongodb-user) or Stack Overflow with the mongodb tag. There you will have a large user base (including MongoDB employees) who may be able to recommend alternatives.

The SERVER project is for reporting bugs or feature suggestions for the MongoDB server and unfortunately we can't answer support questions like these here.

Kind Regards,
Thomas

Comment by Lucille Wilson [ 15/Aug/14 ]

I agree this is a duplicate. But I wasn't really complaining about the bug. I was trying to see what Mongo recommends to do under these circumstances? People are using timeseries collections which record massive amounts of events. How does Mongo get around this problem?

Comment by Thomas Rueckstiess [ 15/Aug/14 ]

Hi Lucille,

This is a known issue and is tracked under SERVER-10711. Currently the $setOnInsert conflicts with other (non-insert-only) operations when using the same fields. I'm closing this as duplicate, please feel free to vote for and watch SERVER-10711.

Regards,
Thomas

Generated at Thu Feb 08 03:36:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.