[SERVER-16735] capped document fields Created: 06/Jan/15 Updated: 24/Jan/15 Resolved: 09/Jan/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | GridFS, Sharding, Storage |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | guipulsar | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Participants: | |||||||||
| Description |
|
Hi, So thats actually the same logic of capped collectio but trashed data come oly from one field in each documents |
| Comments |
| Comment by Stennie Steneker (Inactive) [ 09/Jan/15 ] | |||||||||||||||||||||||||
|
Hi pulsar, As I noted on your followup issue Fields with unbounded growth are a definite performance and scaling anti-pattern (see: Why shouldn't I embed large arrays in my documents?) and raising the document limit or trying to cap the array size are definitely more hacky solutions than addressing the underlying schema design. I look forward to your post in mongodb-users so we can work out appropriate solutions. Thanks, | |||||||||||||||||||||||||
| Comment by guipulsar [ 08/Jan/15 ] | |||||||||||||||||||||||||
|
In addition i would say thats this discution http://blog.mongolab.com/2013/04/thinking-about-arrays-in-mongodb/ | |||||||||||||||||||||||||
| Comment by guipulsar [ 08/Jan/15 ] | |||||||||||||||||||||||||
|
Thanks i understand ur point but i don't understand ur point I 'm deep thinking about that but finaly the only others alternatives i found were not acceptable ; Problem is , this kind of shema simply doesn't work in my case , lets say you want to retrieve all profiles who are in ur blacklist.. 1) first you get all _id ref in blacklist collection This kind of query Really if i miss something say me , because i had lots of lecture and this kind of 2 query or multi query step is pretty hidden/unclear and not enought documented from my point of vue. | |||||||||||||||||||||||||
| Comment by Ramon Fernandez Marina [ 07/Jan/15 ] | |||||||||||||||||||||||||
|
pulsar, the usage of arrays described above is far from ideal schema design, and it will give you very bad performance in the long run. Allowing documents to grow past 16MB to accommodate this usage will only make things worse, so I'd strongly encourage you to revisit your schema design if you want to avoid performance problems down the line. I'd recommend you consider using a capped collection to hold the blacklist information, and maybe even a capped collection per _id. For further discussion in schema design pleabse post on the mongodb-user group or Stack Overflow with the mongodb tag, where your question will reach a larger audience. Adding this feature to the server may encourage the use of sub-optimal schemas, so even if it seems useful now it may not be very attractive when performance degrades later on. | |||||||||||||||||||||||||
| Comment by guipulsar [ 07/Jan/15 ] | |||||||||||||||||||||||||
|
thanks for this other tricks, not a really solution be cause the solution should be this feature request itself | |||||||||||||||||||||||||
| Comment by Ramon Fernandez Marina [ 06/Jan/15 ] | |||||||||||||||||||||||||
|
I believe this could be best handled at the application level, as I think the use case is very specific. In the scenario described in this ticket, one could try updating the document, and if the update fails then use $pop repeatedly until the update succeeds; in python-like pseudocode:
pulsar, would this meet your needs? | |||||||||||||||||||||||||
| Comment by guipulsar [ 06/Jan/15 ] | |||||||||||||||||||||||||
|
Thanks , i understoud ur tips now but it's more or less a hack rather than a solution , ur query say keep last 1000 items but thats not do the job.. I want to keep the maximum of items and i don't know what and when there will not enought space for contain thoses items. My feature request make sens for me | |||||||||||||||||||||||||
| Comment by Eliot Horowitz (Inactive) [ 06/Jan/15 ] | |||||||||||||||||||||||||
|
How do you add items to the blacklist field?
You could instead do
Which will only keep the newest 1000 entries in that field. | |||||||||||||||||||||||||
| Comment by guipulsar [ 06/Jan/15 ] | |||||||||||||||||||||||||
|
with capped field i mean , i don't care to loose some olders blacklist field datas but i cannot loose any other data. | |||||||||||||||||||||||||
| Comment by guipulsar [ 06/Jan/15 ] | |||||||||||||||||||||||||
|
So my english is very poor i guess.. Here is my structure, the blaklist field size increaze day after days.. In 6 month i would happy to have an autodelete (capped field ) on this field , so new entryes will replace the olders, if not the 16mb limit will cause trouble.. i have thousands item in each blacklist field..
| |||||||||||||||||||||||||
| Comment by Eliot Horowitz (Inactive) [ 06/Jan/15 ] | |||||||||||||||||||||||||
|
I don't think I understand what you're trying to do. | |||||||||||||||||||||||||
| Comment by guipulsar [ 06/Jan/15 ] | |||||||||||||||||||||||||
|
heu seems i made mistake , the limit is 2MO infact.. so my need for this feature is bigger now ..lol | |||||||||||||||||||||||||
| Comment by guipulsar [ 06/Jan/15 ] | |||||||||||||||||||||||||
|
not really, perhaps some kind of hack i'm not sure.. My need is simple, i have only one field "temp" in each document of my collection who can increase to the 16mo limit in future, i'd like to don't use grid , and i'd like to see the "temp" field autotrash (autodelete) and replace new entry by old when 16mo limit collection is reatched; | |||||||||||||||||||||||||
| Comment by Eliot Horowitz (Inactive) [ 06/Jan/15 ] | |||||||||||||||||||||||||
|
I think $push with $slice is what you want. |