[SERVER-3536] templates for schemas Created: 04/Aug/11  Updated: 06/Dec/22  Resolved: 06/Mar/18

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Jason Voorhees Assignee: Backlog - Query Team (Inactive)
Resolution: Done Votes: 9
Labels: insert, query, schema, update
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-13949 Input/Update trigger to validate data Closed
is duplicated by SERVER-14481 Validating schema on the server side Closed
is duplicated by SERVER-17049 Allow $ref and $schema keywords to su... Closed
Related
related to SERVER-863 Tokenize the field names Closed
Assigned Teams:
Query
Participants:

 Description   

The user is said to gain more "flexibility" in a schema-less database. However the option to gain schema-based advantages are NOT available. Proposal:

  • Create document "templates" that list all fields and default values for a specific type of database write or read.
  • Create "subset templates" that reference a template and represent a subset of its information.
  • Documents written from a template are indicated as such, but lose this status on their first non-template write.
  • Compare templates at declaration time; writes on the same document from multiple templates preserve flags if those templates do not conflict.
  • Queries using a template return only documents written with a compatible template (original or subset)

This would provide guaranteed queries (skip those pesky membership tests) and assist developers in planning and maintaining a mongodb instance by simplifying and solidifying common and/or boilerplate data operations.



 Comments   
Comment by Asya Kamsky [ 06/Mar/18 ]

3.6 added support for JSON Schema both as query operator and schema validation syntax.

This seems to address many of the asks in this ticket - ability to only allow documents that match the schema, or optionally accepting ones that don't with a warning logged, plus ability to query for documents that match a particular schema.

I'm going to close this ticket - components of original request that aren't addressed by JSON Schema (providing default values for fields not set) should be tracked by separate smaller/more narrowly defined tickets (some already exist list SERVER-24430 for setting default).

Comment by Marcel Bennett [ 26/Jan/15 ]

As noted in SERVER-17049 (despite being marked as duplicate when it wasn't meant to appear as one): as a minimum support for JSON Schema the two $ prefixed fields $schema and $ref, from the JSON schema draft 3 and above would need to be added as valid keys requiring a fully qualified URI as a value. Also potentially being added in draft-5 is $data requiring a Relative JSON Pointer as a value (potentially there could be a sub-task created to provide an option to resolve these $data pointers when querying).

I'm not as concerned with these being part of this issue and would prefer the other issue re-opened, but this is needed simply to store JSON Schemas in Mongo at all, like for a Node application for managing Schema records.

Comment by Marcel Bennett [ 24/Jan/15 ]

I would rather see JSON Schema used with one non-standard element than something completely incompatible. My workplace is investing more and more in JSON Schema tools and re-implementing things like product catalogues using JSON Schema and we've never really used any non-standard JSON when using Mongo, so we could get away with fully standard JSON Schema, so if there's only a few core types to add, then please make JSON Schema an option!

Comment by Asya Kamsky [ 07/Nov/14 ]

Our documents are extended JSON since we store more types than JSON understands, so I'm not sure any standard will be able to handle it, unless it allows types that are more extensive than what JSON supports.

Allowing something like a query that any document going into the collection would have to satisfy seems like the most flexible yet covering all use cases (as long as you can specify if it's strict or minimal - i.e. are fields not in query document allowed or not allowed).

Comment by Daniel Coupal [ 07/Nov/14 ]

Closing SERVER-14481 as a duplicate, however adding the information that it would be nice to use a standard like JSON Schema validation to represent the constraints.

Comment by Vincent [ 06/Aug/14 ]

IMO https://jira.mongodb.org/browse/SERVER-863 is asking pretty much the same thing (duplicate?) in a more "NoSQL" way. It would be even easier to check if a field exist or not (is it present or not in the hashmap?).
Also, the feature requested by Hari would be very easy to implement (freeze automatic hashmap updates).
However those feature requests are 4 years old/2 years old, and I don't really think they will ever be implemented... even if they would lead to major improvements in term of memory consumption...

Comment by Thomas Rueckstiess [ 06/Aug/14 ]

With such schema templates, we could also look at performance improvements for certain edge cases. For example building a sparse index on a non-existing field would not have to iterate documents in a collection, see SERVER-14772. We could just add the empty index, after which the user could modify the schema template and allow future documents to contain the field.

Comment by Hari Dosapati [ 15/Apr/14 ]

This a big concern from the maintainance to adapt Mongo DB. In simple terms, we are looking for some thing like this

Would like to freeze the schema(collection) and no client should add any unknown property to the collection after that.

Comment by Eliot Horowitz (Inactive) [ 20/Apr/12 ]

Not sure we'll do exactly this, but definitely something with similar motivation.

Comment by Ben McCann [ 20/Apr/12 ]

Being able to optionally have a schema on some collections would be very nice.

Generated at Thu Feb 08 03:03:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.