[GODRIVER-2306] Adding indexes to fields using StructTags Created: 11/Feb/22  Updated: 02/Jun/22  Resolved: 22/Feb/22

Status: Closed
Project: Go Driver
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Unknown
Reporter: Shaurya Agarwal Assignee: Benji Rewis (Inactive)
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Defining indexes or unique fields in MongoDB in Golang requires one to create it manually for each field. It takes the index definition away from the main model definition.

I would like to propose a method to create indexes for each field using the StructTags

type User struct {
	Id       primitive.ObjectID
	Email    string             `mongo:"unique"`
}

I created something similar for this here: Migrate() function
The above-mentioned method is definitely not perfect, since I created it for one-time use.

The working on the method can be tested here: ShauryaAg/ProductAPI

I would love to work on this.



 Comments   
Comment by Benji Rewis (Inactive) [ 24/Feb/22 ]

From what I can tell, the Javascript (Node) driver does not have any extra functionality in its CreateCollectionOptions as compared to the Go driver. Validator is also a document (interface{}) that can be specified in CreateCollectionOptions in the Go driver. Maybe I’m not understanding what you’re referring to, but minimum and maximum fields in the Validator document can certainly be used in the Go driver and are distinct from the Min and Max fields in IndexOptions. The former specify ranges for particular fields as part of document validation; the latter specify inclusive boundaries for longitude and latitude values of 2D indexes. Validator, as you mentioned, does not (in both the Go and Node drivers) allow the setting of index options like Min, Max, Unique or Sparse.

I do understand the desire to have a combined method for collection creation and index creation. However, we do try to minimize our number of administrative helpers, and a function like CreateCollectionWithIndexes or CreateCollectionsFromModels seems to be easily replaced with separate calls to CreateCollection and IndexView#CreateMany.

Comment by Shaurya Agarwal [ 22/Feb/22 ]

Hi Benji, 

Whilst I agree with some of the things you said, the main goal is to keep the model definition and index definition in one place.

The Javascript driver for MongoDB allows one to specify minimum, maximum under the validator. Reference.. Although we can't specify unique: true, or sparse: true, here as well, I believe this is something that can be added quite easily.

The ultimate vision I had for a function such as Migrate (or CreateCollectionsFromModels) is something that can help specify all the values under IndexOptions (including minimum and maximum such as in the Javascript driver) along with the schema itself.

Comment by Benji Rewis (Inactive) [ 22/Feb/22 ]

Hello again, agarwal.lucky.shaurya11@gmail.com! Thanks for all the clarification. It sounds like our hypothetical CreateCollectionFromModels method serves two purposes: it creates collections based on the names of the models structs, and it creates indexes on those collections based on the existence of Go struct tags. It looks like the other fields in the model structs are ignored. While I think that method could be helpful in a particular use-case, I don't think the driver is the right place for it.

Firstly, the proposed CreateCollectionFromModels method is fairly targeted at a specific use-case. A fully-featured migration API that allows modifications to an existing schema (removal and addition of fields/indexes) would be more generally usable, but would be significantly more complex. We adhere to a cross-drivers mantra to minimize the number and complexity of administrative helpers (like CreateCollectionsFromModels) in our driver. The best place for this new method may be outside of the driver.

Secondly, adding a method like CreateCollectionsFromModels would make the Go driver deviant from other drivers, as that method is not defined in our cross-drivers specifications. While we could make a change to add this method across drivers, the concept of using struct tags to specify desired indexes is distinct to Go, and may be difficult to replicate in other drivers.

Given these reasons, I’m closing this ticket as “Won’t Do” for now, but let me know if I can be of help in any other way.

Comment by Shaurya Agarwal [ 17/Feb/22 ]

Hi Benji,
While MongoDB does not need the users to pre-define the schema for each model, in my opinion, in case someone is defining an index on a field, they'd have some idea of what the data would look like in the given collection and they'd be having some pre-defined fields defined somewhere. We can still do some research on this part. However, using StructTags would make it easier for them to create the indexes, and keep the complete model definition and index definition in one place.

I believe a new method such as the one you mentioned CreateCollectionsFromModels(models ...interface{}) would work absolutely fine. The function could be called after connecting with the database.

Something of the order

package db
 
func init() {
    ......
    client, _ := mongo.Connect(ctx, options.Client().ApplyURI("connectionUri"))
    db := client.Database("db")
    db.CreateCollectionsFromModels(models...)
    .....
}

Comment by Benji Rewis (Inactive) [ 17/Feb/22 ]

Thanks again for your improvement suggestion, agarwal.lucky.shaurya11@gmail.com . I took a look at your ProductAPI project and the associated Migrate function. Creating collections by passing in “model” structs (like User in your example above) that are struct-tagged with “mongo” to represent desired indexes and index options like Unique and Sparse is a very cool idea. The Go driver API does not natively allow the creation of collections with model structs as you’re describing.

While an improvement like the one you’re suggesting could be interesting, two things are unclear to me. Firstly, I am unsure how common your use-case is; many users use MongoDB because there’s no need to define a schema for each collection a priori. Specifically, BSON’s flexible document schema usually means you don’t need to know exactly what data will look like in your collection beforehand, so a function like Migrate might not be that common. Secondly, I don’t think we have any existing API that could be altered to automatically create indexes from struct tags, as there is no function/method akin to Migrate in the Go driver. Were we to add a method on Database with a function-signature like CreateCollectionsFromModels(models… []interface), I believe we’d have to do so as part of a cross-drivers change.

I’m curious what you were imagining for the Go driver: a new method on Database? An alteration to an existing method?

Comment by Benji Rewis (Inactive) [ 16/Feb/22 ]

Thanks for your improvement suggestion, agarwal.lucky.shaurya11@gmail.com! I'm taking a look now.

Generated at Thu Feb 08 08:38:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.