-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
Context
We currently require a user to know and provide the dimension of an embedding model even when we create the vector index for them.
This can easily be inferred from the Embeddings model: n = len(embedding.embed_query("foo"))
Expected Logic:
(relevant arguments in constructor: auto_create_index, dimensions, embedding
if auto_create_index == True and dimensions == -1: automatically set the index and change dimensions to the provided length
if auto_create_index == False and dimensions == -1:
case 1. if NO index is found, throw an exception
case 2. if INDEX is found, ensure index's embedding size and the embedders size match up. If not, throw an error.
Definition of done
Make the change shown above; pass tests; document behavior; bump version and release.
Pitfalls
We need to consider versioning and compatibility. This would be changing the default so a breaking change.