Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-3372

Allow indexing fields of arbitrary length

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: Indexing
    • Labels:
      None

      Description

      The current index format has a maximum key length limitation and starting in 2.6, prevents inserting new documents that contain indexed fields exceeding this length.

        Issue Links

          Activity

          Hide
          dwight_10gen Dwight Merriman added a comment -

          would need to point to start of the document in the collection rather than its exact position as exact field positions move over time when updated

          Show
          dwight_10gen Dwight Merriman added a comment - would need to point to start of the document in the collection rather than its exact position as exact field positions move over time when updated
          Hide
          geoff.gallaway Geoffrey Gallaway added a comment -

          Is there some way to figure out how many documents in a collection have keys that will exceed the maximum index size? db.collection.find({$where: "this.something.length > 800"}) is very vague. Do indexes contain the column names, etc...

          Show
          geoff.gallaway Geoffrey Gallaway added a comment - Is there some way to figure out how many documents in a collection have keys that will exceed the maximum index size? db.collection.find({$where: "this.something.length > 800"}) is very vague. Do indexes contain the column names, etc...
          Hide
          guanqun Guanqun Lu added a comment -

          The indexes doesn't contain the column name. These column names are stored in namespaces.

          Show
          guanqun Guanqun Lu added a comment - The indexes doesn't contain the column name. These column names are stored in namespaces.
          Hide
          guanqun Guanqun Lu added a comment -

          If this gets implemented, how do we deal with covered index? We'd lose the gain from SERVER-192.

          Show
          guanqun Guanqun Lu added a comment - If this gets implemented, how do we deal with covered index? We'd lose the gain from SERVER-192 .
          Hide
          scotthernandez Scott Hernandez (Inactive) added a comment -

          Right now those keys can't be indexed and aren't covered anyway. This is no worse in that case.

          Show
          scotthernandez Scott Hernandez (Inactive) added a comment - Right now those keys can't be indexed and aren't covered anyway. This is no worse in that case.
          Hide
          scotthernandez Scott Hernandez (Inactive) added a comment -

          The index-spec has the key/field names. The index entries don't store
          the field name, it is true.

          Show
          scotthernandez Scott Hernandez (Inactive) added a comment - The index-spec has the key/field names. The index entries don't store the field name, it is true.
          Hide
          mdcallag Mark Callaghan added a comment -

          Will this work with pluggable engines that impose a smaller limit on max key length? For example InnoDB keys must be less than half of a page.

          Show
          mdcallag Mark Callaghan added a comment - Will this work with pluggable engines that impose a smaller limit on max key length? For example InnoDB keys must be less than half of a page.
          Hide
          jakedempsey Jake Dempsey added a comment -

          I'd love to see the this increased at a minimum. In my current application where we have to build trees with 1M+ nodes, we leverage the materialized path approach and even with using short descriptors we are limited to a tree depth of ~75. We are already approaching 60 levels deep in the tree. This limit is causing us to have to consider a different vendor because in the near future our application will just not work b/c of this limit.

          Show
          jakedempsey Jake Dempsey added a comment - I'd love to see the this increased at a minimum. In my current application where we have to build trees with 1M+ nodes, we leverage the materialized path approach and even with using short descriptors we are limited to a tree depth of ~75. We are already approaching 60 levels deep in the tree. This limit is causing us to have to consider a different vendor because in the near future our application will just not work b/c of this limit.
          Hide
          jakedempsey Jake Dempsey added a comment -

          Has there been any movement on this issue? I am really in a tight spot. In our application there are millions of nodes in the tree and querying the tree and subtrees fast is critical. We also perform billions of db transactions on the tree and we have just found the materialized path approach to be the best mechanism for our needs. Is it possible to at least increase the allowed length? Even just doubling the allowed key length would help us immensely.

          Show
          jakedempsey Jake Dempsey added a comment - Has there been any movement on this issue? I am really in a tight spot. In our application there are millions of nodes in the tree and querying the tree and subtrees fast is critical. We also perform billions of db transactions on the tree and we have just found the materialized path approach to be the best mechanism for our needs. Is it possible to at least increase the allowed length? Even just doubling the allowed key length would help us immensely.

            People

            • Votes:
              16 Vote for this issue
              Watchers:
              30 Start watching this issue

              Dates

              • Created:
                Updated: