Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-5771

What is the max index field size we can validate against?

    XMLWordPrintableJSON

Details

    • Icon: Question Question
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • None
    • 2.0.4
    • Index Maintenance
    • None
    • Centos 6 Mongo 2.0.4

    Description

      We are seeing this in the logs during indexing. While we do not expect the field (slug) to be this huge, we are battling a spam infestation that leads to these long slugs. We want to cap this length in the API and wanted to get the number on how long the slug should be allowed to avoid this situation:

      Sun May 6 17:48:19 [rsSync] article-api.system.indexes Btree::insert: key too large to index, skipping article-api.articles.$metadata.slug_1_metadata.articleType_1_metadata.blogName_1 2757 { : "-eternal-darkness-sanitys-requiem-was-an-incredible-game-that-could-definitely-benefit-from-a-current-gen-revival-originally-published-in-2002-as-nint...", : "post", : "skarabrae" }
      Sun May 6 17:48:19 [rsSync] article-api.system.indexes Btree::insert: key too large to index, skipping article-api.articles.$metadata.slug_1_metadata.articleType_1_metadata.blogName_1 3231 { : "-los-tipos-y-estilos-de-son-totalmente-gafas-ray-ban-baratas-diferente-de-otras-gafas-de-sol-que-existen-en-las-gafas-de-sol-marketthey-a-menudo-hacen...", : "post", : "arfranceke" }
      Sun May 6 17:48:19 [rsSync] article-api.system.indexes Btree::insert: key too large to index, skipping article-api.articles.$metadata.slug_1_metadata.articleType_1_metadata.blogName_1 1660 { : "-silver-isnt-any-genuine-plus-guy-isnt-great-at-one-time-there-seemed-to-be-a-male-whom-want-to-get-mouse-your-dog-helped-bring-the-cat-good-at-captur...", : "post", : "tina-tina" }
      Sun May 6 17:48:19 [rsSync] article-api.system.indexes Btree::insert: key too large to index, skipping article-api.articles.$metadata.slug_1_metadata.articleType_1_metadata.blogName_1 3355 { : "hbotv-boxing-high-sourcehd-channel-light-heavyweight-online-rematch-watch-bernard-hopkins-vs-chad-dawson-live-streamchad-dawson-give-his-opinion-that-...", : "post", : "aamirul012" }
      Sun May 6 17:48:19 [rsSync] article-api.system.indexes Btree::insert: key too large to index, skipping article-api.articles.$metadata.slug_1_metadata.articleType_1_metadata.blogName_1 2463 { : "hodan-property-management-amp-development-inc-was-founded-in-1975-as-a-full-service-property-management-company-that-strives-to-meet-all-the-property-...", : "post", : "daniellbrown" }
      Sun May 6 17:48:20 [rsSync] article-api.system.indexes Btree::insert: key too large to index, skipping article-api.articles.$metadata.slug_1_metadata.articleType_1_metadata.blogName_1 2611 { : "strong-vimax-pills-reviews-vimax-no-1-choice-of-consumers-in-the-marketstrong-seek-review-of-a-hrefhttpwwwvimaxpillsexpertcomstrongvimax-pillsstronga-...", : "post", : "himura23k" }
      Sun May 6 17:48:20 [rsSync] article-api.system.indexes Btree::insert: key too large to index, skipping article-api.articles.$metadata.slug_1_metadata.articleType_1_metadata.blogName_1 1923 { : "tn-nike-durante-next-year-trois-des-additionally-populaires-chaussures-lebron-james-vi-kobe-et-lhyperdunk-2011-holders-conceptpour-the-huge-public-wit...", : "post", : "timberland" }
      Sun May 6 17:48:20 [rsSync] article-api.system.indexes Btree::insert: key too large to index, skipping article-api.articles.$metadata.slug_1_metadata.articleType_1_metadata.blogName_1 1908 { : "tn-requin-durante-2012-trois-des-plus-populaires-chaussures-lebron-vi-kobe-et-lhyperdunk-this-years-packages-approachdump-le-awesome-community-minus-i...", : "post", : "timberland" }
      Sun May 6 17:48:20 [rsSync] warning: not all entries were added to the index, probably some keys were too large

      Attachments

        Activity

          People

            Unassigned Unassigned
            mpandit@ign.com Manish Pandit
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: