[SERVER-3317] Regular expression or query on large column (more than 800 characters) which has an Index results in inconsistent records Created: 23/Jun/11  Updated: 30/Mar/12  Resolved: 02/Sep/11

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 1.8.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Manikandan Ehambaram Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: indexing, query
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows x64


Operating System: ALL
Participants:

 Description   

I have inserted a record with a value greater than 850 characters.

> db.testcol.insert({_id:"len850",v:"http://frame.allposters.com/frameimagehandler/universal/frameimage.jpg?frame=[FAP:0+PRT:[PAP=3945180|PRW=24|PRH=36|PIP=%5c28%5c2810%5cZQIOD00Z.jpg|LFN=GBGN0439.jpg|LFL=|PWL=300|PHL=450|PWO=2397|PHO=3596]+MLD:[MID=1153331|MIP=|MDW=1|MDH=0.5625]+GLS:0+NMM:0+MT1:[MTD=0|MTP=|MTC=|MTL=0|MTT=0|MTR=0|MTB=0|MDP=0.2|MBV=0.08]+MT2:[MTD=0|MTP=|MTC=|MTL=0|MTT=0|MTR=0|MTB=0|MDP=0.2|MBV=0.08]+MT3:[MTD=0|MTP=|MTC=|MTL=0|MTT=0|MTR=0|MTB=0|MDP=0.2|MBV=0.08]+CRP:[CID=0|CRX=0|CRY=0|CRW=0|CRH=0]+LIN:[LID=0|LFN=|LNW=0|LND=0]+CAN:[CVD=0|CVC=|CVF=|CVS=0|ECR=|ECB=|BKC=|EDW=0|CTX=1|GWP=0]+MLF:[FLD=0|FLW=0|FFP=]+LNF:[FLD=0|FLW=0|FFP=]+TMF:[FLD=0|FLW=0|FFP=]+MMF:[FLD=0|FLW=0|FFP=]+BMF:[FLD=0|FLW=0|FFP=]+FDP:0+MXD:1000+MXW:1244+MXH:492+SHI:5+WMT:APC+QLT:85+OVH:[TOH=0.25|ROH=0.25|BOH=0.25|LOH=0.25]+CRZ:[CZT=0|CRI=4|LPC=0|TPC=0|WPC=0|HPC=0|CMD=0]+RTP:[RID=0|RFC=FFFFFF" });

After that I have used a regular expression to see if the record exists and it returns the record
> db.testcol.find(

{v:/frame/i}

);

I have added an index on the column with the following command

> db.testcol.ensureIndex(

{v:1}

);

I reissued the same regular expression and returns no records. If I remove the index, the results are returned.
> db.testcol.find(

{v:/frame/i}

);

This issue happens for any value greater than 800+. I am not exactly sure what exact character length the index causes the regular expression to fail.



 Comments   
Comment by Eliot Horowitz (Inactive) [ 02/Sep/11 ]

Fix is SERVER-3372

Comment by Manikandan Ehambaram [ 27/Jun/11 ]

Yes I see a log entry in the Mongod console.

test.system.indexes Btree::insert: key too large to index, skipping test.testcol.$v_1 862 { : ".." }

So the solution is, not to index a text field more than 800 bytes or split the column into multiple 800 byte fields.

Comment by Eliot Horowitz (Inactive) [ 25/Jun/11 ]

The largest field allowed in an index is 800 bytes.
Larger than that, and the document isn't indexed.
You should see an error in the logs or on getLastError after inserting or creating index

Comment by Manikandan Ehambaram [ 24/Jun/11 ]

Verified it in the latest production release 1.8.2 and the issue is reproducible.

Generated at Thu Feb 08 03:02:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.