[SERVER-6999] Embedded "dot-notation" fields broken for tag based balancing Created: 10/Sep/12  Updated: 06/Dec/22  Resolved: 12/Oct/17

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 2.2.0
Fix Version/s: 3.6.0-rc0

Type: Task Priority: Critical - P2
Reporter: Scott Hernandez (Inactive) Assignee: [DO NOT USE] Backlog - Sharding Team
Resolution: Done Votes: 18
Labels: PM497, sharding, tagging
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-24174 Inconsistent rule for storing dotted ... Closed
Duplicate
is duplicated by SERVER-7000 addTagRange doesn't work with embedde... Closed
is duplicated by SERVER-13690 Can't add tag ranges when sharding on... Closed
is duplicated by SERVER-11828 Tag ranges for embedded shard keys Closed
Related
related to SERVER-29334 Cannot write shard chunks documents w... Closed
related to SERVER-6852 Disallow empty path components in doc... Backlog
related to SERVER-6357 Add tag based sharding commands Closed
Assigned Teams:
Sharding
Backwards Compatibility: Fully Compatible
Sprint: 2.3.0
Participants:

 Description   

> sh.addTagRange("test.test", {"a.b":1}, {"a.b":2}, "range1")
Mon Sep 10 14:07:34 uncaught exception: can't have . in field names [a.b]



 Comments   
Comment by Nicholas Zolnierz [ 12/Oct/17 ]

Fixed as part of SERVER-29334, and integration testing added as part of SERVER-24174.

Comment by Asya Kamsky [ 09/Oct/17 ]

It might work now as a side effect of changes in SERVER-29334?

Comment by Asya Kamsky [ 09/Oct/17 ]

Just tested this in the latest (3.5.13) and it works fine now - I believe the shell helper now works correctly.

I believe this ticket can be closed.

sh.addTagRange("test.test", {"a.b":1}, {"a.b":2}, "range1")
{
	"ok" : 1,
	"$clusterTime" : {
		"clusterTime" : Timestamp(1507563925, 1),
		"signature" : {
			"hash" : BinData(0,"mgg3A8GmQ33aT0Pl7M6sSu7QK9I="),
			"keyId" : NumberLong("6468898768493215745")
		}
	},
	"operationTime" : Timestamp(1507563925, 1)
}

Comment by Xavier Vdb [ 24/Apr/14 ]

same issue with mongo 2.6

i have solved the problem with the same fix of Scott, without rewriting 'sh.addTagRange' method :

DBCollection.prototype._validateForStorage = function( o ){
    this._validateObject( o );
    for ( var k in o ){
        if ( k.indexOf( "$" ) == 0 && ! DBCollection._allowedFields[k] ) {
            throw "field names cannot start with $ [" + k + "]";
        }
 
        if ( o[k] !== null && typeof( o[k] ) === "object" ) {
            this._validateForStorage( o[k] );
        }
    }
};
 

Comment by Scott Hernandez (Inactive) [ 29/Dec/12 ]

Quick fix is to bypass validation in the shell for inserts. The better solution is to move these to commands which can be used by all drivers/languages as specified in this issue: SERVER-6357.

DBCollection.prototype._validateForStorage = function( o ){
    this._validateObject( o );
    for ( var k in o ){
        if ( k.indexOf( "$" ) == 0 && ! DBCollection._allowedFields[k] ) {
            throw "field names cannot start with $ [" + k + "]";
        }
 
        if ( o[k] !== null && typeof( o[k] ) === "object" ) {
            this._validateForStorage( o[k] );
        }
    }
};
 
sh.addTagRange = function( ns, min, max, tag ) {
    var config = db.getSisterDB( "config" );
    config.tags.insert( {_id: { ns : ns , min : min }, ns : ns , min : min , max : max , tag : tag  } );
    sh._checkLastError( config );    
}
 
//now add tags ranges (this will not allow updating existing ones, just adding new ones)
 
sh.addTagRange("users.contacts", {"address.zip":MinKey}, {"address.zip":MaxKey}, "contacts_collection") // three of the four shards are tagged for this collection

The reason this works in python will depend on the version and level of validation. It should not work as that is not valid and should be caught by all drivers/languages, but clearly your python version let it slip by.

Comment by Alon Horev [ 26/Oct/12 ]

This is interesting, running addTagRange's code from pymongo doesn't result in an error.
I was in a need to add tag ranges programmatically, looking at the source code revealed a simple update: config.tags.update(

{ns:ns, min:min}

,

{ns:ns, min:min, max:max, tag:tag}

, true);

now for the question: what's the difference between running this update from the shell vs. pymongo? how come pymongo succeeds and the shell doesn't?

Generated at Thu Feb 08 03:13:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.