[SERVER-30899] Adapting meaning of 'fromRouter' causes problems in mixed version clusters Created: 30/Aug/17  Updated: 30/Oct/23  Resolved: 06/Sep/17

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Sharding
Affects Version/s: None
Fix Version/s: 3.5.13

Type: Bug Priority: Major - P3
Reporter: Charlie Swanson Assignee: Tess Avitabile (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-18940 Optimise sharded aggregations that ar... Closed
is related to SERVER-30924 Remove fromRouter parsing in 3.8 Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Query 2017-09-11
Participants:

 Description   

As part of the work for SERVER-18940 we changed 'fromRouter' to mean 'command came from mongos', instead of its previous meaning, which was 'output from this command is going to be merged with output from other shards'. For the latter case, we added another field, 'needsMerge'.

In a 3.4/3.6 mixed version cluster, a 3.4 mongos will set 'fromRouter' when talking to a 3.6 node, which will not interpret that as meaning 'needsMerge'. This can cause problems like the 3.6 node failing to serialize metadata such as the sort key.



 Comments   
Comment by Githook User [ 06/Sep/17 ]

Author:

{'username': 'tessavitabile', 'name': 'Tess Avitabile', 'email': 'tess.avitabile@mongodb.com'}

Message: SERVER-30899 Aggregations sent from a 3.4 mongos should serialize 3.4 metadata
Branch: master
https://github.com/mongodb/mongo/commit/ed601dd01169b8c1fad9fb8d388da0523a1b48f5

Comment by Bernard Gorman [ 04/Sep/17 ]

tess.avitabile: yes, $currentOp is the only stage which needs to know fromMongos as distinct from needsMerge. The plan above looks good to me, and appears to cover all scenarios where this could cause a problem.

Comment by Tess Avitabile (Inactive) [ 01/Sep/17 ]

bernard.gorman, my plan is to rename fromRouter to fromMongos in 3.6 mongos. Then a 3.6 mongos will always send fromMongos=true and send needsMerge=true if the pipeline must produce mergeable output. A 3.6 mongod will set fromMongos=true if it receives fromMongos=true and set needsMerge=true if it receives needsMerge=true or fromRouter=true. My understanding is that 3.6 mongod only needs to know fromMongos when the pipeline contains a $currentOp, which is only possible from a 3.6 mongos, so it will have this information. And a 3.6 mongos cannot talk to a 3.4 mongod, so I believe all the cases are covered. I will file a ticket to remove parsing of fromRouter after the branch. Does that plan sound okay to you?

Comment by Charlie Swanson [ 31/Aug/17 ]

cc david.storch, ian.whalen - as discussed in the query chat, we're assigning to Tess in the current sprint since she's currently blocked and this seems like something we should fix soon.

Generated at Thu Feb 08 04:25:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.