[SERVER-21634] Add mixed version testing for aggregations Created: 23/Nov/15  Updated: 09/Dec/16  Resolved: 10/Dec/15

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Charlie Swanson Assignee: Charlie Swanson
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-27296 should run jsCore passthrough for mul... Closed
Backwards Compatibility: Fully Compatible
Sprint: QuInt D (12/14/15)
Participants:

 Description   

We should add some testing of various aggregation pipelines against various mixed-version topologies. In particular, it would be good to stress:



 Comments   
Comment by Charlie Swanson [ 10/Dec/15 ]

Manual testing done. Results summarized below

New stages, accumulators, expressions

As expected, new operators are not available until the mongos is upgraded to 3.2, as the last step in the cycle.

SERVER-14691: $avg of non-numerics now returns null, not 0

As expected, if the $group stage is run on a 3.0 mongod, it will return an average of 0 for non-numeric values. If the $group stage is run on a 3.2 mongod, it will return an average of null.
To be more precise, the following table describes what an $avg of all non-numeric inputs returns in each scenario:

  unsharded collection sharded collection
primary shard for the database is 3.0 0 0
primary shard for the database is 3.2 null null

Note it is only the version of the primary shard for the database that matters for sharded collections, since a $group stage is always run during the merging portion of the aggregation, which (when you have a 3.0 mongos) always happens on the primary shard for that database. If the primary shard for the database is a replica set, your readPreference will determine which node it will run on, which may impact which version of mongod does the merging.

SERVER-6801 $substr now errors if it would split a single UTF-8 character

All worked as expected, an invalid $substr errors when it is run on a 3.2 mongod. Note the $substr expression can be used in the $project stage, which does not force a split of the pipeline, so might be executed in parallel on all the relevant shards, or might be executed during the merging portion, on the primary shard for the database.

SERVER-8141 Arrays are no longer parsed as literal expressions

This is a little more subtle. If you are using a sharded collection, the mongos will parse the pipeline before sending part of it to the relevant shards. If your mongos is on version 3.0, it will parse arrays as literal, then when it serializes the pipeline again to be sent to the shards, it will wrap literal values in a $const expression (functionally the same as $literal). Thus, on sharded collections, you will not observe any of the different parsing behavior for SERVER-8141 until your mongos is upgraded.

For unsharded collections, the mongos does not need to split the pipeline and re-serialize part of it, so it sends the existing command object, as-is, directly to the shard with that collection. In this case, if the mongod that receives it is on version 3.2, it will use the new parsing behavior.

Summary Of Test Execution, Results

The testing involved starting with a sharded cluster where each process was running version 3.0.7, and slowly upgrading it as described in our upgrade docs. The cluster had one mongos, one config server, and two shards. Each shard was a 3-node replica set.

Data setup

_id's have been removed to improve readability.

Namespace Sharded Primary shard for database Data on shard 1 Data on shard 2
shard1.unsharded no shard1

{x: 1}
{x: 1, y: 2}
{x: 2, string: "ǦŔĔĀŢ"}
{x: 3, transactions: [{price: "$2"}]}

None
shard1.sharded yes shard1

{x: 1}
{x: 1, y: 2}
{x: 2, string: "ǦŔĔĀŢ"}
{x: 3, transactions: [{price: "$2"}]}

{x: 1}
{x: 1, y: 2}
{x: 2, string: "ǦŔĔĀŢ"}
{x: 3, transactions: [{price: "$2"}]}

shard2.unsharded no shard2 None

{x: 1}
{x: 1, y: 2}
{x: 2, string: "ǦŔĔĀŢ"}
{x: 3, transactions: [{price: "$2"}]}

shard2.sharded yes shard2

{x: 1}
{x: 1, y: 2}
{x: 2, string: "ǦŔĔĀŢ"}
{x: 3, transactions: [{price: "$2"}]}

{x: 1}
{x: 1, y: 2}
{x: 2, string: "ǦŔĔĀŢ"}
{x: 3, transactions: [{price: "$2"}]}

Behaviors Tested
Pipeline run Behavior intended to test

[{$sample: {size: 10}}] 

if a new stage would be recognized.

[{$project: {
  x: {$ceil: "$x"}
}}] 

if a new expression would be recognized.

[{$group: {
  _id: null,
  x: {$stdDevSamp: "$x"}
}}] 

if a new accumulator would be recognized.

[{$group: {
  _id: null,
  x: {$avg: "$non-existent"}
}}] 

what an average of no numeric values would return (SERVER-14691)

[{$project: {
  x: {$substr: ["$string", 0, 4]}
}}] 

if an invalid substring would error, or return an invalid document (SERVER-6801)

[{$project: {
  x: {
    $setIntersection: [
      "$transactions.price",
      ["$2"]
    ]
  }
}}] 

if "$2" would be treated as a string literal, or as a field path (SERVER-8141)
Testing Process
  1. Started a sharded cluster, everything on 3.0.7, verify the following expected behaviors (columns correspond to tests described in 'Behaviors Tested' section)
    $sample $ceil $stdDevSamp $avg $substr $setIntersection
    fails fails fails 0 invalid UTF-8 character treated as literal, so matches
  2. Upgrade the secondaries of shard1 to 3.2. Verify the following behavior
    ReadPreference namespace $sample $ceil $stdDevSamp $avg $substr $setIntersection
    "primary" shard1.unsharded fails fails fails 0 invalid UTF-8 character treated as literal, so matches
      shard1.sharded fails fails fails 0 invalid UTF-8 character treated as literal, so matches
      shard2.unsharded fails fails fails 0 invalid UTF-8 character treated as literal, so matches
      shard2.sharded fails fails fails 0 invalid UTF-8 character treated as literal, so matches
    "secondary" shard1.unsharded fails fails fails null error treated as field path, so no match
      shard1.sharded fails fails fails null error treated as literal, so matches
      shard2.unsharded fails fails fails 0 invalid UTF-8 character treated as literal, so matches
      shard2.sharded fails fails fails 0 error treated as literal, so matches
  3. Upgrade the primary of shard1 to 3.2, observe the following behavior:
    namespace $sample $ceil $stdDevSamp $avg $substr $setIntersection
    shard1.unsharded fails fails fails null error treated as field path, so no match
    shard1.sharded fails fails fails null error treated as literal, so matches
    shard2.unsharded fails fails fails 0 invalid UTF-8 character treated as literal, so matches
    shard2.sharded fails fails fails 0 error treated as literal, so matches
  4. Upgrade the secondaries of shard2 to 3.2, observe the following behavior:
    ReadPreference namespace $sample $ceil $stdDevSamp $avg $substr $setIntersection
    "primary" shard1.unsharded fails fails fails null error treated as field path, so no match
      shard1.sharded fails fails fails null error treated as literal, so matches
      shard2.unsharded fails fails fails 0 invalid UTF-8 character treated as literal, so matches
      shard2.sharded fails fails fails 0 error treated as literal, so matches
    "secondary" shard1.unsharded fails fails fails null error treated as field path, so no match
      shard1.sharded fails fails fails null error treated as literal, so matches
      shard2.unsharded fails fails fails null error treated as field path, so no match
      shard2.sharded fails fails fails null error treated as literal, so matches
  5. Upgrade primary of shard 2 to 3.2, observe the following behavior:
    ReadPreference namespace $sample $ceil $stdDevSamp $avg $substr $setIntersection
    "primary" shard1.unsharded fails fails fails null error treated as field path, so no match
      shard1.sharded fails fails fails null error treated as literal, so matches
      shard2.unsharded fails fails fails null error treated as field path, so no match
      shard2.sharded fails fails fails null error treated as literal, so matches
    "secondary" shard1.unsharded fails fails fails null error treated as field path, so no match
      shard1.sharded fails fails fails null error treated as literal, so matches
      shard2.unsharded fails fails fails null error treated as field path, so no match
      shard2.sharded fails fails fails null error treated as literal, so matches
  6. Upgrade the config server, then the mongos to 3.2, observe the following behavior:
    namespace $sample $ceil $stdDevSamp $avg $substr $setIntersection
    shard1.unsharded succeeds succeeds succeeds null error treated as field path, so no match
    shard1.sharded succeeds succeeds succeeds null error treated as field path, so no match
    shard2.unsharded succeeds succeeds succeeds null error treated as field path, so no match
    shard2.sharded succeeds succeeds succeeds null error treated as field path, so no match
Comment by Charlie Swanson [ 09/Dec/15 ]

After discussing with kamran.khan, we've decided to just manually verify that things work as expected in an upgrade/downgrade scenario. Adding an automated test for these cases won't add much value, and we don't think it's a high-risk area. I will do the manual tests and close the ticket once verified. I'll comment with the specifics of how it was tested once it's been done.

Generated at Thu Feb 08 03:57:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.