Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-59923

Retry reshardCollection command from background thread in ReshardingTest fixture

    XMLWordPrintable

Details

    • Fully Compatible
    • v5.1, v5.0
    • Sharding 2021-09-20, Sharding 2021-10-04, Sharding 2021-10-18
    • 162
    • 1

    Description

      Mongos will retry the _shardsvrReshardCollection command on the primary shard of the database only a finite number of times. If the current primary of that replica set is shut down (or steps down and goes into rollback) more than the number of retries from mongos, then the command error response will cause the test to fail.


      [js_test:resharding_fuzzer-120e1-1630670495216-2] d20021| 2021-09-03T12:09:07.675+00:00 I  REPL     21358   [BackgroundSync] "Replica set state transition","attr":{"newState":"ROLLBACK","oldState":"SECONDARY"}
      ...
      [js_test:resharding_fuzzer-120e1-1630670495216-2] s20032| 2021-09-03T12:09:07.679+00:00 D1 ASSERT   23074   [conn46] "User assertion","attr":{"error":"InterruptedDueToReplStateChange: operation was interrupted","file":"src/mongo/s/commands/cluster_reshard_collection_cmd.cpp","line":80}
      ...
      [js_test:resharding_fuzzer-120e1-1630670495216-2] s20032| 2021-09-03T12:09:07.680+00:00 D1 ASSERT   23074   [conn46] "User assertion","attr":{"error":"InterruptedDueToReplStateChange: operation was interrupted","file":"src/mongo/util/future_impl.h","line":1087}
      [js_test:resharding_fuzzer-120e1-1630670495216-2] s20032| 2021-09-03T12:09:07.680+00:00 D1 SHARDING 22772   [conn46] "Exception thrown while processing command","attr":{"db":"admin","headerId":5142,"error":"InterruptedDueToReplStateChange: operation was interrupted"}
      [js_test:resharding_fuzzer-120e1-1630670495216-2] s20032| 2021-09-03T12:09:07.681+00:00 I  COMMAND  51803   [conn46] "Slow query","attr":{"type":"command","ns":"test_reshard.reshard_coll","appName":"MongoDB Shell","command":{"reshardCollection":"test_reshard.reshard_coll","key":{"recipient":1,"slot":1},"_presetReshardedChunks":[{"min":{"recipient":{"$minKey":1},"slot":{"$minKey":1}},"max":{"recipient":"recipient0","slot":{"$minKey":1}},"recipientShardId":"shard0"},{"min":{"recipient":"recipient0","slot":{"$minKey":1}},"max":{"recipient":"recipient0","slot":10},"recipientShardId":"shard0"},{"min":{"recipient":"recipient0","slot":10},"max":{"recipient":"recipient0","slot":20},"recipientShardId":"shard0"},{"min":{"recipient":"recipient0","slot":20},"max":{"recipient":"recipient0","slot":30},"recipientShardId":"shard0"},{"min":{"recipient":"recipient0","slot":30},"max":{"recipient":"recipient0","slot":40},"recipientShardId":"shard0"},{"min":{"recipient":"recipient0","slot":40},"max":{"recipient":"recipient1","slot":{"$minKey":1}},"recipientShardId":"shard0"},{"min":{"recipient":"recipient1","slot":{"$minKey":1}},"max":{"recipient":"recipient1","slot":10},"recipientShardId":"shard1"},{"min":{"recipient":"recipient1","slot":10},"max":{"recipient":"recipient1","slot":20},"recipientShardId":"shard1"},{"min":{"recipient":"recipient1","slot":20},"max":{"recipient":"recipient1","slot":30},"recipientShardId":"shard1"},{"min":{"recipient":"recipient1","slot":30},"max":{"recipient":"recipient1","slot":40},"recipientShardId":"shard1"},{"min":{"recipient":"recipient1","slot":40},"max":{"recipient":"recipient2","slot":{"$minKey":1}},"recipientShardId":"shard1"},{"min":{"recipient":"recipient2","slot":{"$minKey":1}},"max":{"recipient":"recipient2","slot":10},"recipientShardId":"shard2"},{"min":{"recipient":"recipient2","slot":10},"max":{"recipient":"recipient2","slot":20},"recipientShardId":"shard2"},{"min":{"recipient":"recipient2","slot":20},"max":{"recipient":"recipient2","slot":30},"recipientShardId":"shard2"},{"min":{"recipient":"recipient2","slot":30},"max":{"recipient":"recipient2","slot":40},"recipientShardId":"shard2"},{"min":{"recipient":"recipient2","slot":40},"max":{"recipient":"recipient3","slot":{"$minKey":1}},"recipientShardId":"shard2"},{"min":{"recipient":"recipient3","slot":{"$minKey":1}},"max":{"recipient":{"$maxKey":1},"slot":{"$maxKey":1}},"recipientShardId":"shard0"}],"lsid":{"id":{"$uuid":"702b58ee-193c-41f9-b0ae-342688d9eced"}},"$db":"admin"},"numYields":0,"ok":0,"errMsg":"operation was interrupted","errName":"InterruptedDueToReplStateChange","errCode":11602,"reslen":241,"readConcern":{"level":"local","provenance":"implicitDefault"},"remote":"10.122.50.122:34602","protocol":"op_msg","durationMillis":34486}
      ...
      [js_test:resharding_fuzzer-120e1-1630670495216-2] 	"errmsg" : "operation was interrupted",
      [js_test:resharding_fuzzer-120e1-1630670495216-2] 	"codeName" : "HostUnreachable",
      

      https://evergreen.mongodb.com/lobster/build/bd7546f0b83c3529ec1dda233b4d926a/test/61320ff1c2ab687bbc0802b6#bookmarks=0%2C65494%2C65582%2C65590%2C65591%2C65592%2C94518%2C94520%2C104193&f~=000~%22operation%20was%20interrupted%22&f~=000~%5C%5BResharding.%2AService&f~=000~%5C%5BReshardingCoordinatorService&f~=000~20032%5C%7C&f~=100~d2002%5B012%5D%5C%7C&l=1

      Attachments

        Issue Links

          Activity

            People

              max.hirschhorn@mongodb.com Max Hirschhorn
              max.hirschhorn@mongodb.com Max Hirschhorn
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: