Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-39598

Unwanted High Network IO in MongoDB Shard Cluster

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • ALL

      Hi,

      I'm having a MogoDB Shard cluster in my production environment. The mongoDB shard cluster is being resided in AWS Cloud under EC2 Instances. Following diagram illustrates big picture of the MongoDB sharded Cluster.

      According to the production environment, I'm maintaining a cluster environment including two shards with replica set.

       

      Shard Master-Node Availability Zone
      Shard-01 Replica-C us-east-a
      Shard-02 Replica-A us-east-a
      MongoS MongoS us-east-a

      In this mongoDB which includes total around 30GB of data, but I do observe very high network in/out in between those two master nodes in the cluster and the Mongos server(Green coloured servers). This huge network traffic is being progressed with 25Mbps for continuous 10hours everyday, starting each day @5.05am UTC and it transfers around 2TB of data in total daily. This is massive cost hitting network traffic and now it's being terrible.

      Following network traffic monitoring dashboard gives the clear idea of spikes.

      Need to find out a solution immediately for this unwanted network traffic. Any help would be greatly appreciated.

      Thanks.

        1. 2019-02-15_16-12-24.png
          2019-02-15_16-12-24.png
          94 kB
        2. INC-mongo-db-cluster-1.jpg
          INC-mongo-db-cluster-1.jpg
          75 kB

            Assignee:
            daniel.hatcher@mongodb.com Danny Hatcher (Inactive)
            Reporter:
            madurad Madura Dissanayake
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: