Uploaded image for project: 'C# Driver'
  1. C# Driver
  2. CSHARP-1061

Mongo InsertBatch and IO read performance degradaion

    XMLWordPrintable

Details

    • Task
    • Status: Closed
    • Major - P3
    • Resolution: Works as Designed
    • 1.9
    • None
    • Performance
    • Windows 64 bit. 8GB RAM. i5 2540M 2,6 GHz

    Description

      So here is the use case: I have a locally saved large log file (>500 MB) which I read in predefined chunks in c# and every time i get a chunk I insert that to a localhost Mongo instance. Next chunk I append it to the same collection. I use "InsertBatch" method from MongoCollection.cs.
      Previously I was using SqlLite for the same mechanism and every chunk of approx 10 MB size had following numbers in my machine:
      1. Chunk Read time = ~ 0.5 s
      2. Chunk write to SqlLite = ~ 3s

      Now with Mongo I have gained better performance in write but my reading performance has degraded drastically. For general chunks numbers are now like:
      1. Chunk Read Time = ~ 3s . For few chunks (although the size is same) it shoots up like a spike and goes beyond 10-15 s)
      2. Write time = ~2s

      So if I consider "Total time", the performance has degraded with MongoDB because of the log reading time. How can this be related is my first question. Second is, how to get rid of this?

      Attachments

        1. Capture.PNG
          Capture.PNG
          261 kB
        2. Logfile.PML
          172.86 MB
        3. ReadProfile-latest.app8results
          82.05 MB

        Activity

          People

            Unassigned Unassigned
            nilanjan Nilanjan Dutta
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: