Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-2133

mongoimport shows system out of memory but system have memory. please help to solve.

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Won't Fix
    • Affects Version/s: 4.0.1
    • Fix Version/s: No versions
    • Component/s: mongoimport
    • Labels:
    • Environment:
      Test Environment.

      Description

      [arcus@ip-172-22-17-11 bkn]$ mongoimport -d arcusairdb -u incusdba -p incus@123 --host 172.22.17.11 --file somethingimages.json -c somethingimages --drop
      2018-09-22T04:06:50.513+0000    connected to: 172.22.17.11
      2018-09-22T04:06:50.514+0000    dropping: arcusairdb.somethingimages
      2018-09-22T04:06:53.500+0000    [##......................] arcusairdb.somethingimages     256MB/2.61GB (9.6%)
      2018-09-22T04:06:56.500+0000    [####....................] arcusairdb.somethingimages     512MB/2.61GB (19.2%)
      2018-09-22T04:06:59.500+0000    [#########...............] arcusairdb.somethingimages     1024MB/2.61GB (38.4%)
      2018-09-22T04:07:02.500+0000    [#########...............] arcusairdb.somethingimages     1024MB/2.61GB (38.4%)
      2018-09-22T04:07:05.500+0000    [#########...............] arcusairdb.somethingimages     1024MB/2.61GB (38.4%)
      2018-09-22T04:07:08.500+0000    [##################......] arcusairdb.somethingimages     2.00GB/2.61GB (76.7%)
      2018-09-22T04:07:11.500+0000    [##################......] arcusairdb.somethingimages     2.00GB/2.61GB (76.7%)
      fatal error: runtime: out of memory
       
      runtime stack:
      runtime.throw(0x85b391, 0x16)
              /opt/go/src/runtime/panic.go:566 +0x95
      runtime.sysMap(0xc520200000, 0x100000000, 0xc3c400, 0xc5a798)
              /opt/go/src/runtime/mem_linux.go:219 +0x1d0
      runtime.(*mheap).sysAlloc(0xc414c0, 0x100000000, 0x417400)
              /opt/go/src/runtime/malloc.go:407 +0x37a
      runtime.(*mheap).grow(0xc414c0, 0x80000, 0x0)
              /opt/go/src/runtime/mheap.go:726 +0x62
      runtime.(*mheap).allocSpanLocked(0xc414c0, 0x80000, 0xc4200ca910)
              /opt/go/src/runtime/mheap.go:630 +0x4f2
      runtime.(*mheap).alloc_m(0xc414c0, 0x80000, 0x100000000, 0x4302a6)
              /opt/go/src/runtime/mheap.go:515 +0xe0
      runtime.(*mheap).alloc.func1()
              /opt/go/src/runtime/mheap.go:579 +0x4b
      runtime.systemstack(0x7f3415baecd8)
              /opt/go/src/runtime/asm_amd64.s:314 +0xab
      runtime.(*mheap).alloc(0xc414c0, 0x80000, 0x10100000000, 0xc42001a000)
              /opt/go/src/runtime/mheap.go:580 +0x73
      runtime.largeAlloc(0xfffffe00, 0x7f3415baed01, 0x45a1ca)
              /opt/go/src/runtime/malloc.go:774 +0x93
      runtime.mallocgc.func1()
              /opt/go/src/runtime/malloc.go:669 +0x3e
      runtime.systemstack(0xc42001a000)
              /opt/go/src/runtime/asm_amd64.s:298 +0x79
      runtime.mstart()
              /opt/go/src/runtime/proc.go:1079
       
      goroutine 25 [running]:
      runtime.systemstack_switch()
              /opt/go/src/runtime/asm_amd64.s:252 fp=0xc420035c80 sp=0xc420035c78
      runtime.mallocgc(0xfffffe00, 0x7c2960, 0x1, 0x4)
              /opt/go/src/runtime/malloc.go:670 +0x903 fp=0xc420035d20 sp=0xc420035c80
      runtime.makeslice(0x7c2960, 0x7ffffe00, 0xfffffe00, 0x40000000, 0x40000000, 0x0)
              /opt/go/src/runtime/slice.go:57 +0x7b fp=0xc420035d78 sp=0xc420035d20
      github.com/mongodb/mongo-tools/common/json.(*Decoder).readValue(0xc42012f520, 0x0, 0x0, 0x0)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/common/json/stream.go:164 +0x26e fp=0xc420035e18 sp=0xc420035d78
      github.com/mongodb/mongo-tools/common/json.(*Decoder).ScanObject(0xc42012f520, 0x0, 0x0, 0x0, 0x0, 0x0)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/common/json/stream.go:72 +0x46 fp=0xc420035e80 sp=0xc420035e18
      github.com/mongodb/mongo-tools/mongoimport.(*JSONInputReader).StreamDocument.func1(0xc4200f22a0, 0xc4200f2900, 0xc4200f2960)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/json.go:124 +0x52 fp=0xc420035f98 sp=0xc420
       
      goroutine 25 [running]:
      runtime.systemstack_switch()
              /opt/go/src/runtime/asm_amd64.s:252 fp=0xc420035c80 sp=0xc420035c78
      runtime.mallocgc(0xfffffe00, 0x7c2960, 0x1, 0x4)
              /opt/go/src/runtime/malloc.go:670 +0x903 fp=0xc420035d20 sp=0xc420035c80
      runtime.makeslice(0x7c2960, 0x7ffffe00, 0xfffffe00, 0x40000000, 0x40000000, 0x0)
              /opt/go/src/runtime/slice.go:57 +0x7b fp=0xc420035d78 sp=0xc420035d20
      github.com/mongodb/mongo-tools/common/json.(*Decoder).readValue(0xc42012f520, 0x0, 0x0, 0x0)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/common/json/stream.go:164 +0x26e fp=0xc420035e18 sp=0xc420035d78
      github.com/mongodb/mongo-tools/common/json.(*Decoder).ScanObject(0xc42012f520, 0x0, 0x0, 0x0, 0x0, 0x0)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/common/json/stream.go:72 +0x46 fp=0xc420035e80 sp=0xc420035e18
      github.com/mongodb/mongo-tools/mongoimport.(*JSONInputReader).StreamDocument.func1(0xc4200f22a0, 0xc4200f2900, 0xc4200f2960)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/json.go:124 +0x52 fp=0xc420035f98 sp=0xc420035e80
      runtime.goexit()
              /opt/go/src/runtime/asm_amd64.s:2086 +0x1 fp=0xc420035fa0 sp=0xc420035f98
      created by github.com/mongodb/mongo-tools/mongoimport.(*JSONInputReader).StreamDocument
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/json.go:141 +0x9d
       
      goroutine 1 [chan receive]:
      github.com/mongodb/mongo-tools/mongoimport.channelQuorumError(0xc4200f2780, 0x2, 0xc4200f2780, 0xc4200f4700)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/common.go:142 +0x6d
      github.com/mongodb/mongo-tools/mongoimport.(*MongoImport).importDocuments(0xc4200f4700, 0xc27340, 0xc4200f22a0, 0x0, 0x0, 0x0)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/mongoimport.go:400 +0x8b5
      github.com/mongodb/mongo-tools/mongoimport.(*MongoImport).ImportDocuments(0xc4200f4700, 0x0, 0x0, 0x0)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/mongoimport.go:335 +0x464
      main.main()
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/mongoimport/main/mongoimport.go:78 +0x563
       
      goroutine 17 [syscall, locked to thread]:
      runtime.goexit()
              /opt/go/src/runtime/asm_amd64.s:2086 +0x1
       
      goroutine 5 [syscall]:
      os/signal.signal_recv(0x0)
              /opt/go/src/runtime/sigqueue.go:116 +0x157
      os/signal.loop()
              /opt/go/src/os/signal/signal_unix.go:22 +0x22
      created by os/signal.init.1
              /opt/go/src/os/signal/signal_unix.go:28 +0x41
       
      goroutine 9 [select, locked to thread]:
      runtime.gopark(0x88cad0, 0x0, 0x854cfb, 0x6, 0x18, 0x2)
              /opt/go/src/runtime/proc.go:259 +0x13a
      runtime.selectgoImpl(0xc420024730, 0x0, 0x18)
              /opt/go/src/runtime/select.go:423 +0x11d9
      runtime.selectgo(0xc420024730)
              /opt/go/src/runtime/select.go:238 +0x1c
      runtime.ensureSigM.func1()
              /opt/go/src/runtime/signal1_unix.go:304 +0x2f3
      runtime.goexit()
              /opt/go/src/runtime/asm_amd64.s:2086 +0x1
       
      goroutine 19 [select]:
      github.com/mongodb/mongo-tools/common/signals.handleSignals(0x0, 0xc4200f21e0)
            /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/common/signals/signals.go:54 +0x44c
      created by github.com/mongodb/mongo-tools/common/signals.HandleWithInterrupt
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/common/signals/signals.go:31 +0x67
       
      goroutine 20 [select]:
      github.com/mongodb/mongo-tools/common/progress.(*Bar).start(0xc4200f2300)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/common/progress/progress_bar.go:152 +0x155
      created by github.com/mongodb/mongo-tools/common/progress.(*Bar).Start
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/common/progress/progress_bar.go:69 +0xc8
       
      goroutine 21 [select]:
      gopkg.in/mgo%2ev2.(*mongoCluster).syncServersLoop(0xc4200d6300)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/vendor/src/gopkg.in/mgo.v2/cluster.go:394 +0x3b3
      created by gopkg.in/mgo%2ev2.newCluster
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/vendor/src/gopkg.in/mgo.v2/cluster.go:78 +0x194
       
      goroutine 22 [chan receive]:
      github.com/mongodb/mongo-tools/mongoimport.channelQuorumError(0xc4200f2960, 0x2, 0xc4200f2960, 0xc4200f2900)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/common.go:142 +0x6d
      github.com/mongodb/mongo-tools/mongoimport.(*JSONInputReader).StreamDocument(0xc4200f22a0, 0x0, 0xc4200f2720, 0x0, 0x0)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/json.go:148 +0xfd
      github.com/mongodb/mongo-tools/mongoimport.(*MongoImport).importDocuments.func1(0xc4200f2780, 0xc27340, 0xc4200f22a0, 0x7ffe6e393700, 0xc4200f2720)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/mongoimport.go:392 +0x44
      created by github.com/mongodb/mongo-tools/mongoimport.(*MongoImport).importDocuments
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/mongoimport.go:393 +0x865
       
      goroutine 12 [sleep]:
      time.Sleep(0x37e11d600)
              /opt/go/src/runtime/time.go:59 +0xe1
      gopkg.in/mgo%2ev2.(*mongoServer).pinger(0xc42000c460, 0xc4200c0b01)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/vendor/src/gopkg.in/mgo.v2/server.go:301 +0x2ae
      created by gopkg.in/mgo%2ev2.newServer
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/vendor/src/gopkg.in/mgo.v2/server.go:89 +0x15f
       
      goroutine 14 [IO wait]:
      net.runtime_pollWait(0x7f341921bf28, 0x72, 0x4)
              /opt/go/src/runtime/netpoll.go:160 +0x59
      net.(*pollDesc).wait(0xc420052290, 0x72, 0xc420032d58, 0xc420012088)
              /opt/go/src/net/fd_poll_runtime.go:73 +0x38
      net.(*pollDesc).waitRead(0xc420052290, 0xc24400, 0xc420012088)
              /opt/go/src/net/fd_poll_runtime.go:78 +0x34
      net.(*netFD).Read(0xc420052230, 0xc4200e0b10, 0x24, 0x24, 0x0, 0xc24400, 0xc420012088)
              /opt/go/src/net/fd_unix.go:243 +0x1a1
      net.(*conn).Read(0xc420026120, 0xc4200e0b10, 0x24, 0x24, 0x0, 0x0, 0x0)
              /opt/go/src/net/net.go:173 +0x70
      gopkg.in/mgo%2ev2.fill(0xc29120, 0xc420026120, 0xc4200e0b10, 0x24, 0x24, 0x0, 0xd1)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/vendor/src/gopkg.in/mgo.v2/socket.go:536 +0x53
      gopkg.in/mgo%2ev2.(*mongoSocket).readLoop(0xc42000c540)
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/vendor/src/gopkg.in/mgo.v2/socket.go:552 +0x125
      created by gopkg.in/mgo%2ev2.newSocket
              /data/mci/e94377d4e5982a1a4dfc3dd4e985bded/src/src/mongo/gotools/vendor/src/gopkg.in/mgo.v2/socket.go:195 +0x278
       
      goroutine 23 [semacquire]:
      sync.runtime_Semacquire(0xc42015a0fc)
              /opt/go/src/runtime/sema.go:47 +0x30
      sync.(*WaitGroup).Wait(0xc42015a0f0)
              /opt/go/src/sync/waitgroup.go:131 +0x97
      github.com/mongodb/mongo-tools/mongoimport.(*MongoImport).ingestDocuments(0xc4200f4700, 0xc4200f2720, 0x0, 0x0)
      

      top - 04:09:48 up 16 days, 18:28,  7 users,  load average: 0.03, 0.09, 0.07
      Tasks: 112 total,   1 running, 111 sleeping,   0 stopped,   0 zombie
      %Cpu(s):  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
      KiB Mem :  7905380 total,  5015952 free,   244772 used,  2644656 buff/cache
      KiB Swap:        0 total,        0 free,        0 used.  7167040 avail Mem 
       
        PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                              
      13050 root      20   0 1521728 107044   8496 S   0.3  1.4   0:04.47 mongod                                                                                               
          1 root      20   0   46100   2900    532 S   0.0  0.0   0:15.60 systemd                                                                                              
          2 root      20   0       0      0      0 S   0.0  0.0   0:00.24 kthreadd                                                                                             
          3 root      20   0       0      0      0 S   0.0  0.0   0:00.79 ksoftirqd/0                                                                                          
          5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:0H                                                                                         
          7 root      rt   0       0      0      0 S   0.0  0.0   0:00.91 migration/0                                                                                          
          8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh                                                                                               
          9 root      20   0       0      0      0 S   0.0  0.0   1:05.92 rcu_sched                                                                                            
         10 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 lru-add-drain                                                                                        
         11 root      rt   0       0      0      0 S   0.0  0.0   0:03.87 watchdog/0                                                                                           
         12 root      rt   0       0      0      0 S   0.0  0.0   0:02.80 watchdog/1                                                                                           
         13 root      rt   0       0      0      0 S   0.0  0.0   0:01.03 migration/1     
      

        Attachments

          Activity

            People

            Assignee:
            backlog-server-tools Backlog - Server Tools Team
            Reporter:
            manoj@incarnus.com Manoj Kumar Arumugam
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: