[DOCS-7414] Comment on: "mongodb-bi-connector/installation.txt" Created: 14/Mar/16  Updated: 03/Nov/17  Resolved: 23/Sep/16

Status: Closed
Project: Documentation
Component/s: BI Connector
Affects Version/s: None
Fix Version/s: 01112017-cleanup

Type: Bug Priority: Major - P3
Reporter: Docs Collector User (Inactive) Assignee: Andrew Aldridge
Resolution: Won't Fix Votes: 0
Labels: collector-298ba4e7
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS Linux release 7.1.1503 (Core)

Location: https://docs.mongodb.org/bi-connector/installation/
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
Screen Resolution: 1600 x 900
repo: docs-bi-connector
source: installation


Participants:
Days since reply: 7 years, 20 weeks, 5 days ago

 Description   

When loading the generated schema into the MongoDB Connector for BI using mongobischema tool using below command:

mongobischema import biuser schema.drdl

where the schema.drdl file is around 450MB
i am getting below errors:

fatal error: runtime: out of memory
 
runtime stack:
runtime.throw(0x9ce6a0, 0x16)
	/usr/local/go/src/runtime/panic.go:527 +0x90
runtime.sysMap(0xcbc10d0000, 0xe0730000, 0xdf4400, 0xe133d8)
	/usr/local/go/src/runtime/mem_linux.go:203 +0x9b
runtime.mHeap_SysAlloc(0xdf4460, 0xe0730000, 0xc820000d80)
	/usr/local/go/src/runtime/malloc.go:426 +0x160
runtime.mHeap_Grow(0xdf4460, 0x70398, 0x0)
	/usr/local/go/src/runtime/mheap.go:628 +0x63
runtime.mHeap_AllocSpanLocked(0xdf4460, 0x70394, 0x100)
	/usr/local/go/src/runtime/mheap.go:532 +0x5f1
runtime.mHeap_Alloc_m(0xdf4460, 0x70394, 0x100000000, 0xc800000001)
	/usr/local/go/src/runtime/mheap.go:425 +0x1ac
runtime.mHeap_Alloc.func1()
	/usr/local/go/src/runtime/mheap.go:484 +0x41
runtime.systemstack(0x7fcc8a054ce8)
	/usr/local/go/src/runtime/asm_amd64.s:278 +0xab
runtime.mHeap_Alloc(0xdf4460, 0x70394, 0x10100000000, 0x41971f)
	/usr/local/go/src/runtime/mheap.go:485 +0x63
runtime.largeAlloc(0xe0727fc0, 0x7fcc00000000, 0x3)
	/usr/local/go/src/runtime/malloc.go:748 +0xb3
runtime.mallocgc.func3()
	/usr/local/go/src/runtime/malloc.go:637 +0x33
runtime.systemstack(0xc82001b500)
	/usr/local/go/src/runtime/asm_amd64.s:262 +0x79
runtime.mstart()
	/usr/local/go/src/runtime/proc1.go:668
 
goroutine 1 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_amd64.s:216 fp=0xc8200cf4e0 sp=0xc8200cf4d8
runtime.mallocgc(0xe0727fc0, 0x9376a0, 0x0, 0x0)
	/usr/local/go/src/runtime/malloc.go:638 +0x9c4 fp=0xc8200cf5b0 sp=0xc8200cf4e0
runtime.newarray(0x9376a0, 0x12b4355, 0xc820078000)
	/usr/local/go/src/runtime/malloc.go:780 +0xc9 fp=0xc8200cf5f0 sp=0xc8200cf5b0
runtime.growslice(0x7ee6c0, 0xcb0d7e0000, 0xef6900, 0xef6900, 0xef6901, 0x0, 0x0, 0x0)
	/usr/local/go/src/runtime/slice.go:97 +0x2c1 fp=0xc8200cf660 sp=0xc8200cf5f0
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).nextEvent(0xc820078000)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:172 +0x346 fp=0xc8200cf7e0 sp=0xc8200cf660
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).scalar(0xc820078000, 0x802600, 0xc88d0f3b20, 0xd8)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:523 +0x320 fp=0xc8200cf978 sp=0xc8200cf7e0
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).parse(0xc820078000, 0x7e7d00, 0xc88d0f3b20, 0x16)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:210 +0x402 fp=0xc8200cfab8 sp=0xc8200cf978
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).mappingStruct(0xc820078000, 0x8f8640, 0xc88d0f8a00, 0xd9)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:463 +0x172 fp=0xc8200cfbf8 sp=0xc8200cfab8
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).mapping(0xc820078000, 0x8f8640, 0xc88d0f8a00, 0xd9)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:402 +0x790 fp=0xc8200cfda8 sp=0xc8200cfbf8
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).parse(0xc820078000, 0x7d9f00, 0xc88cf0c040, 0xd6)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:206 +0x852 fp=0xc8200cfee8 sp=0xc8200cfda8
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).sequence(0xc820078000, 0x90e4c0, 0xc88cfc6278, 0xd7)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:353 +0xa18 fp=0xc8200d00e0 sp=0xc8200cfee8
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).parse(0xc820078000, 0x90e4c0, 0xc88cfc6278, 0xd7)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:202 +0x62a fp=0xc8200d0220 sp=0xc8200d00e0
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).mappingStruct(0xc820078000, 0x904c60, 0xc88cfc6240, 0xd9)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:493 +0x503 fp=0xc8200d0360 sp=0xc8200d0220
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).mapping(0xc820078000, 0x904c60, 0xc88cfc6240, 0xd9)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:402 +0x790 fp=0xc8200d0510 sp=0xc8200d0360
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).parse(0xc820078000, 0x8ff800, 0xc87feb3ab0, 0xd6)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:206 +0x852 fp=0xc8200d0650 sp=0xc8200d0510
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).sequence(0xc820078000, 0x9023c0, 0xc820072970, 0xd7)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:353 +0xa18 fp=0xc8200d0848 sp=0xc8200d0650
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).parse(0xc820078000, 0x9023c0, 0xc820072970, 0xd7)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:202 +0x62a fp=0xc8200d0988 sp=0xc8200d0848
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).mappingStruct(0xc820078000, 0x8d3ba0, 0xc820072960, 0xd9)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:493 +0x503 fp=0xc8200d0ac8 sp=0xc8200d0988
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).mapping(0xc820078000, 0x8d3ba0, 0xc820072960, 0xd9)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:402 +0x790 fp=0xc8200d0c78 sp=0xc8200d0ac8
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).parse(0xc820078000, 0x8cd000, 0xc8200fa140, 0xd6)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:206 +0x852 fp=0xc8200d0db8 sp=0xc8200d0c78
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).sequence(0xc820078000, 0x7eb9c0, 0xc8200fa020, 0xd7)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:353 +0xa18 fp=0xc8200d0fb0 sp=0xc8200d0db8
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).parse(0xc820078000, 0x7eb9c0, 0xc8200fa020, 0xd7)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:202 +0x62a fp=0xc8200d10f0 sp=0xc8200d0fb0
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).mappingStruct(0xc820078000, 0x8bc9a0, 0xc8200fa020, 0xd9)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:493 +0x503 fp=0xc8200d1230 sp=0xc8200d10f0
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).mapping(0xc820078000, 0x8bc9a0, 0xc8200fa020, 0xd9)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:402 +0x790 fp=0xc8200d13e0 sp=0xc8200d1230
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).parse(0xc820078000, 0x7d9e40, 0xc8200fa020, 0x16)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:206 +0x852 fp=0xc8200d1520 sp=0xc8200d13e0
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).document(0xc820078000, 0x7d9e40, 0xc8200fa020, 0x16)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:182 +0x19a fp=0xc8200d15b8 sp=0xc8200d1520
github.com/cloudfoundry-incubator/candiedyaml.(*Decoder).Decode(0xc820078000, 0x7d9e40, 0xc8200fa020, 0x0, 0x0)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:133 +0x27a fp=0xc8200d16b8 sp=0xc8200d15b8
github.com/cloudfoundry-incubator/candiedyaml.Unmarshal(0xc82014a000, 0x1d557f2f, 0x1d55812f, 0x7d9e40, 0xc8200fa020, 0x0, 0x0)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/cloudfoundry-incubator/candiedyaml/decode.go:102 +0xf5 fp=0xc8200d1718 sp=0xc8200d16b8
github.com/10gen/bi-connector/schematools/mongobischema.ReadDrdl(0x7fffed4eb82e, 0x27, 0x4, 0x0, 0x0)
	/home/mongodb/biconnector/schematools/.gopath/src/github.com/10gen/bi-connector/schematools/mongobischema/builder.go:150 +0x130 fp=0xc8200d18f8 sp=0xc8200d1718
github.com/10gen/bi-connector/schematools/mongobischema.(*SchemaBuilder).Build(0xc8200d1b10, 0x0, 0x0)
	/home/mongodb/biconnector/schematools/.gopath/src/github.com/10gen/bi-connector/schematools/mongobischema/builder.go:34 +0xf7 fp=0xc8200d1ae8 sp=0xc8200d18f8
github.com/10gen/bi-connector/schematools/mongobischema.(*ImportCommand).Execute(0xc8200127f0, 0xc82000a400, 0x1, 0x4, 0x0, 0x0)
	/home/mongodb/biconnector/schematools/.gopath/src/github.com/10gen/bi-connector/schematools/mongobischema/import.go:21 +0xba fp=0xc8200d1b40 sp=0xc8200d1ae8
github.com/jessevdk/go-flags.(*Parser).ParseArgs(0xc8200e85f0, 0xc82006c100, 0x4, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/mongodb/biconnector/schematools/vendor/src/github.com/jessevdk/go-flags/parser.go:301 +0xa16 fp=0xc8200d1e08 sp=0xc8200d1b40
github.com/10gen/bi-connector/schematools/mongobischema.Execute(0xc82006c100, 0x4, 0x4, 0x0, 0x0)
	/home/mongodb/biconnector/schematools/.gopath/src/github.com/10gen/bi-connector/schematools/mongobischema/options.go:37 +0xb8 fp=0xc8200d1e90 sp=0xc8200d1e08
main.main()
	/home/mongodb/rpmbuild/BUILD/biconnector/schematools/mongobischema/main/mongobischema.go:15 +0x47 fp=0xc8200d1f30 sp=0xc8200d1e90
runtime.main()
	/usr/local/go/src/runtime/proc.go:111 +0x2b0 fp=0xc8200d1f80 sp=0xc8200d1f30
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8200d1f88 sp=0xc8200d1f80
 
goroutine 17 [syscall, 1 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1
 
goroutine 20 [syscall, 1 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
	/usr/local/go/src/os/signal/signal_unix.go:28 +0x37



 Comments   
Comment by Andrew Aldridge [ 23/Sep/16 ]

A 450MB DRDL file strongly points towards a dataset that cannot be reasonably worked with in a relational context.

Generated at Thu Feb 08 07:54:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.