[SERVER-8521] Lazily clean up temp collections Created: 12/Feb/13  Updated: 11/Jul/16  Resolved: 14/Feb/13

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 2.4.0-rc0
Fix Version/s: 2.4.0-rc1

Type: Bug Priority: Major - P3
Reporter: Eric Milkie Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-8515 Unable to start DB with > 1024 files ... Closed
Related
related to SERVER-8600 improve handling of non-fsyncable dir... Closed
Operating System: ALL
Participants:

 Description   

Cease opening all dbs at startup – this pushes the fd count past 1024 and breaks things when we try to use select() on sockets.

Instead, do the temp cleanup lazily at Database open time.



 Comments   
Comment by Alvin Richards (Inactive) [ 16/Feb/13 ]

Tested successfully on OS-X with git version: 1bd8b84c64214356f482fa3164d88e664f585243

Comment by Alvin Richards (Inactive) [ 16/Feb/13 ]

Just checked the latest open-vmtools source 9.2.2 and its says the same thing

http://sourceforge.net/projects/open-vm-tools/files/open-vm-tools/stable-9.2.x/

Ok, that eliminate the HGFS from the equation, will re-test on vanilla disks.

Comment by Eric Milkie [ 16/Feb/13 ]

Weird, that code says it does nothing and just returns "success"! Not the behavior we're seeing here.

Comment by Alvin Richards (Inactive) [ 16/Feb/13 ]

Certainly the VMWare Tool 4.x circa 2006 explicitly states no support for fsync, see here

http://www.ieo.es/ftp/software/open-vm-tools-4.0.0-162945/modules/linux/vmhgfs/file.c

I'll see if I can find any later changes that would include support.

Comment by Eric Milkie [ 16/Feb/13 ]

I'm going to file a separate ticket to clean up this error message regarding fsync. I've seen other users run into this issue and the logging could be clearer in this situation.

Comment by Eric Milkie [ 16/Feb/13 ]

I don't think you can call fsync on an HGFS (shared VMWare) mount? It might not be supported. Was this working before using that mounted device as a data directory? Did you upgrade VMWare in the interim?

Comment by Alvin Richards (Inactive) [ 16/Feb/13 ]

Sorry, full stack is here (cut & paste problem)

Sat Feb 16 07:04:41.093 [initandlisten] MongoDB starting : pid=2518 port=27017 dbpath=/mnt/hgfs/data/db/bug/ 64-bit host=alvin-ubuntu910-server64
Sat Feb 16 07:04:41.093 [initandlisten] db version v2.4.0-rc1-pre-, pdfile version 4.5
Sat Feb 16 07:04:41.093 [initandlisten] git version: b73eb512ad5429bce3e796e53eaeb56a70e0140a
Sat Feb 16 07:04:41.093 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49
Sat Feb 16 07:04:41.093 [initandlisten] allocator: tcmalloc
Sat Feb 16 07:04:41.093 [initandlisten] options:

{ dbpath: "/mnt/hgfs/data/db/bug/", logpath: "/mnt/hgfs/data/db/bug/server.log", noprealloc: true, smallfiles: true }

Sat Feb 16 07:04:41.098 [initandlisten] Assertion: 13651:Couldn't fsync directory '/mnt/hgfs/data/db/bug': errno:22 Invalid argument
0xdc1be1 0xd832ab 0xd837ec 0x9211a3 0x9e5d51 0x6d3309 0x6d3b9d 0x6da46d 0x6dc1d9 0x7f95e65ddabd 0x6cc9c9
./mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdc1be1]
./mongod(_ZN5mongo11msgassertedEiPKc+0x9b) [0xd832ab]
./mongod [0xd837ec]
./mongod(_ZN5mongo16flushMyDirectoryERKN5boost11filesystem34pathE+0x293) [0x9211a3]
./mongod(_ZN5mongo15acquirePathLockEb+0x1d1) [0x9e5d51]
./mongod(_ZN5mongo14_initAndListenEi+0x389) [0x6d3309]
./mongod(_ZN5mongo13initAndListenEi+0x1d) [0x6d3b9d]
./mongod [0x6da46d]
./mongod(main+0x9) [0x6dc1d9]
/lib/libc.so.6(__libc_start_main+0xfd) [0x7f95e65ddabd]
./mongod(__gxx_personality_v0+0x499) [0x6cc9c9]
Sat Feb 16 07:04:41.102 [initandlisten] exception in initAndListen: 13651 Couldn't fsync directory '/mnt/hgfs/data/db/bug': errno:22 Invalid argument, terminating
Sat Feb 16 07:04:41.102 dbexit:
Sat Feb 16 07:04:41.102 [initandlisten] shutdown: going to close listening sockets...

Comment by Eliot Horowitz (Inactive) [ 16/Feb/13 ]

Can you post stack + git version?
Its almost impossible for it to be this.

Comment by Alvin Richards (Inactive) [ 16/Feb/13 ]

reduces to < 1024 files, same stack trace.

Comment by Alvin Richards (Inactive) [ 16/Feb/13 ]

OS-X version not availbale (yet) to re-tried on Linux, but got the following on startup

Sat Feb 16 06:07:21.065 [initandlisten] MongoDB starting : pid=2063 port=27017 dbpath=/mnt/hgfs/data/db/bug 64-bit host=alvin-ubuntu910-server64
Sat Feb 16 06:07:21.065 [initandlisten] db version v2.4.0-rc1-pre-, pdfile version 4.5
Sat Feb 16 06:07:21.065 [initandlisten] git version: b73eb512ad5429bce3e796e53eaeb56a70e0140a
Sat Feb 16 06:07:21.065 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49
Sat Feb 16 06:07:21.065 [initandlisten] allocator: tcmalloc
Sat Feb 16 06:07:21.065 [initandlisten] options:

{ dbpath: "/mnt/hgfs/data/db/bug", logpath: "/mnt/hgfs/data/db/bug/server.log", noprealloc: true, smallfiles: true }

Sat Feb 16 06:07:21.069 [initandlisten] Assertion: 13651:Couldn't fsync directory '/mnt/hgfs/data/db/bug': errno:22 Invalid argument
0xdc1be1 0xd832ab 0xd837ec 0x9211a3 0x9e5d51 0x6d3309 0x6d3b9d 0x6da46d 0x6dc1d9 0x7f84fec35abd 0x6cc9c9

That directory path and permission are all fine. Checking setup.

Comment by Alvin Richards (Inactive) [ 15/Feb/13 ]

Downloads page still has 2013-02-12 build. Waiting for a build with at least git commit 2d71e6ab2d42c3c1cc31d42e579dec8041f6cca4 on OS-X.

Comment by auto [ 14/Feb/13 ]

Author:

{u'date': u'2013-02-12T23:56:35Z', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: SERVER-8521: clear temp collections when a database is opened, not server startup
Branch: master
https://github.com/mongodb/mongo/commit/f62bc8177b52420e7dc4d0086b6902ccd188f725

Comment by auto [ 14/Feb/13 ]

Author:

{u'date': u'2013-02-12T17:16:35Z', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: SERVER-8521: remove Database::_openAllFiles
Branch: master
https://github.com/mongodb/mongo/commit/93b0ecaf65b24109717981cc32be3fe153d33e2e

Generated at Thu Feb 08 03:17:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.