[CDRIVER-549] mongoc_init() and mongoc_cleanup() may make memory “still reachable” Created: 27/Feb/15  Updated: 03/May/17  Resolved: 17/Aug/15

Status: Closed
Project: C Driver
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Chengcheng Pei Assignee: Hannes Magnusson
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to CDRIVER-984 Cleanup openssl properly Closed

 Description   

I have a class to do mongodb operation using mongodb-c-driver. In the constructor, mongoc_init() is called. In deconstructor, mongoc_cleanup() is called. Then, valgrind 3.10.1 told me that some memory "still reachable". Part of the valgrind output is following:

==23222== 16,384 bytes in 1 blocks are still reachable in loss record 586 of 586

==23222== at 0x4C2D199: realloc (vg_replace_malloc.c:692)

==23222== by 0x6243894: CRYPTO_realloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)

==23222== by 0x62BE1F1: lh_insert (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)

==23222== by 0x62C0828: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)

==23222== by 0x62C0243: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)

==23222== by 0x5FCB01D: ERR_load_SSL_strings (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)

==23222== by 0x5083623: _mongoc_ssl_init (in /usr/local/lib/libmongoc-1.0.so.0.0.0)

==23222== by 0x5077028: _mongoc_do_init (in /usr/local/lib/libmongoc-1.0.so.0.0.0)

==23222== by 0x529A3FF: pthread_once (pthread_once.S:104)

==23222== by 0x400F305: call_init.part.0 (dl-init.c:85)

==23222== by 0x400F3DE: call_init (dl-init.c:52)

==23222== by 0x400F3DE: _dl_init (dl-init.c:134)

==23222== by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)

==23222== LEAK SUMMARY:

==23222== still reachable: 91,832 bytes in 3,075 blocks

When I commented these two functions, mongoc_init() and mongoc_cleanup(), valgrind told me that All heap blocks were freed.

It seems that it is the problem of the mongodb-c-driver. Others reported the same issue under mongodb c driver project. https://jira.mongodb.org/browse/CDRIVER-478

Any idea?

Thanks.



 Comments   
Comment by Hannes Magnusson [ 17/Aug/15 ]

That is correct cpei.

Valgrind and OpenSSL are known for not getting very well along.

The specific block that is causing the "still reachable" above can we resolved by your application – but is not something we can do in mongoc automatically as it can be "dangerous" to do without knowing what exactly your application will do next (or even how/why you are calling mongoc_cleanup()).

If you want to resolve this at your application level, you can call use the cleanup routines Jason mentions – but even so, it might not catch 100% of OpenSSL reachable blocks (even though the one above would go away, another one might show up).
Due to the nature of OpenSSL being very security sensitive, we would prefer not to aggressively attempt to cleanup every last byte which OpenSSL itself does not cleanup in normal circumstances. Doing so could easily cause more harm then good.

We provide our own valgrind suppression file which we use for our testing.
These are things that there isn't actually anything we can do to fix.
See: https://github.com/mongodb/mongo-c-driver/blob/master/valgrind.suppressions

Comment by Chengcheng Pei [ 04/Mar/15 ]

Do you mean I should not do anything. Because, the memory will be cleaned up by OS. and Because it is not allocated on a per operation basis, the size of memory "still reachable" will not increase.

Am I right?

Thank you for your help.

Comment by Mira Carey [ 03/Mar/15 ]

I think this issue may be slightly different from that one.

What you're seeing with "still reachable" memory coming out of valgrind is allocations made by openssl. While we could take the time to deallocate that memory in cleanup, it's somewhat dangerous (the deallocation routines aren't thread safe) and takes longer than letting the OS cleanup itself. This memory isn't actually "leaked" per se (openssl still has pointers to it internally), isn't allocated on a per operation basis and shouldn't make a difference to your application, unless you're on an OS that can't reclaim memory on process exit.

My best guess for the kind of thing you'd call in cleanup is:

CONF_modules_unload(1);
OBJ_cleanup();
EVP_cleanup();
CRYPTO_cleanup_all_ex_data();
ERR_remove_thread_state(0);
ERR_free_strings();

But even that doesn't get everything.

Comment by Chengcheng Pei [ 02/Mar/15 ]

C Driver
CDRIVER-478

mongoc_collection memory leak reported by Xcode Instruments

I think this is the similar issue under Xcode.

Comment by Chengcheng Pei [ 02/Mar/15 ]

OK. In order to debug, I commented most of the codes. The below is my current version. I just use mongoc_init () and mongoc_cleanup(). But there is still the same problem.

#include <iostream>
#include <string.h>
#include <mongoc.h>
 
#include "mongoconnection.h"
 
 
MONGOConnection::MONGOConnection(std::string dbURI, 
	std::string dbName, std::string dbTableName) : 
	m_client(NULL), m_collection(NULL), 
	m_dbURI(dbURI), m_dbName(dbName), 
	m_dbTableName(dbTableName)
{
	Connect(m_dbURI, m_dbName, m_dbTableName);
}
 
MONGOConnection::~MONGOConnection()
{
	Disconnect();
}
 
bool 
MONGOConnection::Connect(
	std::string dbURI, 
	std::string dbName, 
	std::string dbTableName)
{
 
	m_dbURI = dbURI;
	m_dbName = dbName;
	m_dbTableName = dbTableName;
	
	mongoc_init ();
 
	const char *p_dbURI = dbURI.c_str();
	const char *p_dbName = dbName.c_str();
	const char *p_dbTableName = dbTableName.c_str();
	/* Codes to connect to Mongo DB */
	
	return true;
}
 
void
MONGOConnection::Disconnect()
// destroy m_client, destroy m_collection, and cleanup.
{
	if( m_collection )
	{
		mongoc_collection_destroy(m_collection);
	}
 
 
	if( m_client )
	{
		mongoc_client_destroy(m_client);
	}
 
	mongoc_cleanup();
}

Comment by A. Jesse Jiryu Davis [ 28/Feb/15 ]

Thanks for the report. Would you mind sharing your code with us, ideally a minimal source file that demonstrates this issue?

Generated at Wed Feb 07 21:09:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.