[CDRIVER-110] support sending mongo messages on network without additional malloc where available Created: 23/Jan/12 Updated: 11/Nov/13 Resolved: 11/Nov/13 |
|
| Status: | Closed |
| Project: | C Driver |
| Component/s: | None |
| Affects Version/s: | 0.4 |
| Fix Version/s: | 0.90.0 |
| Type: | Improvement | Priority: | Minor - P4 |
| Reporter: | Gifford Hesketh | Assignee: | Gary Murakami |
| Resolution: | Done | Votes: | 0 |
| Labels: | performance | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Linux, Windows |
||
| Description |
|
I understand that the C driver has broad compatibility as a specific goal, but easy platform-specific optimizations can be very useful. Functions like mongo_insert() currently include a call to mongo_message_create() which then calls bson_malloc(). This supports the requirement to transmit the existing bson data prefixed by the collection name in a single send() call on the socket. Linux and Windows both support the required network semantics without the additional bson_malloc() by using vector-oriented send() equivalents. Here is a simplified example : int mongo_insert_bson( mongo * pMongo, const char * pCollection, bson * pBSON ) int nBSON = *( ( int32_t * ) pBSON->data ) ; header.len = sizeof( header ) + sizeof( ZERO ) + nCollection + nBSON ; #if defined( _WIN32 ) pSend[ 0 ].len = sizeof( header ) ; if( 0 != WSASend( pMongo->sock, pSend, 1, &( nSent ), 0, 0, 0 ) ) { pMongo->err = MONGO_IO_ERROR ; return MONGO_ERROR ; }pSend[ 0 ].len = sizeof( ZERO ) ; pSend[ 0 ].buf = ( char * ) &( ZERO ) ; pSend[ 1 ].len = nCollection ; pSend[ 1 ].buf = ( char * ) pCollection ; pSend[ 2 ].len = nBSON ; pSend[ 2 ].buf = ( char * ) pBSON->data ; if( 0 != WSASend( pMongo->sock, pSend, 3, &( nSent ), 0, 0, 0 ) ) { pMongo->err = MONGO_IO_ERROR ; return MONGO_ERROR ; } return MONGO_OK ; #if defined( linux ) ssize_t nSent ; pSend[ 0 ].iov_len = sizeof( header ) ; if( pSend[ 0 ].iov_len = sizeof( ZERO ) ; pSend[ 0 ].iov_base = &( ZERO ) ; pSend[ 1 ].iov_len = nCollection ; pSend[ 1 ].iov_base = pCollection ; pSend[ 2 ].iov_len = nBSON ; pSend[ 2 ].iov_base = pBSON->data ; if( { pMongo->err = MONGO_IO_ERROR ; return MONGO_ERROR ; } return MONGO_OK ; #else #endif // defined( linux ) || defined( _WIN32 ) } // mongo_insert() Incorporating this kind of optimization generically in the mongo C driver probably requires a fair amount of well-considered refactoring, so I am not just sending in a hasty patch. I am, however, more than happy to collaborate on this effort. |
| Comments |
| Comment by Christian Hergert [ 11/Nov/13 ] |
|
Hi, The new C driver[1] uses scatter/gather I/O (with sendmsg()/recvmsg()) for communication with the MongoDB server. In many cases it will not malloc during the send phase of the message. Additionally, the recv phase of an operation opportunistically buffers to try to reduce the number of syscalls. |
| Comment by Kyle Banker [ 23/Jan/12 ] |
|
Gifford, Thanks for the idea. I think it'd be worthwhile to set up a simple benchmark to see exactly how much performance is improved. A non-trivial increase would obviously make the refactoring worthwhile. Kyle |
| Comment by Gifford Hesketh [ 23/Jan/12 ] |
|
Sorry the sample code looks so ugly. The formatting got lost in translation ... |