[CDRIVER-2810] misaligned address Created: 30/Aug/18  Updated: 06/Sep/18  Resolved: 06/Sep/18

Status: Closed
Project: C Driver
Component/s: libbson, libmongoc
Affects Version/s: 1.12.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Adrian Imboden Assignee: A. Jesse Jiryu Davis
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Ubuntu 18.04
Clang 6.0.0
libc++

libasan build


Attachments: Text File out.txt    
Issue Links:
Duplicate
duplicates CDRIVER-596 Figure out what to do about bson_t al... Closed
Related
related to CDRIVER-2813 Remove BSON_EXTRA_ALIGNMENT option Backlog

 Description   

I updated the mongo-cxx-driver, mongo-c-driver and libbson (which is now part of mongo-c-driver) to the newest versions.

The mongo-c-driver is now version 1.12 (changeset a690091bae086f267791bd2227400f2035de99e8).

At our company, we use a self built toolchain:

  • Clang 6.0.0 with libc++

We test our software with many sanitizers. UBSan is one of them. Since the update, I get the following error:

/src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-read-prefs.c:29:16: runtime error: member access within misaligned address 0x61300000ffc0 for type 'mongoc_read_prefs_t' (aka 'struct _mongoc_read_prefs_t'), which requires 128 byte alignment
0x61300000ffc0: note: pointer points here
 0e 00 00 4a  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00
              ^ 
    #0 0xb644d52 in mongoc_read_prefs_new /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-read-prefs.c:29:16
    #1 0xb7b4e98 in mongoc_uri_new_with_error /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-uri.c:1443:22
    #2 0xb7b7f45 in mongoc_uri_new /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-uri.c:1493:10
    #3 0xb494873 in mongocxx::v_noabi::uri::uri(bsoncxx::v_noabi::string::view_or_value) /src/mongo-cxx-driver/src/mongocxx/uri.cpp:51:37
... more application specific backtraceSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-read-prefs.c:29:16 in 
/src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-read-prefs.c:29:16: runtime error: store to misaligned address 0x61300000ffc0 for type 'mongoc_read_mode_t', which requires 128 byte alignment
0x61300000ffc0: note: pointer points here
 0e 00 00 4a  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00
              ^ 
    #0 0xb644d86 in mongoc_read_prefs_new /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-read-prefs.c:29:21
    #1 0xb7b4e98 in mongoc_uri_new_with_error /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-uri.c:1443:22
    #2 0xb7b7f45 in mongoc_uri_new /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-uri.c:1493:10
    #3 0xb494873 in mongocxx::v_noabi::uri::uri(bsoncxx::v_noabi::string::view_or_value) /src/mongo-cxx-driver/src/mongocxx/uri.cpp:51:37
... more application specific backtrace
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-server-stream.c:48:57 in 
/src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-server-stream.c:49:37: runtime error: member access within misaligned address 0x6130000202c0 for type 'mongoc_server_stream_t' (aka 'struct _mongoc_server_stream_t'), which requires 128 byte alignment
0x6130000202c0: note: pointer points here
 0a 00 00 30  04 00 00 00 be be be be  80 26 02 00 d0 61 00 00  be be be be be be be be  be be be be
              ^ 
    #0 0xb74bd09 in mongoc_server_stream_cleanup /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-server-stream.c:49:37
    #1 0xb5de014 in _mongoc_cursor_run_command /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-cursor.c:1017:4
    #2 0xb5e50f6 in _mongoc_cursor_response_refresh /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-cursor.c:1582:8
    #3 0xb5eb5db in _prime /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-cursor-find-cmd.c:36:4
    #4 0xb5ea4d2 in _prime /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-cursor-find.c:61:11
    #5 0xb5e04c7 in _call_transition /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-cursor.c:1121:12
    #6 0xb5dfba4 in mongoc_cursor_next /src/mongo-c-driver/src/libmongoc/src/mongoc/mongoc-cursor.c:1189:23
    #7 0xb2bd258 in mongocxx::v_noabi::cursor::iterator::operator++() /src/mongo-cxx-driver/src/mongocxx/cursor.cpp:51:9
    #8 0xb2befd4 in mongocxx::v_noabi::cursor::iterator::iterator(mongocxx::v_noabi::cursor*) /src/mongo-cxx-driver/src/mongocxx/cursor.cpp:79:5
    #9 0xb2be6a6 in mongocxx::v_noabi::cursor::begin() /src/mongo-cxx-driver/src/mongocxx/cursor.cpp:66:12
... more application specific backtrace

hundreds more will come after that.

 

It seems that the memory from bson_malloc0 is not properly aligned for the use case it is being used in this case.



 Comments   
Comment by A. Jesse Jiryu Davis [ 03/Sep/18 ]

Interesting, thanks for sharing that code. So the problem is that malloc guarantees only 16-byte aligned memory. "The address of a block returned by malloc or realloc in GNU systems is always a multiple of eight (or sixteen on 64-bit systems)."

https://www.gnu.org/software/libc/manual/html_node/Aligned-Memory-Blocks.html

libbson and libmongoc made an unfortunate decision a few years ago, we declare a few structs like bson_t as 128-byte aligned. Structs that include a bson_t, such as a mongoc_read_prefs_t, must also be 128-byte aligned as a side-effect. The intent was to enforce efficient use of the CPU cache, but it causes problems like this one. We'll remove the alignment specifier when we make our next ABI-breaking release, libmongoc 2.0: CDRIVER-2813.

Meanwhile, it's best to override the default: execute CMake with -DENABLE_EXTRA_ALIGNMENT=OFF and rebuild the driver.

Comment by Adrian Imboden [ 03/Sep/18 ]

Hi

  1. No, we do not change the memory allocation configuration
  2. With the cmake flag, the warning does not occur anymore
  3. When I add the following to the beginning, then the warning stays the same as before:

static void* my_calloc(size_t n_members, size_t num_bytes) {
    void* p = malloc(n_members * num_bytes);
    std::memset(p, 0, n_members * num_bytes);
    return p;
}
 
int main() {
    bson_mem_vtable_t vtable = {
        malloc,
        my_calloc,
        realloc,
        free,
        {},
    };
    bson_mem_set_vtable(&vtable);
...

As you said, malloc and calloc should already return memory suitable for any alignment.

 

https://en.cppreference.com/w/c/memory/malloc states, that the memory returned is "... allocated memory block that is suitably aligned for any object type with fundamental alignment", which according to the doc is normally https://en.cppreference.com/w/c/memory/aligned_alloc, which is most of the time 4 or 8 bytes depending on the platform.

The warning states, that a 128 byte alignment is expected for the struct mongoc_read_prefs_t (and the others mentioned in the output). I did not check how it gets to a 128 byte alignment, but if I add this code at the beginning of my program, no warnings occur (this example is only C11 compatible, but I think the alignment can be forced by hand as well of course):

static void* my_malloc(size_t num_bytes) {                                                        
    void* p = aligned_alloc(128, num_bytes);                
    return p;                                                     
}                                                             
                                    
static void* my_calloc(size_t n_members, size_t num_bytes) {                                         
    void* p = aligned_alloc(128, n_members * num_bytes);
    std::memset(p, 0, n_members * num_bytes);
    return p;                                                         
}
                                                      
int main() {                                         
    bson_mem_vtable_t vtable = {                               
        my_malloc,                                     
        my_calloc,                   
        realloc,                                          
        free,
        {},
    };
    bson_mem_set_vtable(&vtable);
...

The 128 bytes are arbitrary and should be calculated using the given type or the num_bytes.

 

I hope that helps to find out more about the problem.

Comment by A. Jesse Jiryu Davis [ 01/Sep/18 ]

Hi, bson_malloc0 uses calloc by default. C89 says, "The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object." I have three questions to help us diagnose the issue.

1. Does your custom toolchain alter memory allocation from clang's defaults?

2. What happens if you recompile the driver, running CMake with "-DENABLE_EXTRA_ALIGNMENT=OFF"?

3. What happens if you replace our calloc call with malloc, using code something like this at the very beginning of your program?:

void *my_calloc (size_t n_members, size_t num_bytes) {
   return malloc (n_members * num_bytes);
} 
 
int main() {
   bson_mem_vtable_t vtable = {
      malloc,
      my_calloc,
      reallocf,
      free
   };
 
   bson_mem_set_vtable (&vtable);
   mongoc_init ();
 
   /* the rest of your program */
}

Generated at Wed Feb 07 21:16:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.