Attempting to serialize a structure containing a type BSON doesn't know to serialize creates a segmentation fault within the C extension. This behavior extends to any attempt to insert a record into MongoDB using the Mongo Ruby driver if the hash key is a non-primitive Ruby object.

I believe that this operation should raise a Ruby exception so that developers can catch and respond to the error without their application crashing.

An IRB session follows demonstrating both a success case and a crash case:

[12:34]/projects/aarweb(groups)$ irb

>> require 'mongo'
=> true

>> BSON.serialize(:foo => 'bar')
=> #<ByteBuffer:0x00000100c768e8 @buf="\x12\x00\x00\x00\x02foo\x00\x04\x00\x00\x00bar\x00\x00", @cursor=18, @order=:little_endian, @int_pack_order="V", @double_pack_order="E">

>> o = Object.new
=> #<Object:0x00000100c8f900>

>> BSON.serialize(:object => o)
/opt/local/lib/ruby/gems/1.9.1/gems/mongo-0.18.1/lib/mongo/util/bson_c.rb:5: [BUG] Segmentation fault
ruby 1.9.1p376 (2009-12-07 revision 26041) [i386-darwin10]

– control frame ----------
c:0025 p:---- s:0093 b:0093 l:000092 d:000092 CFUNC :serialize
c:0024 p:0032 s:0088 b:0087 l:000086 d:000086 METHOD /opt/local/lib/ruby/gems/1.9.1/gems/mongo-0.18.1/lib/mongo/util/bson_c.rb:5
c:0023 p:0022 s:0082 b:0082 l:001a88 d:000081 EVAL (irb):4
c:0022 p:---- s:0080 b:0080 l:000079 d:000079 FINISH
c:0021 p:---- s:0078 b:0078 l:000077 d:000077 CFUNC :eval
c:0020 p:0027 s:0071 b:0071 l:000070 d:000070 METHOD /opt/local/lib/ruby/1.9.1/irb/workspace.rb:80
c:0019 p:0031 s:0064 b:0063 l:000062 d:000062 METHOD /opt/local/lib/ruby/1.9.1/irb/context.rb:218
c:0018 p:0030 s:0058 b:0058 l:0017e8 d:000057 BLOCK /opt/local/lib/ruby/1.9.1/irb.rb:157
c:0017 p:0039 s:0050 b:0050 l:000049 d:000049 METHOD /opt/local/lib/ruby/1.9.1/irb.rb:271
c:0016 p:0011 s:0045 b:0045 l:0017e8 d:000044 BLOCK /opt/local/lib/ruby/1.9.1/irb.rb:154
c:0015 p:0132 s:0041 b:0041 l:000024 d:000040 BLOCK /opt/local/lib/ruby/1.9.1/irb/ruby-lex.rb:244
c:0014 p:---- s:0038 b:0038 l:000037 d:000037 FINISH
c:0013 p:---- s:0036 b:0036 l:000035 d:000035 CFUNC :loop
c:0012 p:0009 s:0033 b:0033 l:000024 d:000032 BLOCK /opt/local/lib/ruby/1.9.1/irb/ruby-lex.rb:230
c:0011 p:---- s:0031 b:0031 l:000030 d:000030 FINISH
c:0010 p:---- s:0029 b:0029 l:000028 d:000028 CFUNC :catch
c:0009 p:0023 s:0025 b:0025 l:000024 d:000024 METHOD /opt/local/lib/ruby/1.9.1/irb/ruby-lex.rb:229
c:0008 p:0042 s:0022 b:0022 l:0017e8 d:0017e8 METHOD /opt/local/lib/ruby/1.9.1/irb.rb:153
c:0007 p:0011 s:0019 b:0019 l:000b38 d:000018 BLOCK /opt/local/lib/ruby/1.9.1/irb.rb:70
c:0006 p:---- s:0017 b:0017 l:000016 d:000016 FINISH
c:0005 p:---- s:0015 b:0015 l:000014 d:000014 CFUNC :catch
c:0004 p:0174 s:0011 b:0011 l:000b38 d:000b38 METHOD /opt/local/lib/ruby/1.9.1/irb.rb:69
c:0003 p:0039 s:0006 b:0006 l:001a88 d:0002a8 EVAL /opt/local/bin/irb:12
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:001a88 d:001a88 TOP
---------------------------
– Ruby level backtrace information-----------------------------------------
/opt/local/lib/ruby/gems/1.9.1/gems/mongo-0.18.1/lib/mongo/util/bson_c.rb:5:in `serialize'
/opt/local/lib/ruby/gems/1.9.1/gems/mongo-0.18.1/lib/mongo/util/bson_c.rb:5:in `serialize'
(irb):4:in `irb_binding'
/opt/local/lib/ruby/1.9.1/irb/workspace.rb:80:in `eval'
/opt/local/lib/ruby/1.9.1/irb/workspace.rb:80:in `evaluate'
/opt/local/lib/ruby/1.9.1/irb/context.rb:218:in `evaluate'
/opt/local/lib/ruby/1.9.1/irb.rb:157:in `block (2 levels) in eval_input'
/opt/local/lib/ruby/1.9.1/irb.rb:271:in `signal_status'
/opt/local/lib/ruby/1.9.1/irb.rb:154:in `block in eval_input'
/opt/local/lib/ruby/1.9.1/irb/ruby-lex.rb:244:in `block (2 levels) in each_top_level_statement'
/opt/local/lib/ruby/1.9.1/irb/ruby-lex.rb:230:in `loop'
/opt/local/lib/ruby/1.9.1/irb/ruby-lex.rb:230:in `block in each_top_level_statement'
/opt/local/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `catch'
/opt/local/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `each_top_level_statement'
/opt/local/lib/ruby/1.9.1/irb.rb:153:in `eval_input'
/opt/local/lib/ruby/1.9.1/irb.rb:70:in `block in start'
/opt/local/lib/ruby/1.9.1/irb.rb:69:in `catch'
/opt/local/lib/ruby/1.9.1/irb.rb:69:in `start'
/opt/local/bin/irb:12:in `<main>'

– C level backtrace information -------------------------------------------

[NOTE]
You may encounter a bug of Ruby interpreter. Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

Abort trap
[12:35]/projects/aarweb(groups)$