Hi!

When using WT3.0 our application crashes due to failed assertion in __txn_op_log_row_key_check() function when debug-enabled build of WiredTiger is used (built with --enable-diagnostic).
We use tables with custom collator and one of the keys has padding byte which can have random value since it's ignored by collator.

I will attach small reproducer:

$ ./random_key
[1516727132:293604][31814:0x7f157d93a740], file:main.wt, WT_CURSOR.remove: __txn_op_log_row_key_check, 54: key.size == cursor->key.size && memcmp(key.data, cursor->key.data, key.size) == 0
[1516727132:293654][31814:0x7f157d93a740], file:main.wt, WT_CURSOR.remove: aborting WiredTiger library
Aborted (core dumped)