The WiredTiger btree cursor search and remove functions check the size of the key to ensure it's not too large. Looking over the code paths, I don't think there's any reason we need to check the size of the key, if it's too large, the search will fail, and that's OK.