__session_open_cursor has many special cases. The most typical 99% or more path should be opening a "table:" cursor, and getting a cached cursor. Something like this could be at the top of the function:

     if (to_dup == NULL && uri != NULL) {
        __wt_cursor_get_hash(session, uri, to_dup, &hash_value);
        if ((ret = __wt_cursor_cache_get(session, uri, hash_value, to_dup, cfg, &cursor)) == 0)
            goto got_cached;
     }

Where got_cached returns quickly (the done label makes an extra check which would not apply in this case).

This would probably get rid of 4-5 branches.