Set the socket to non blocking, then try to do sync reads/writes and check for would_block. If the operation would block, fall back to async with timers.

Hopefully this will improve the performance.