The IByteBuffer interface represents a logical byte buffer that might be backed by various means (contiguous, segmented, using buffers from a pool or not, etc...).

But the IByteBuffer interface probably has too many methods. Many of the methods could be moved out to extension methods on the interface (so they only need to be implemented once).

A new streamlined interface that deals only with the buffer abstraction could look like this:

public interface IByteBuffer : IDisposable
{
    // properties
    int Capacity { get; }
    int Length { get; set; }

    // methods
    void EnsureCapacity(int minimumCapacity);
    ArraySegment<byte> AccessBackingBytes(int offset, int limit);
}

This reduces the responsibility of this interface to a single responsibility, namely to represent a (possibly) segmented variable length byte array.