Performance of c# FileStream.Seek vs FileStream.ReadBytes. Why is the latter faster?

Question

In my application I need to frequently move the position pointer of a file stream forward a small number of bytes. Experimentation with using FileStream.ReadBytes rather than FileStream.Seek for this purpose revealed that the former was significantly faster than the latter.

Consider the following minimal example:

    static void OptimizedSeek(Stream theStream, long offset)
    {
        if (offset!= 0)
        {
            if (offset> 0 && offset< 4096)
            {
                theStream.Read(new byte[offset], 0, (int)offset);
            }
            else
            {
                theStream.Seek(offset, SeekOrigin.Current);
            }
        }
    }

    static void Main(string[] args)
    {
        string inputFile = "datafile.dat";

        FileStream myStream = new FileStream(inputFile,FileMode.Open);            

        Random r = new Random();
        Console.WriteLine(myStream.Position);
        var maxLength = myStream.Length;
        int delta = r.Next(0, 1000);
        long pos = delta;
       
        while (pos < maxLength)
        {
            //myStream.Seek(delta, SeekOrigin.Current);
            OptimizedSeek(myStream, delta);
            delta = r.Next(0, 1000);
            pos += delta;
        }

        Console.WriteLine(myStream.Position);

    }

If I instead use myStream.Seek(delta, SeekOrigin.Current) instead of OptimizedSeek, performance is significantly worse (10 time on my system). datafile.dat is here around 800mb.

Anyone with knowledge as to why I see this difference in performance? At the outset I would expect Seek to be optimized for this exact purpose.

I'm using .net5 and compiling for Windows.