Streamreader skips lines

[muro]

Dear friends,

i have got this code where the Streamreader sometimes reads the file fine and sometimes skips a few lines and i have absoloutely no idea why it is doing this.

            FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://myfile.txt");
            request.Method = WebRequestMethods.Ftp.DownloadFile;
            request.Credentials = new NetworkCredential("user, "pw");
            FtpWebResponse response = (FtpWebResponse)request.GetResponse();
            Stream responseStream = response.GetResponseStream();
            StreamReader reader = new StreamReader(responseStream);

            while (!reader.EndOfStream)
            {
                MessageBox.Show(reader.ReadLine());
            }
            response.Close();
            reader.Close();

 

[NoUserHere]

According to Johns link, the endings of the file do not matter, which appears to be correct when being parsed by the stream reader and ruled out by Microsoft docs.

I've attached the file I used.

I thought the file being downloaded was being disposed of by the file stream completing the download, whilst the reader was reading, thus would throw a exception about the stream. But then I thought, surely to god Microsoft would allow the end user to manage and dispose of their own streams. Right? And it did dawn on me that the file might be in some way responsible for the error, but I didn't and don't have enough time to play with it right now to find out.

So that's interesting that it would throw a wobbler over a empty line. But there are other empty lines, so why didn't it throw on one of them. Why the last one? If you have time for deeper research, I'd love to know why. I still think something else is probably causing it. How many times did you test it?

Btw, my file has no empty last line. ?

 

[Skydiver]

Here's the reason for the exception thrown by StreamReader.EndOfStream:
StreamReader.ReadLine() calls its own ReadBuffer() as part of its work to build up a line of text to return. ReadBuffer() calls the stream's Read() method as seen is the relevant code here:

        internal virtual int ReadBuffer() {
            charLen = 0;
            charPos = 0;

            if (!_checkPreamble)
                byteLen = 0;
            do {
                if (_checkPreamble) {
                    Contract.Assert(bytePos <= _preamble.Length, "possible bug in _compressPreamble.  Are two threads using this StreamReader at the same time?");
                    int len = stream.Read(byteBuffer, bytePos, byteBuffer.Length - bytePos);
                    Contract.Assert(len >= 0, "Stream.Read returned a negative number!  This is a bug in your stream class.");
            :

In this case, the stream's Read() implementation it is FtpDataStream.Read():

        public override int Read(byte[] buffer, int offset, int size) {
            CheckError();
            int readBytes;
            try {
                readBytes = m_NetworkStream.Read(buffer, offset, size);
            } catch {
                CheckError();
                throw;
            }
            if (readBytes == 0)
            {
                m_IsFullyRead = true;
                Close();
            }
            return readBytes;
        }

Notice how it decides to call Close() on itself which disposes itself when zero bytes are read.

Update after: Oops. I forgot to mention that ReadLine() essentially keeps calling ReadBuffer() until a line is read, or no more bytes can be read in. Since the line ends at the end of file, the call to ReadBuffer() at the end of the do-while induces the FTP data stream class to read zero bytes.

StreamReader.EndOfStream also calls ReadBuffer() and will return true if zero bytes are read. Unfortunately, as we've seen above, ReadBuffer() will end up calling the stream's Read() method -- kaboom!

 

[Skydiver]

Looks to be a bug in the FtpDataStream class. The regular MemoryStream doesn't suffer from the same problem. Here's the console code from post #15 with an extra step to copy from the FTP data stream over into a memory stream.

Spoiler

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading.Tasks;

namespace TestFtp
{
    class Program
    {
        static void Main(string[] args)
        {
            FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://localhost/x64.txt");
            request.Method = WebRequestMethods.Ftp.DownloadFile;
            request.Credentials = new NetworkCredential("user", "password");
            FtpWebResponse response = (FtpWebResponse)request.GetResponse();
            Stream responseStream = response.GetResponseStream();

            Stream copyStream = new MemoryStream();
            responseStream.CopyTo(copyStream);
            copyStream.Seek(0, SeekOrigin.Begin);

            StreamReader reader = new StreamReader(copyStream);

            while (!reader.EndOfStream)
            {
                Console.WriteLine(reader.ReadLine());
            }
            response.Close();
            reader.Close();
        }
    }
}

 

[NoUserHere]

I will dig into this some night. Some night when i have free time. So after all, my hunch was confirmed, something was disposing, and it happened to be the stream from the FTP. Correct?

I'll bookmark this for a re-read during the week. Good job

 

[Skydiver]

Here's a way to reproduce the problem without requiring the special file without a blank line at the end:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading.Tasks;

namespace TestFtp
{
    class Program
    {
        static void Main()
        {
            FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://localhost/x64.txt");
            request.Method = WebRequestMethods.Ftp.DownloadFile;
            request.Credentials = new NetworkCredential("user", "password");
            FtpWebResponse response = (FtpWebResponse)request.GetResponse();
            Stream responseStream = response.GetResponseStream();

            var buffer = new byte[1];

            // Read zero bytes
            responseStream.Read(buffer, 0, 0);

            // Now try to read 1 byte
            responseStream.Read(buffer, 0, 1);
        }
    }
}

 

[Skydiver]

I've reported the bug to Microsoft and has been triaged. Now the question is if MS will fix or just leave it alone.

 

[muro]

Wow thanks. I am using anothe rmethod now. I just download the file and then read it out. Works good. Thanks everyone.