3

I'm currently building an application that is, among other things, going to download large files from a FTP server. Everything works fine for small files (< 50 MB) but the files I'm downloading are way bigger, mainly over 2 GB.

I've been trying with a Webclient using DownloadfileAsync() and a list system as I'm downloading these files one after the other due to their sizes.

DownloadClient.DownloadProgressChanged += new DownloadProgressChangedEventHandler(DownloadProgress);
DownloadClient.DownloadFileCompleted += new AsyncCompletedEventHandler(DownloadCompleted);

private void FileDownload()
{
    DownloadClient.DownloadFileAsync(new Uri(@"ftp://" + RemoteAddress + FilesToDownload[0]), LocalDirectory + FilesToDownload[0]));
}

private void DownloadProgress(object sender, DownloadProgressChangedEventArgs e)
{
    // Handle progress
}

private void DownloadCompleted(object sender, AsyncCompletedEventArgs e)
{
    FilesToDownload.RemoveAt(0);
    FileDownload();
}

It works absolutely fine this way on small files, they are all downloaded one by one, the progress is reported and DownloadCompleted fires after each file. This issue I'm facing with big files is that it launches the first download without any issue but doesn't do anything after that. The DownloadCompleted event never fires for some reasons. It looks like the WebClient doesn't know that the file has finished to download, which is an issue as I'm using this event to launch the next download in the FilesToDownload list.

I've also tried to do that synchronously using WebClient.DownloadFile and a for loop to cycle through my FilesToDownload list. It downloads the first file correctly and I get an exception when the second download should start: "The underlying connection was closed: An unexpected error occurred on a receive".

Finally, I've tried to go through this via FTP using edtFTPnet but I'm facing download speed issues (i.e. My download goes full speed with the WebClient and I just get 1/3 of the full speed with edtFTPnet library).

Any thoughts? I have to admit that I'm running out of ideas here.

3
  • Try creating a filestream or a MemoryStream to do this by getting the bytes and when you say "large" what is your definition of large in reference to GIGS or MB's
    – MethodMan
    Dec 6, 2011 at 20:54
  • My larges files are over 2 gigs.
    – Xeaz
    Dec 6, 2011 at 21:16
  • remember operating system deals with file sizes of 2 gig but 4 gig limit are if you use .NET 4.0 Microsoft Changed the file size limit in .NET 4.0 so u will have to change your int sizes to be longs or Int64 type can you paste the exact example you are using.. also gigs of data will always take longer . keep in mind swap files and tmp files it's not just a straight copy you have to have the available storage space on the target media
    – MethodMan
    Dec 6, 2011 at 21:26

2 Answers 2

0
public string GetRequest(Uri uri, int timeoutMilliseconds)
{
     var request = System.Net.WebRequest.Create(uri);
     request.Timeout = timeoutMilliseconds;
     using (var response = request.GetResponse())
     using (var stream = response.GetResponseStream())
     using (var reader = new System.IO.StreamReader(stream))
     {
         return reader.ReadToEnd();
     }
 } 
0

Forgot to update this thread but I figured how to sort this out a while ago.

The issue was that the Data connection that is opened for a file transfer randomly times out for some reason or is closed by the server before the transfer ends. I haven't been able to figure out why however as there is a load of local and external network interfaces between my computer and the remote server. As it's totally random (i.e the transfer works fine for five files in a row, times out for one file, works fine for the following files etc), the issue may be server or network related.

I'm now catching any FTP exception raised by the FTP client object during the download and issue a REST command with an offset equals to the position in the data stream where the transfer stopped (i.e total bytes amount of the remote file - currently downloaded bytes amount). Doing so allows to get the remaining bytes that are missing in the local file.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.