1

I want to read a huge .txt file and I'm getting a memory overflow because of its sheer size.

Any help?

    private void button1_Click(object sender, EventArgs e)
    {
        using (var Reader = new StreamReader(@"C:\Test.txt"))
        {
            textBox1.Text += Reader.ReadLine();
        }
    }

Text file is just:

Line1
Line2
Line3

Literally like that.

I want to load the text file to a multiline textbox just as it is, 100% copy.

6
  • 2
    Read and process it one line at a time, or break it into chunks and deal with the chunks individually. You can also show us the code you have, and tell us what you are trying to accomplish with it. May 19, 2010 at 18:10
  • How would I do that? I saw a similar question here in SO but I can't find it. Search doesn't work well for me.
    – delete
    May 19, 2010 at 18:11
  • @Robert If you post this, you'll get an upvote :)
    – Simon
    May 19, 2010 at 18:12
  • @Sergio search tip : try keyword1 +keyword2 etc : )
    – rlb.usa
    May 19, 2010 at 18:12
  • How are you getting a memory overflow when you're reading only one line of text?
    – Ben M
    May 19, 2010 at 18:28

4 Answers 4

3

Firstly, the code you posted will only put the first line of the file into the TextBox. What you want is this:

using (var reader = new StreamReader(@"C:\Test.txt"))
{
    while (!reader.EndOfStream)
        textBox1.Text += reader.ReadLine();
}

Now as for the OutOfMemoryException: I haven't tested this, but have you tried the TextBox.AppendText method instead of using +=? The latter will certainly be allocating a ton of strings, most of which are going to be nearly the length of the entire file by the time you near the end of the file.

For all I know, AppendText does this as well; but its existence leads me to suspect it's put there to deal with this scenario. I could be wrong -- like I said, haven't tested personally.

5
  • So the reader needs a call to EndOfStream to consume the previous line, so that the next line can be read? I didn't know that. Seems a bit counterintuitive. May 19, 2010 at 18:37
  • @Robert Harvey: No, it doesn't need to call EndOfStream to consume the previous line; it just needs to check that property to know when to stop. The issue was that the OP was not utilizing a loop at all; he was continuously creating a new StreamReader and calling ReadLine once per instance.
    – Dan Tao
    May 19, 2010 at 18:40
  • 2
    The += on the text section of the text box, is just a += on a string. That is the worst way to append text to a string in .net. As the size of the text increases the memory usage gets worse and worse.
    – Justin
    May 19, 2010 at 19:28
  • @Justin: I realize that; this is why I suggested trying AppendText instead. But I had to be honest and admit that I'm not really sure AppendText isn't just calling += itself.
    – Dan Tao
    May 19, 2010 at 19:39
  • Tested this with a file of size 7 GB on x64 machine. worked good,
    – Rajat
    Mar 31, 2017 at 8:59
3

You'll get much faster performance with the following:

textBox1.Text = File.ReadAllText(@"C:\Test.txt");

It might also help with your memory problem, since you're wasting an enormous amount of memory by allocating successively larger strings with each line read.

Granted, the GC should be collecting the older strings before you see an OutOfMemoryException, but I'd give the above a shot anyway.

2

First use a rich text box instead of a regular text box. They're much better equiped for the large amounts of data you're using. However you still need to read the data in.

// use a string builer, the += on that many strings increasing in size
// is causing massive memory hoggage and very well could be part of your problem
StringBuilder sb = new StringBuilder();

// open a stream reader
using (var reader = new StreamReader(@"C:\Test.txt"))
{
    // read through the stream loading up the string builder
    while (!reader.EndOfStream)
    {
       sb.Append( reader.ReadLine() );
    }
}

// set the text and null the string builder for GC
textBox1.Text = sb.ToString();
sb = null;
2

Read and process it one line at a time, or break it into chunks and deal with the chunks individually. You can also show us the code you have, and tell us what you are trying to accomplish with it.

Here is an example: C# Read Text File Containing Data Delimited By Tabs Notice the ReadLine() and WriteLine() statements.

TextBox is severely limited by the number of characters it can hold. You can try using the AppendText() method on a RichTextBox instead.

4
  • Robert, it's reading the first line "Line1" correctly, but it doesn't grab Line2 or Line3. I'm trying to create a literal copy of the text file, formatting and all.
    – delete
    May 19, 2010 at 18:17
  • @Sergio: Try the RichTextBox. If the file isn't amazingly large, it should work; RichTextBox is far better suited for this sort of thing. May 19, 2010 at 18:19
  • The ReadLine() method is only returning my first line over and over and over again; how can I tell it to read the first line then continue moving along?
    – delete
    May 19, 2010 at 18:22
  • @Sergio: You need to do this in a while loop, checking the EndOfStream property on each iteration. Take a look at my answer.
    – Dan Tao
    May 19, 2010 at 18:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.