3

i am working on a C# application and this application is facing memory crunch, because many objects are getting memory allocation in the Large object Heap.

My C# application has to work on many large file (as an string object) and therefore the memory for this string type object is getting allocated again and again from the Large object heap (thus leading to LOH fragmentation).

Since string is an immutable object, a new memory in LOH is always assigned to this object. My question is, Is there a way, I can pre-allocate some memory in the large object heap and always allocate the same memory to the string object.

Here is the thing in more detail: As I mentioned, I am doing processing on these large files. To do the processing , I have to convert it into the string. Even if I use stringBuilder, it won't be much help, because as soon as I convert it into the String a separate memory for this get allocated in the LOH.

So, I was expecting to allocate a bunch of let's say 100 KB in the memory and whenever I read a new file and convert it into string, these 100 KB gets allocated.

7
  • 2
    Consider the use of StringBuilder which is basically a mutable string. Jun 11, 2011 at 13:06
  • 5
    why are you allocating the same string over and over again? If you've created it once then just pass it to all consumers.
    – Andrei
    Jun 11, 2011 at 13:06
  • 2
    Why do you think LOH fragmentation is your problem?
    – svick
    Jun 11, 2011 at 13:12
  • 1
    Why is your code reading all file content in memory at once? Can't you read the file in smaller chunks?
    – Dan
    Jun 11, 2011 at 14:28
  • It's not the same string, since i am processing different file, the content would be different.
    – Kunal
    Jun 12, 2011 at 11:46

3 Answers 3

3

You should use the StringBuilder object to work with strings, as it is a mutable string and provide many methods to work with it.

You are saying that every strings are created again and again. If, for some reason, you must use the strings, just Intern them - they will stored once, and will not be added on memory heap again

2
  • 2
    Interning will only work it it's about many times the same string (from file(s)). Not too likely. Jun 11, 2011 at 13:54
  • I've just say all the variants
    – VMAtm
    Jun 11, 2011 at 16:04
1

have you considered using a memory mapped file? http://msdn.microsoft.com/en-us/library/dd997372.aspx

i dont know what you do with the strings, but is there a reason not to work with streams?

1

It's unclear what kind of actions you want to do with the strings, your best shot is an estimate of the final result(s). Use a pessimistic (large) estimate and maybe add a margin.

Then create a StringBuilder with the capacity parameter:

var buffer = new StringBuilder(LargestExpectedSize);

This essentially is pre-allocation on the LOH. Avoid (auto-)growing of StringBuilders or List<>s

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.