9

I am running a Java application with a maximum heap size of 128 MB (-Xmx128M). It is running to successful completion with no OutOfMemoryError, or any other unhandled exception. Therefore, I am assuming that its actual heap size did stay within the declared limit of 128 MB.

However, when observing the process for this Java application, I am seeing a peak total memory usage of 4,188,548 KB (~4 GB). This is a growth of more than 30 times the controlled maximum size of the heap. Although I understand that this value includes virtual memory allocated that may be significantly greater than the actual physical memory used, it affects hard limits such as those imposed by Sun Grid Engine, and therefore it is meaningful.

How exactly is this possible? I understand that the total memory consumed by the JVM includes quite a bit more than the size of the heap, but I do not understand how it could need several GB of extra memory beyond what the application actually needs to create its objects and perform its computation.

I am using Sun Java 1.6.0.31, on a 64-bit RHEL Linux distribution.

5
  • 2
    Are you creating an unbounded number of new Thread()s? Each native thread you spawn has a stack of memory that is not part of the java heap.
    – Affe
    Mar 15, 2012 at 18:17
  • Memory usage by what measure? Allocated address space that doesn't have any actual content mapped to it probably isn't what you care about; be sure you're measuring RSS, not VIRT. Mar 15, 2012 at 18:24
  • @Affe It is not a threading issue. The application is single-threaded.
    – jjcarver
    Mar 15, 2012 at 18:33
  • @Charles I am using "pmap -x <pid>" to measure total memory usage. This value does correspond to "VIRT" when using top. This is important, as I mentioned, since SGE will kill jobs whose "VIRT" value exceeds its "h_vmem" threshold.
    – jjcarver
    Mar 15, 2012 at 18:33
  • @jjcarver ...then there are a lot of things which can impact virtual size, including memory-mapped IO. I wouldn't describe what you're trying to reduce the use of as "memory", though, but rather "address space". Mar 15, 2012 at 18:47

1 Answer 1

4

There are several memory sinks besides the Java heap controlled by -Xmx:

  • Thread stacks
  • PermGen space
  • direct ByteBuffers and mapped ByteBuffer
  • memory allocated by native code / libraries

Without knowing details of your system I would guess, that something uses mapped ByteBuffers.

But you could dig into the issue by examing the output of the pmap command. It lists all memory regions of the process together with the filenames any region is mapped to (if the regions is mapped of course).

2
  • 1
    That's a great suggestion, although in the pmap output, unfortunately all of the seriously offending memory regions have a "Mapping" column of "anon", e.g. 000000052d4c0000 7722240 0 0 rw--- [ anon ]
    – jjcarver
    Mar 15, 2012 at 19:28
  • anon regions reduce the above list to "native libs" and to "direct ByteBuffers".
    – A.H.
    Mar 15, 2012 at 20:46

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.