12

What is the difference between Metaspace and Native memory?

Metaspace can be tracked using jconsole, jvisualvm, jstat cmds.
It seems Native Memory can be tracked using jcmd. Link

Is metaspace also Native Memory?

Where do NIO buffers get stored? In the metaspace or native memory?

1 Answer 1

28

Is metaspace also Native Memory?

Yes, Metaspace is part of the Native Memory(process memory) and is limited by host operating system

source http://karunsubramanian.com/websphere/one-important-change-in-memory-management-in-java-8/ You can monitor it using jmap -permstat PID. If your application ends up allocating lot of memory in metaspace then it will effect the entire system not just the JVM.That's why its recommended that you use -XX:MetaspaceSize to explicitly set the max metaspace size for your app.

Java NIO APIs use ByteBuffers as the source and destination of I/O calls, and come in two flavours:

  • Heap Byte Buffer (wrap a byte[] array, allocated in the garbage collected Java heap)
  • Direct Byte Buffer (wrap memory allocated outside the Java heap )

Since only "native" memory can be passed to operating system calls, so it won't be moved by the garbage collector, it means that when you use a heap ByteBuffer for I/O, it is copied into a temporary direct ByteBuffer. The JDK caches one temporary buffer per thread, without any memory limits (i.e. an unbounded cache). As a result, if you call I/O methods with large heap ByteBuffers from multiple threads, your process can use a huge amount of additional native memory.

You can refer to this & this article for more details.

Updated

RSS=OffHeap (mapped files, JVM internal code (.bss segments), thread stacks, direct buffers) + GC internal structure+ Heap + Structures used & allocated by native libraries (e.g IO libraries) + Metaspace + Shared Libraries of the JVM + CodeCache

You can use, on linux pmap -x to analyse the memory map of your JVM process. Furthermore, Jemalloc can help to write a profile to disk after every x GB/ x kB of memory allocation/stack trace by setting appropriate MALLOC_CONF env variables. Once you have a generated file try using
Jeprof for visualisation. E.g.

 jeprof --show_bytes --pdf `which w` jeprof.heap > sample.pdf

to generate the PDF for call graphs.

8
  • I am using Tomcat enabled with NIO connectors. RSS value in ps unix cmd is not matching with (heap+metaspace). So if native memory is more than metaspace, how to measure it. And if possible could you mention the break up of memory used(RSS)
    – prem kumar
    Commented Sep 24, 2016 at 11:16
  • 1
    There can be lot of reasons like to many threads, may be problem with glibc version > 2.10, etc. You can use pmap - x to get more insight into the memory.
    – sol4me
    Commented Sep 24, 2016 at 11:44
  • 2
    A large chunk of your answer has been copied from evanjones.ca/java-bytebuffer-leak.html Commented Sep 24, 2016 at 13:38
  • 1
    @TheLostMind I reformatted my post , for some reason the formatting was not applied properly the first time. It should be fine now
    – sol4me
    Commented Sep 24, 2016 at 16:06
  • 2
    I knew about metaspace, but until this answer, I'd never heard about "meatspace" Is that where the JVM allocates memory for tube steak?
    – neildo
    Commented Dec 6, 2017 at 19:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.