29

I want to force the Linux kernel to allocate more memory to applications after the cache starts taking up too much memory (as can be seen by the output of 'free').

I've run

sudo sync; sudo sysctl -w vm.drop_caches=3; free

(to free both disc dentry/inode cache and page cache) and I see that only about half of the used cache was freed - the rest remains. How can I tell what is taking up the rest of the cache and force it to be freed?

2
  • The chache that stays is just cache that gets thrown away immediately as new memory is needed. It is not beneficial to force the removal of these cache regions. Applications will always be first citizens for memory and don't have to fight with cache for it.
    – ypnos
    Mar 2, 2009 at 11:14
  • 1
    I have a problem with my server, and I believe that something is abusing the cache to the point where applications are relegated to swapped out pages and are thrashing due to lack of memory. It might be a kernel issue it might not be. Purging the cache might be a good way to test.
    – Guss
    Mar 2, 2009 at 14:57

3 Answers 3

18
+100

You may want to increase vfs_cache_pressure as well as set swappiness to 0.

Doing that will make the kernel reclaim cache faster, while giving processes equal or more favor when deciding what gets paged out.

You may only want to do this if processes you care about do very little disk I/O.

If a network I/O bound process has to swap in to serve requests, that's a problem and the real solution is to put it on a less competitive server.

With the default swappiness setting, the kernel is almost always going to favour keeping FS related cache in real memory.

As such, if you increase the cache pressure, be sure to equally adjust swappiness.

1
  • Thanks for the answer. Though its not what I was hoping for, I'll accept it as it includes information I didn't know and gives some guidelines to achieve a target. I would still like to know better how to figure out what the cache is used for, especially when it is not being cleaned by drop_caches.
    – Guss
    Mar 31, 2009 at 17:21
18

The contents of /proc/meminfo tell you what the kernel uses RAM for.

You can use /proc/sys/vm/vfs_cache_pressure to force the kernel to reclaim memory that is used for filesystem-related caches more lazily or eagerly.

Note that your application may only benefit from tuning this parameter if it does little or no disk I/O.

4
  • 1
    My app does very little IO - most data is from a remote DB. I'm a little hazy on the /proc/meminfo: I see about Cached: 1.9GB, SwapCached: 8MB. IIRC, that means that Cached is composed of 8MB swap caching, 1.892GB other stuff (disk cache, and what else?). Where do memory mapped files are counted?
    – Guss
    Mar 1, 2009 at 12:53
  • Memory mapped files (if the app is at all good) should have their own advisories and pressure, preferably by the application calling posix_madvise(). You might also want to read up on your 'swappiness' setting.
    – user50049
    Mar 1, 2009 at 13:13
  • Indeed a good answer, just not an answer to my question :-). Do I understand correctly that memory mapped files are counted against their application's RES memory footprint?
    – Guss
    Mar 1, 2009 at 13:44
  • Against their applications VIRT which may or may not be resident depending on if all the pages are accessed and paged in.
    – stsquad
    Apr 18, 2009 at 10:20
15

You might find John Nilsson's answer to my Question useful for purging the cache in order to test whether that is related to your problem:

sync && echo 1 > /proc/sys/vm/drop_caches

Though I'm guessing the only real difference is 1 vs 3

2
  • 1 only clears the disk cache - it was relevant to your question. I want to also clear the page cache as I think that one of my apps is leaking memory and thrashing badly.
    – Guss
    Mar 25, 2009 at 20:49
  • 2
    1 clears page cache, 2 clears dentries and inodes, 3 clear page cache, dentries and inodes Apr 17, 2013 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.