Questions tagged [memory-fragmentation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
1 answer
138 views

Calling a routine on ESP8266 decreases progressively the Free Heap

I am using a routine printIPs to print on serial and on a display the last byte of the IP of all connected stations to a ESP8266 used as NAT. The whole schetch works if I do not call that routine. If ...
Ilmanowar's user avatar
1 vote
0 answers
141 views

Slow initialization of Python script using EasyOCR and Transformers after some runs, requires reboot or Nvidia module reset for optimal performance

I have a Python script that uses EasyOCR and Transformers to parse images. After running the script multiple times, the initialization process becomes slow and the script takes longer to start. The ...
Ajinkya Bhapkar's user avatar
0 votes
2 answers
468 views

Why does memory necessarily have to be contiguous? If it weren't, wouldn't this solve the issue of memory fragmentation?

My background knowledge: To my understanding, to be allocated/used properly, memory must be contiguous in the virtual address space, but doesn't have to be actually contiguous in the physical memory ...
rajrana22's user avatar
1 vote
0 answers
63 views

Best allocation strategy that need not preserve pointer validity

I have a contiguous heap of heterogeneous objects that need not preserve the validity of pointers to previously allocated objects upon a call to allocate, i.e: int* p = heap.allocate(1); int* q = heap....
fortytoo's user avatar
  • 452
0 votes
1 answer
114 views

How to measure fragmentation in OSX and WIN

Our clients experience OOM in some specific scenarios and we can't pinpoint any memory leaks so we would like to understand if it's a memory fragmentation issue. There are a lot of different memory ...
Murat Şeker's user avatar
  • 1,681
0 votes
0 answers
43 views

How does the heap memory not get fragmented

So I've tried to emulate a memory management in Java, mainly for educational purposes. This program should contain a malloc(int capacity) and a free(int address) method. But as it turns out, doing so ...
Mark Blace's user avatar
1 vote
1 answer
842 views

Memory Fragmentation with byte[] in C#

The C#/.NET application I am working on makes use of huge byte arrays and is having memory fragmentation issues. Checked memory usage using CLRMemory The Code we use is as follows PdfLoadedDocument ...
Phani Bnr's user avatar
1 vote
3 answers
2k views

Custom memory allocation - C v. C++

I have been learning C++, and have come across the topics of custom memory allocators. I have understood that by designing an allocator and using this allocator with standard library containers, we ...
thatmarkdude's user avatar
1 vote
1 answer
625 views

Does malloc allocated fragmented chunks?

I found out there are kernel drivers for Contiguous memory allocations. I though malloc coalesced memory and return best fit and if memory wasn't available it would return 0. If malloc only allocated ...
user282909's user avatar
2 votes
4 answers
941 views

What decides where on the heap memory is allocated?

Let me clear up: I understand how new and delete (and delete[]) work. I understand what the stack is, and I understand when to allocate memory on the stack and on the heap. What I don't understand, ...
INEEDANSWERS's user avatar
0 votes
1 answer
255 views

Is there any optimization or different API for page aligned memory allocations on the heap?

I am about to write a class representing a double-ended queue, just like std::dequeue, but with the capability to store any trivially destructible type, and without indexing support. Iteration or pop ...
Frede's user avatar
  • 1
4 votes
1 answer
2k views

Why can a user-process invoke the Linux OOM-killer due to memory fragmentation, even though plenty of RAM is available?

I've got a headless ARM-based Linux (v3.10.53-1.1.1) system with no swap space enabled, and I occasionally see processes get killed by the OOM-killer even though there is plenty of RAM available. ...
Jeremy Friesner's user avatar
0 votes
1 answer
377 views

Does use of hash tables cause memory fragmentation?

My understanding of hash tables is that they use hash functions to relate keys to locations in memory, with a total number of "buckets" pre-allocated in memory. The goal is for there to be enough ...
Tal's user avatar
  • 109
0 votes
0 answers
706 views

How to figure out when Android memory is too fragmented

I understand that my question is too vague, but here is the problem. I parse a very large file creating a linked list of my objects (BTW, they are not related to bitmaps). At some moment I get ...
Alex B's user avatar
  • 367
0 votes
0 answers
695 views

Algorithms for memory allocation which produces low fragmentation

I've read Modern Operating Systems 4th Edition, Andrew Tanenbaum in which are presented some ways to handle the memory management (with bitmaps, with linked lists) and some of the algorithms that can ...
Ovidiu Firescu's user avatar
0 votes
1 answer
664 views

libc malloc vs linux kernel buddy allocator

Does malloc wory about internal fragmentation in linux kernel? For example when I want to allocate 5 pages, will malloc round up size to make it power of 2: 5->8 to avoid internal fragmentation in ...
scopichmu's user avatar
  • 135
1 vote
0 answers
146 views

How to avoid virtual fragmentation problem for 32 bit linux kernel?

Everybody knows the problem of virtual memory fragmentation in a system. How can we improve the fragmentation problem from the kernel driver perspective in 32-bit kernel? Since we can't control the ...
Rahul Khandelwal's user avatar
0 votes
0 answers
606 views

C#, byte arrays, avoid memory fragmentation

I'm writing a data structure, which main purpose will be containing binary data (byte arrays) organized in such way, that user can insert data in the middle without performance penalties (moving large ...
Spook's user avatar
  • 25.5k
0 votes
1 answer
427 views

How to avoid VRAM fragmentation?

I realize there's no way to avoid it for certain, as OpenGL says nothing about VRAM fragmentation. But all the same, I have fragmentation in my app and I want to try reducing it on common platforms. ...
Stefan Monov's user avatar
  • 11.4k
10 votes
2 answers
10k views

"Memory Fragmentation" is it still an issue? [closed]

I'm a little bit confused. In the OS course we were told that all OSes take care of memory fragmentation by paging or segmentation and there is no contiguous physical memory allocation at all. OS uses ...
Afshin's user avatar
  • 392
2 votes
2 answers
211 views

What algorithm to apply for continuious reallocation of small memory chunks?

In C program I face transactions that require to have alot of memory chunks, I need to know if there is an algorithm or best practice teqnique used to handle all these malloc/free, I've used arrays to ...
Mohammad Nabil's user avatar
1 vote
0 answers
47 views

What does the CLR do before throwing an OutOfMemoryException?

I would like to know precisely what the CLR does before it throws an OutOfMemoryException (OOME). I understand that the likely cause is not having a contiguous chunk of memory available, but does the ...
Kit's user avatar
  • 20.6k
0 votes
0 answers
615 views

Memory usage in Base64 conversion?

I'm using the base64.b64encode library in python to convert my images into their b64 version. I noticed that while the b64 specs says that it takes 1.3 times the memory of the original data. I observe ...
tsar2512's user avatar
  • 2,856
1 vote
1 answer
1k views

Does OpenGL takes care of GPU memory fragmentation?

So basically whenever I create buffer objects Opengl allocates some memory on the GPU. Consider scenario 1 where I generate 2 uniform buffers for 2 uniform variables. Now consider scenario 2 where ...
gallickgunner's user avatar
7 votes
1 answer
3k views

How to Solve Gen2 Heap Fragmentation

I am running a C# application that services HTTP requests. I have noticed recently that it's taking up more memory then I expect. I grabbed some dumps, popped them in Windbg, and found that most of ...
shortspider's user avatar
  • 1,045
1 vote
0 answers
800 views

What is causing an extremely low memory fragmentation ratio?

We are seeing some odd memory issues with our Redis 4.0.2 instances. The master instance has a ratio of 0.12, whereas the slaves have reasonable ratios that hover just above 1. When we restart the ...
Andrew Eisenberg's user avatar
0 votes
2 answers
776 views

massif reported heap usage much less than VmRss, what could be wrong?

massif output: time=3220706 mem_heap_B=393242041 mem_heap_extra_B=73912175 mem_stacks_B=93616 heap_tree=peak process shows 1.2GB in VmRss, so the huge difference comes from where? (I saw Rss grows ...
bugs king's user avatar
  • 564
1 vote
1 answer
218 views

Internal fragmentation basic concept

Suppose I have two processes of 50 bytes and have only one partition of 100 bytes. Suppose the first process takes up the partition and 50 bytes is remaining . Can the second process reside in the ...
Zephyr's user avatar
  • 1,541
1 vote
1 answer
10k views

How to deal with external fragmentation, how paging helps with external fragmentation?

I know that there is a lot of questions regarding the issue I'm pointing here, but I couldn't find any complex answer (neither on StackOverflow nor in other sources). I would like to ask about heap (...
user1770426's user avatar
0 votes
1 answer
345 views

Arduino Mega slows down after 4 hours

I'm making a small project that makes some POST request to an webserver and do some tasks based on server response. Server always respond with JSON or an array Clasic response from server looks like ...
Nick's user avatar
  • 21
5 votes
1 answer
1k views

Does memory fragmentation slows down New/Malloc?

Short background: I'm developing a system that should run for months and using dynamic allocations. The question: I've heard that memory fragmentation slows down new and malloc operators because ...
OopsUser's user avatar
  • 4,652
0 votes
0 answers
667 views

Linux oom killer - find fragmented process

I am currently facing major issues with OOM after upgrading from Debian 7 to Debian 8 on arm (cubietruck). I have already read a lot about oom mechanism and causes, still I miss an answer to How to ...
Alex.'s user avatar
  • 53
17 votes
2 answers
9k views

Memory fragmentation

When I use malloc()s and free()s randomly, nested and with different sizes, at some point the memory will be fragmented because these operations leave a large list of small memory areas behind that ...
Elmi's user avatar
  • 5,928
3 votes
2 answers
1k views

Memory fragmentation when concatenating or adding strings but not with string.Format?

So a professor in university just told me that using concatenation on strings in C# (i.e. when you use the plus sign operator) creates memory fragmentation, and that I should use string.Format instead....
Gaspa79's user avatar
  • 5,567
0 votes
1 answer
77 views

How to send files without worrying about memory fragmentation?

I have a simple VB.NET web application that allows users to download particular files on the server's hard drive. However, some of these files are extremely large, up to 1GB in size. Sometimes when ...
painiyff's user avatar
  • 2,579
1 vote
1 answer
859 views

Erlang memory fragmentation

Our Erlang server looks have a serious memory leak, the VM memory usage is low but top is high. Env Erlang: R16B02 OS: Ubuntu 12.04.5 LTS \n \l X86_64 Erlang VM > erlang:memory(). [{total,...
linbo's user avatar
  • 2,393
4 votes
0 answers
442 views

What does it mean if totalMemory is small, but totalPss is very huge?

We have a problem with our app. Over time we have noticed that the totalPss value gets very large (depending on the device, it will be 300-700Mb): int pid = android.os.Process.myPid(); int ...
1800 INFORMATION's user avatar
2 votes
1 answer
1k views

Redis used_memory_rss is more than the config set 'maxmemory'?

We have Set 120 GB Max Memory in Redis Server. So in info output used_memory is always less than equal to 120GB but used_memory_rss is ~140GB Can Someone Please explain the reason of this?
Virender Dubey's user avatar
1 vote
1 answer
372 views

boost multi_index_container and memory fragmentation

I'm using MIC for my LRU cache in server, it has replaced list/map LRU since I was suspecting that this is what caused some unexplained memory footprint. Memory leaks are out of picture, at least no ...
kreuzerkrieg's user avatar
  • 3,019
0 votes
1 answer
2k views

How many bits required for page frame number on 32-bit system with 2^20 Bytes physical memory and 4KB page size?

How many bits required for page frame number on 32-bit system with 2^20 Bytes physical memory and 4KB page size? is it 11 bits?? Need an answer please help! Thank you.
Sasanka Pabasara's user avatar
3 votes
2 answers
858 views

How do I measure fragmentation in Hotspot's Metaspace?

I'm looking into debugging an "OutOfMemoryError: Metaspace" error in my application. Right before the OOME I see the following in the gc logs: {Heap before GC invocations=6104 (full 39): par new ...
mabi's user avatar
  • 5,279
2 votes
1 answer
740 views

SharpDX memory fragmentation

I am working on a .NET 3.5 application which uses SharpDX to render tiled 2D images. Textures (Texture2D) are loaded into a cache on-demand, and are created in the managed pool. Textures are ...
Ben Owen's user avatar
  • 102
15 votes
1 answer
1k views

Understanding internal fragmentation properties of Hotspot JVM process

For both on-heap and off-heap allocations. On-heap - in the context of three major garbage collectors: CMS, Parallel Old and and G1. What I know (or think that I know) to the moment: all object (on-...
leventov's user avatar
  • 14.9k
2 votes
2 answers
1k views

How to get the largest available chunk of memory that can be allocated? [duplicate]

How can I get the largest size (continious) in bytes my application can allocate in one malloc call without returning out of memory? (biggest available chunk) P.S.: I would like a general answer for ...
Gizmo's user avatar
  • 2,070
3 votes
1 answer
737 views

Excessive Gen 2 Free Blocks in Crash Dump

On inspecting a crash dump file for an out of memory exception reported by a client the results of !DumpHeap -stat showed that 575MB of memory is being taken up by 45,000 objects of type "Free" most ...
Chiune Sugihara's user avatar
3 votes
1 answer
872 views

how to avoid strings causing fragmentation into LOH

My application read lot of data from text files that get into large object heap which eventually cause fragmentation issue. Are there any alternative design approaches that can help to not get these ...
whoami's user avatar
  • 1,759
4 votes
0 answers
262 views

Can PerfView be used for finding source of fragmentation

I am finding PerfView to be very useful tool for determining root cause of various type of problems. However I am not sure if it can also be used to diagnose LOH fragmentation problem. I have ...
crazy novice's user avatar
  • 1,777
1 vote
2 answers
2k views

Solution to external fragmentation in contiguous memory allocation

In contiguous memory allocation we have a problem of external fragmentation , but cant we just combine all the available small holes of free memory to create a big one according to our requirement?
varun's user avatar
  • 1,493
0 votes
2 answers
49 views

Each single request on server for empty html page makes memoy increase. What could be causing this?

First of all, I am a junior developper and new to asp .Net Our ASp.Net website should be put in production soon enough. Except that i realized memory was increasing all the time, even if i refresh an ...
ouss court's user avatar
0 votes
1 answer
370 views

A sample test java code which can demonstrate memory fragmentation in heap

I would like to understand on how a memory fragmentation can occur by writing some bad code in the java user space. Please point out me with some ideas which can help me to understand this. In ...
Navaneeth Sen's user avatar