Questions tagged [memory-fragmentation]
The memory-fragmentation tag has no usage guidance.
86
questions
1
vote
1
answer
138
views
Calling a routine on ESP8266 decreases progressively the Free Heap
I am using a routine printIPs to print on serial and on a display the last byte of the IP of all connected stations to a ESP8266 used as NAT.
The whole schetch works if I do not call that routine. If ...
1
vote
0
answers
141
views
Slow initialization of Python script using EasyOCR and Transformers after some runs, requires reboot or Nvidia module reset for optimal performance
I have a Python script that uses EasyOCR and Transformers to parse images. After running the script multiple times, the initialization process becomes slow and the script takes longer to start. The ...
0
votes
2
answers
468
views
Why does memory necessarily have to be contiguous? If it weren't, wouldn't this solve the issue of memory fragmentation?
My background knowledge:
To my understanding, to be allocated/used properly, memory must be contiguous in the virtual address space, but doesn't have to be actually contiguous in the physical memory ...
1
vote
0
answers
63
views
Best allocation strategy that need not preserve pointer validity
I have a contiguous heap of heterogeneous objects that need not preserve the validity of pointers to previously allocated objects upon a call to allocate, i.e:
int* p = heap.allocate(1);
int* q = heap....
0
votes
1
answer
114
views
How to measure fragmentation in OSX and WIN
Our clients experience OOM in some specific scenarios and we can't pinpoint any memory leaks so we would like to understand if it's a memory fragmentation issue.
There are a lot of different memory ...
0
votes
0
answers
43
views
How does the heap memory not get fragmented
So I've tried to emulate a memory management in Java, mainly for educational purposes. This program should contain a malloc(int capacity) and a free(int address) method. But as it turns out, doing so ...
1
vote
1
answer
842
views
Memory Fragmentation with byte[] in C#
The C#/.NET application I am working on makes use of huge byte arrays and is having memory fragmentation issues. Checked memory usage using CLRMemory
The Code we use is as follows
PdfLoadedDocument ...
1
vote
3
answers
2k
views
Custom memory allocation - C v. C++
I have been learning C++, and have come across the topics of custom memory allocators. I have understood that by designing an allocator and using this allocator with standard library containers, we ...
1
vote
1
answer
625
views
Does malloc allocated fragmented chunks?
I found out there are kernel drivers for Contiguous memory allocations. I though malloc coalesced memory and return best fit and if memory wasn't available it would return 0. If malloc only allocated ...
2
votes
4
answers
941
views
What decides where on the heap memory is allocated?
Let me clear up: I understand how new and delete (and delete[]) work. I understand what the stack is, and I understand when to allocate memory on the stack and on the heap.
What I don't understand, ...
0
votes
1
answer
255
views
Is there any optimization or different API for page aligned memory allocations on the heap?
I am about to write a class representing a double-ended queue, just like std::dequeue, but with the capability to store any trivially destructible type, and without indexing support. Iteration or pop ...
4
votes
1
answer
2k
views
Why can a user-process invoke the Linux OOM-killer due to memory fragmentation, even though plenty of RAM is available?
I've got a headless ARM-based Linux (v3.10.53-1.1.1) system with no swap space enabled, and I occasionally see processes get killed by the OOM-killer even though there is plenty of RAM available.
...
0
votes
1
answer
377
views
Does use of hash tables cause memory fragmentation?
My understanding of hash tables is that they use hash functions to relate keys to locations in memory, with a total number of "buckets" pre-allocated in memory. The goal is for there to be enough ...
0
votes
0
answers
706
views
How to figure out when Android memory is too fragmented
I understand that my question is too vague, but here is the problem. I parse a very large file creating a linked list of my objects (BTW, they are not related to bitmaps). At some moment I get ...
0
votes
0
answers
695
views
Algorithms for memory allocation which produces low fragmentation
I've read Modern Operating Systems 4th Edition, Andrew Tanenbaum in which are presented some ways to handle the memory management (with bitmaps, with linked lists) and some of the algorithms that can ...
0
votes
1
answer
664
views
libc malloc vs linux kernel buddy allocator
Does malloc wory about internal fragmentation in linux kernel?
For example when I want to allocate 5 pages, will malloc round up size to make it power of 2: 5->8 to avoid internal fragmentation in ...
1
vote
0
answers
146
views
How to avoid virtual fragmentation problem for 32 bit linux kernel?
Everybody knows the problem of virtual memory fragmentation in a system. How can we improve the fragmentation problem from the kernel driver perspective in 32-bit kernel?
Since we can't control the ...
0
votes
0
answers
606
views
C#, byte arrays, avoid memory fragmentation
I'm writing a data structure, which main purpose will be containing binary data (byte arrays) organized in such way, that user can insert data in the middle without performance penalties (moving large ...
0
votes
1
answer
427
views
How to avoid VRAM fragmentation?
I realize there's no way to avoid it for certain, as OpenGL says nothing about VRAM fragmentation.
But all the same, I have fragmentation in my app and I want to try reducing it on common platforms.
...
10
votes
2
answers
10k
views
"Memory Fragmentation" is it still an issue? [closed]
I'm a little bit confused. In the OS course we were told that all OSes take care of memory fragmentation by paging or segmentation and there is no contiguous physical memory allocation at all. OS uses ...
2
votes
2
answers
211
views
What algorithm to apply for continuious reallocation of small memory chunks?
In C program I face transactions that require to have alot of memory chunks, I need to know if there is an algorithm or best practice teqnique used to handle all these malloc/free, I've used arrays to ...
1
vote
0
answers
47
views
What does the CLR do before throwing an OutOfMemoryException?
I would like to know precisely what the CLR does before it throws an OutOfMemoryException (OOME). I understand that the likely cause is not having a contiguous chunk of memory available, but does the ...
0
votes
0
answers
615
views
Memory usage in Base64 conversion?
I'm using the base64.b64encode library in python to convert my images into their b64 version. I noticed that while the b64 specs says that it takes 1.3 times the memory of the original data. I observe ...
1
vote
1
answer
1k
views
Does OpenGL takes care of GPU memory fragmentation?
So basically whenever I create buffer objects Opengl allocates some memory on the GPU.
Consider scenario 1 where I generate 2 uniform buffers for 2 uniform variables.
Now consider scenario 2 where ...
7
votes
1
answer
3k
views
How to Solve Gen2 Heap Fragmentation
I am running a C# application that services HTTP requests. I have noticed recently that it's taking up more memory then I expect. I grabbed some dumps, popped them in Windbg, and found that most of ...
1
vote
0
answers
800
views
What is causing an extremely low memory fragmentation ratio?
We are seeing some odd memory issues with our Redis 4.0.2 instances. The master instance has a ratio of 0.12, whereas the slaves have reasonable ratios that hover just above 1. When we restart the ...
0
votes
2
answers
776
views
massif reported heap usage much less than VmRss, what could be wrong?
massif output:
time=3220706
mem_heap_B=393242041
mem_heap_extra_B=73912175
mem_stacks_B=93616
heap_tree=peak
process shows 1.2GB in VmRss, so the huge difference comes from where? (I saw Rss grows ...
1
vote
1
answer
218
views
Internal fragmentation basic concept
Suppose I have two processes of 50 bytes and have only one partition of 100 bytes.
Suppose the first process takes up the partition and 50 bytes is remaining .
Can the second process reside in the ...
1
vote
1
answer
10k
views
How to deal with external fragmentation, how paging helps with external fragmentation?
I know that there is a lot of questions regarding the issue I'm pointing here, but I couldn't find any complex answer (neither on StackOverflow nor in other sources).
I would like to ask about heap (...
0
votes
1
answer
345
views
Arduino Mega slows down after 4 hours
I'm making a small project that makes some POST request to an webserver and do some tasks based on server response. Server always respond with JSON or an array
Clasic response from server looks like ...
5
votes
1
answer
1k
views
Does memory fragmentation slows down New/Malloc?
Short background:
I'm developing a system that should run for months and using dynamic allocations.
The question:
I've heard that memory fragmentation slows down new and malloc operators because ...
0
votes
0
answers
667
views
Linux oom killer - find fragmented process
I am currently facing major issues with OOM after upgrading from Debian 7 to Debian 8 on arm (cubietruck).
I have already read a lot about oom mechanism and causes, still I miss an answer to
How to ...
17
votes
2
answers
9k
views
Memory fragmentation
When I use malloc()s and free()s randomly, nested and with different sizes, at some point the memory will be fragmented because these operations leave a large list of small memory areas behind that ...
3
votes
2
answers
1k
views
Memory fragmentation when concatenating or adding strings but not with string.Format?
So a professor in university just told me that using concatenation on strings in C# (i.e. when you use the plus sign operator) creates memory fragmentation, and that I should use string.Format instead....
0
votes
1
answer
77
views
How to send files without worrying about memory fragmentation?
I have a simple VB.NET web application that allows users to download particular files on the server's hard drive. However, some of these files are extremely large, up to 1GB in size. Sometimes when ...
1
vote
1
answer
859
views
Erlang memory fragmentation
Our Erlang server looks have a serious memory leak, the VM memory usage is low but top is high.
Env
Erlang: R16B02
OS: Ubuntu 12.04.5 LTS \n \l X86_64
Erlang VM
> erlang:memory().
[{total,...
4
votes
0
answers
442
views
What does it mean if totalMemory is small, but totalPss is very huge?
We have a problem with our app. Over time we have noticed that the totalPss value gets very large (depending on the device, it will be 300-700Mb):
int pid = android.os.Process.myPid();
int ...
2
votes
1
answer
1k
views
Redis used_memory_rss is more than the config set 'maxmemory'?
We have Set 120 GB Max Memory in Redis Server.
So in info output used_memory is always less than equal to 120GB but used_memory_rss is ~140GB
Can Someone Please explain the reason of this?
1
vote
1
answer
372
views
boost multi_index_container and memory fragmentation
I'm using MIC for my LRU cache in server, it has replaced list/map LRU since I was suspecting that this is what caused some unexplained memory footprint. Memory leaks are out of picture, at least no ...
0
votes
1
answer
2k
views
How many bits required for page frame number on 32-bit system with 2^20 Bytes physical memory and 4KB page size?
How many bits required for page frame number on 32-bit system with 2^20 Bytes physical memory and 4KB page size?
is it 11 bits??
Need an answer please help! Thank you.
3
votes
2
answers
858
views
How do I measure fragmentation in Hotspot's Metaspace?
I'm looking into debugging an "OutOfMemoryError: Metaspace" error in my application. Right before the OOME I see the following in the gc logs:
{Heap before GC invocations=6104 (full 39):
par new ...
2
votes
1
answer
740
views
SharpDX memory fragmentation
I am working on a .NET 3.5 application which uses SharpDX to render tiled 2D images.
Textures (Texture2D) are loaded into a cache on-demand, and are created in the managed pool.
Textures are ...
15
votes
1
answer
1k
views
Understanding internal fragmentation properties of Hotspot JVM process
For both on-heap and off-heap allocations. On-heap - in the context of three major garbage collectors: CMS, Parallel Old and and G1.
What I know (or think that I know) to the moment:
all object (on-...
2
votes
2
answers
1k
views
How to get the largest available chunk of memory that can be allocated? [duplicate]
How can I get the largest size (continious) in bytes my application can allocate in one malloc call without returning out of memory? (biggest available chunk)
P.S.: I would like a general answer for ...
3
votes
1
answer
737
views
Excessive Gen 2 Free Blocks in Crash Dump
On inspecting a crash dump file for an out of memory exception reported by a client the results of !DumpHeap -stat showed that 575MB of memory is being taken up by 45,000 objects of type "Free" most ...
3
votes
1
answer
872
views
how to avoid strings causing fragmentation into LOH
My application read lot of data from text files that get into large object heap which eventually cause fragmentation issue. Are there any alternative design approaches that can help to not get these ...
4
votes
0
answers
262
views
Can PerfView be used for finding source of fragmentation
I am finding PerfView to be very useful tool for determining root cause of various type of problems. However I am not sure if it can also be used to diagnose LOH fragmentation problem. I have ...
1
vote
2
answers
2k
views
Solution to external fragmentation in contiguous memory allocation
In contiguous memory allocation we have a problem of external fragmentation , but cant we just combine all the available small holes of free memory to create a big one according to our requirement?
0
votes
2
answers
49
views
Each single request on server for empty html page makes memoy increase. What could be causing this?
First of all, I am a junior developper and new to asp .Net
Our ASp.Net website should be put in production soon enough. Except that i realized memory was increasing all the time, even if i refresh an ...
0
votes
1
answer
370
views
A sample test java code which can demonstrate memory fragmentation in heap
I would like to understand on how a memory fragmentation can occur by writing some bad code in the java user space.
Please point out me with some ideas which can help me to understand this.
In ...