As of May 31, 2023, we have updated our Code of Conduct.

Questions tagged [mmap]

mmap is a POSIX-compliant Unix system call that maps files or devices into memory.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
9 views

How to implement ”privatechat“ in the host version Linux chat room (using message queue),How to implement private chat function in Linux chat room

I have initially realized the function of group chat through shared memory, daemon process, message queue, etc., but there are always errors in private chat. The detailed code is as follows /* "...
dreammaker's user avatar
0 votes
1 answer
58 views

How to fix 'No such file or directory' error when using mmap shared memory in C? [closed]

I am trying to solve this problem in C with shared memory (using mmap) for my operating systems class. The purpose of this code is to use anonymous pipe(P0 to P1), named pipe(P1 to P2) and shared ...
Alina's user avatar
  • 19
0 votes
1 answer
38 views

Question about process race conditions on system calls

When you use mmap with MAP_PRIVATE on a file: It is unspecified whether changes made to the file after the mmap() call are visible in the mapped region Does this mean that if you are mid-read, ...
user129393192's user avatar
0 votes
1 answer
44 views

What happens if you write to `mmap`ed sparse file when disk is full?

Many systems have mmap or equivalent functionality, where the contents of a file can be mapped to a memory address range. Also many systems have sparse files, i.e. files with ranges of non-allocated ...
lvella's user avatar
  • 12.6k
1 vote
0 answers
62 views

Mmap : failing with MAP_SYNC

[linux kernel 6.1.27 on Debian testing, up to date. Clang 16 or gcc 12.2] May someone tell we what's wrong and why mmap returns MAP_FAILED: int fd = ::open( file_name.data(), O_RDWR ); auto *ptr = ...
LeXav's user avatar
  • 61
0 votes
1 answer
27 views

mmap failed on libcamera returned frame data

I'm working with camera on a Raspberry Pi 3B. The frame data returned by libcamera is given by tuples of raw fd, size and offset. And according to the official document I have to mmap them to access ...
jiandingzhe's user avatar
  • 1,831
0 votes
1 answer
20 views

Manually swap memory to files?

On platforms which does not have enough memory and which does not actively swap memory to files, mmap can be used to manually get "more memory than physically available". My question is, if ...
Mr User's user avatar
  • 165
0 votes
0 answers
25 views

file backed mmap with huge page?

-> Linux - Debian - Ext4 - Kernel 6.1 Other discussions other relevant questions: this try This answer This other question Debian's My context I currently use a mmap file ("/user_db/db....
LeXav's user avatar
  • 61
0 votes
1 answer
65 views

How can the parent process perceive the modification of the shared memory

I want to write log files using mmap in multi-process server, the processes can write to the initial log file, but if one process modify the shared address to a new file, other processes can not ...
Klen's user avatar
  • 7
1 vote
0 answers
26 views

boost mapped_file and page fault

I have done the following test using boost::iostreams::mapped_file and encountered some page fault (detected using getrusage). The page faults in step 4 are something I could not understand. the file ...
doraemon's user avatar
  • 2,278
-1 votes
1 answer
54 views

Why mmap fails to allocate executable page on macOS?

I am trying to run the code example in this answer https://stackoverflow.com/a/38974980 on macOS 13, and the program gives segmentation fault. In fact, let's simplify the program linked above to #...
Mr User's user avatar
  • 165
1 vote
1 answer
31 views

How to query how many processes has a page mapped?

I have a pointer returned by mmap with MAP_SHARED. Is there a way to query the kernel for how many processes are currently mapping the pointed page? Preferably in a portable POSIX way, but also ...
lvella's user avatar
  • 12.6k
0 votes
0 answers
19 views

mmap memory kernel space to user space - too slow, any improvements?

In kernel driver, using remap_pfn_range to remap a physical memory block vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); vma->vm_flags |= VMEM_FLAGS | VM_PFNMAP; vma->vm_pgoff = ...
dxyzhou-tiger's user avatar
0 votes
1 answer
39 views

How can I move ld.so higher in virtual memory?

My tool needs to recreate a portion of the memory layout of another processes. To do this reliably, I created a stub/wrapper that turns off ASLR, links only against libdl and libc to minimize the ...
Grisby_2133's user avatar
0 votes
0 answers
46 views

Using multiple threads to read and write the same file without mmap?

Related: Multiple threads reading from the same file mmap can be used to read and write files, but it does not support large files, for example, on a 32-bit system. The stdio FILE* in C, or linux file ...
Mr User's user avatar
  • 165
2 votes
2 answers
39 views

Transparently replace file mapping with anonymous

I am doing a checkpoint-and restore using CRIU; in turn after restore, my application wakes with some threads that have their stack mmaped into files on disk (CRIU doesn't do it by default, this is a ...
Radim Vansa's user avatar
  • 5,666
1 vote
0 answers
16 views

How to detect if `struct page` is backed by a file on a filesystem in Linux?

I am writing a Linux device module that need to occasionally pin userspace memory pages. Some of these pages may be backed by a file placed on a filesystem. That happens if a userspace program which ...
Grigory Rechistov's user avatar
0 votes
0 answers
19 views

Are physical address of huge pages returned by mmap are divided by the huge page size?

It seems to me that OS won't allocate huge pages which are not align to the huge page size (not only the 4KB system page size), but I am not sure about it. The documentation of mmap ensures that the ...
user3563894's user avatar
2 votes
1 answer
80 views

Why pages reclaims still present after using `munmap()`?

for a study project I have to code a reimplementation of malloc() and free() using mmap() and munmap(). I'm running on the last Ubuntu. For my tests I use the command time -v (from /usr/bin/time) ...
lucocozz's user avatar
0 votes
0 answers
19 views

Does reading the void* from mmap as another type cause UB with regards to strict aliasing

When reading from mmap that another process wrote to, technically the object doesn't exist in the reading process. So for example (writing process): void* mmap_memory = mmap(....); memcpy(mmap_memory, ...
Brandon's user avatar
  • 22.5k
0 votes
0 answers
23 views

Host slows down with heavy use of mmap?

I have some single threaded java nodes that which process some batch data. These nodes make heavy use of mmap specifically they write to an mmap file that is around 500gb in size which another process ...
trafalgar's user avatar
-2 votes
1 answer
66 views

Mmap throws an error: mmap object has no attribute 'split'

I am writing a code that should search faster in a large file using mmap. import re import mmap with open('words.txt', 'r', encoding="utf8") as f1, open('source.txt', 'r', encoding="...
Bobby TB's user avatar
0 votes
0 answers
80 views

Linux/C/mmap: Delay involved with IPC mechanism using Shared Memory Object in "dev/shm"

I have two processes(let's say process1 and process2) which are independent processes running on different cores. Also, process1 has higher time frequency than process2. I'm trying to send one piece ...
Pavankumar S V's user avatar
0 votes
0 answers
95 views

Linux/C/mmap(): Avoiding the flushing of memory mapped region created using mmap to the underlying file

I am using mmap() to map device file "mtdblock0" into virtual address space of one process1. Some data whose value changes continuously for some time are stored in memory mapped region as ...
Pavankumar S V's user avatar
0 votes
0 answers
18 views

Android Java Studio - mmap failure (volley http posts)

Thanks for reading. I am getting a weird mmap failure after a while. While running, the following function is being repeated 4 times a second: public void sendLocationData(int id, double latitude, ...
Raz Gavrieli's user avatar
0 votes
0 answers
20 views

Looking for a map structure, shareable between processes and language

I'm looking for an in-memory binary format for a map structure (key-value store) with quick access time. My goal is to: Store this binary format as a file. Be able to retrieve it and with zero-copy, ...
Damien's user avatar
  • 1,904
0 votes
1 answer
49 views

Best way to load a mmap dictionary in Python without deserializing

Context: I've got Python processes running on the same container and I want to be able to share a read-only key-value object between them. I'm aware I could use something like Redis to share that info,...
Damien's user avatar
  • 1,904
4 votes
1 answer
150 views

Reasoning behind high latency when using python ctypes during process interrupts

While investigating a critical path in our python codebase, we found out that the behaviour of ctypes in terms of latencies is quite unpredictable. A bit more background of our application. We have ...
Rohith Uppala's user avatar
0 votes
0 answers
55 views

If I private-`mmap` a file and read it, then another process writes to the same file, will another read at the same location return the same value?

(Context: I'm trying to establish which sequences of mmap operations are safe from the "memory safety" point of view, i.e. what assumptions I can make about mmaped memory without risking ...
ais523's user avatar
  • 657
3 votes
0 answers
269 views

Perf collection on kubernetes pods

I am trying to find performance bottlenecks by using the perf tool on a kubernetes pod. I have already set the following on the instance hosting the pod: "kernel.kptr_restrict" = "0&...
Hadi's user avatar
  • 935
1 vote
1 answer
160 views

mmap and instruction / data cache coherency: Why can we copy and run shared libraries?

On ARM after writing instructions to memory a memory barrier is needed before executing the instructions. Specifically clean the data cache, invalidate the instruction cache, then execute an ...
Timothy Baldwin's user avatar
0 votes
0 answers
102 views

Too many brk() noticed in strace

I have a c++ service, where I am trying to debug the cause of high service startup time. From strace logs I notice a lot of brk() (which contributes to about 300ms, which is very high for our SLA). On ...
LilyEvans's user avatar
3 votes
2 answers
107 views

How do I open a mmap of size of a TB

I need to open a huge memory map. The file is one terabyte. I however am getting an errno: ENOMEM 12 Cannot allocate memory. I don't get what is holding me up. Requesting the RLIMIT_AS results in the ...
Tarick Welling's user avatar
1 vote
1 answer
175 views

x86_64 assembly mmap syscall [duplicate]

I try to make very simple memory allocator, but it gives me error on output, additionally the print_sys_error doesn't work properly. It works if I call it by this code: _start: sub $16, %rsp # ...
some random's user avatar
0 votes
0 answers
59 views

userspace-kernel shared memory with mmap works strange on multi-thread

I'm currently making shared memory between userspace to kernel. The following code is simple version of my mmap method. for (j = 0; j < 3; j++){ RxFrame[j] = kmalloc(4096*6, GFP_KERNEL); ...
linuxman's user avatar
2 votes
1 answer
93 views

How does GLIBC decide segment for malloc

I look at some Linux Glibc(2.25) system and see that when the code use malloc . sometimes the buffer has been allocated at heap segment and sometimes in anonymous segment, It's not relate for size, I ...
Kokomelom's user avatar
  • 143
0 votes
1 answer
45 views

Is there a way to wrap an numpy `ndarray` interface around an existing binary file?

I have a binary network capture (.pcapng) file that contains video data. I am parsing the .pcapng with scapy and I can extract the data, but the video sequences I am working with are very large and ...
K. Nielson's user avatar
0 votes
1 answer
36 views

msync guarantees on linux

Is msync guaranteed to physically write dates into the hard-drive? Can I be 100% sure that if the OS crashes immediately after msync returns, that I won't lose data? By 100% sure I mean that the data ...
Samuel Hapak's user avatar
  • 6,918
0 votes
2 answers
59 views

Musl mmap ENOMEM

I compiled some cross compilers against musl(x86_64, i686, arm). I need to compile code, that allocate like 2048 Mb +- 200Mb. However I noticed some errors with i686 musl compiler: #include <stdio....
Rainbow's user avatar
  • 23
-2 votes
1 answer
205 views

What's the best practice for swap apache arrow data between different processes? [closed]

I have a data api which could get stream data use rust as an independent service process, and plan to write several client process to read the data, because the client process have some function based ...
Hakase's user avatar
  • 197
4 votes
0 answers
53 views

munmap of an unlinked file - is the data flushed?

I am interested in using an mmap'ed file as a private swap space in case the machine doesn't have enough RAM for my huge tables. When I'm done with this processing, I no longer need these tables and ...
Divon's user avatar
  • 47
2 votes
1 answer
142 views

Azerothcore error azerothcore-wotlk-ac-worldserver-1 | MMAP:loadMap: 5303231.mmtile was built with generator v15, expected v16

I recently updated my Azerothcore docker version, but I am encountering an error: "azerothcore-wotlk-ac-worldserver-1 | MMAP:loadMap: 5303231.mmtile was built with generator v15, expected v16&...
Carl Sagan's user avatar
2 votes
2 answers
128 views

Failed to read pci csr via mmap using uint64_t pointer [closed]

I'm trying to reading PCI CSR (Configuration Space Register) on my system via open,mmap /dev/mem. I met some problems when using 8 byte length reading Here is the minimal working example of my code #...
imotfnir's user avatar
0 votes
0 answers
58 views

I am trying to dynamically allocate memory for a matrix in assembly using mmap but it does not work

I am having trouble with the syscall mmap to dynamically allocate memory for a matrix,%eax should return at the end of the call the memory address but it returns a big negative value and the program ...
toni06's user avatar
  • 1
0 votes
0 answers
61 views

How could I do to mmap a large memory area with Anonymous Mappings but no-zero-filled to be done?

As *The Linux Programming Interface * said Anonymous Mappings: With both the MAP_ANONYMOUS and the /dev/zero techniques, the bytes of the resulting mapping are initialized to 0. I want to mmap a ...
LeBron GanDalf's user avatar
1 vote
1 answer
64 views

Use mmap for random read from file but the read throuput is not as expected

I write k-v data into file with value size of 100KB and key size of 10Bytes. Only value data is written into file and indices of keys are stored in ram. I write 10M keys and I get a 1TB value file. I ...
GOGO's user avatar
  • 589
0 votes
2 answers
108 views

How to pass void pointer by reference

I want to have a void* as a function parameter and then inside a function modify that pointer (change it to NULL). Code below doesn't change memory1 to null after function call. How can I change that. ...
George's user avatar
  • 21
1 vote
1 answer
65 views

mallinfo doesn't show mmap allocation's information

In mallinfo structure there are two fields hblks and hblkhd. The man documentation says that they are responsible for the number of blocks allocated by mmap and the total number of bytes. But when I ...
Vadim Sadokhov's user avatar
3 votes
0 answers
90 views

Valgrind massif maximum mmap size

When I use mmaping for a large file(80-150 GB) valgrind failed with the error (mmap): Invalid argument, intuitively, I realized that the problem was probably with the file size, I tried to reduce the ...
Vladislav Markov's user avatar
4 votes
1 answer
90 views

Window control for mmapped large file(linux, mmap)

How can we control the window in RSS when mapping a large file? Now let me explain what i mean. For example, we have a large file that exceeds RAM by several times, we do shared memory mmaping for ...
Vladislav Markov's user avatar

1
2 3 4 5
39