Questions tagged [shared-memory]
Memory that may be simultaneously accessed by multiple programs
3,151
questions
0
votes
1
answer
54
views
Why is dynamically allocated memory in a shared memory reset to the initial value it was initialised to
I have the following code that initializes a shared memory, with the starting values:
void initializeSharedMemory(SharedMemoryPtr sharedMemoryPtr, int size) {
sharedMemoryPtr->...
0
votes
0
answers
41
views
How could I synchronize nine child processes to read a text file and write it into pipes to the parent?
I have nine child processes named (p1.c, p2.c, p3.c, p4.c, p5.c, p6.c, p7.c, p8.c, p9.c) that have to read through an input text file that is passed to the parent program as a command line argument.
...
4
votes
1
answer
95
views
Data structure alignment difference in Go and C
I have initialized a structure inside my C program and attached it to the Shared memory. The structure is as below:
#define DrvMaxTag 1024
#define DrvMaxStr 128
#define StructLEN 32
typedef struct ...
0
votes
0
answers
53
views
How to avoid 100% CPU when i use shared memory IPC?
I have three process, which need to pass data from one to another.
considering latency, I select shared memory to share data (IPC).
To get the best performance, the consumer process use busy-waiting ...
0
votes
0
answers
31
views
How do ShareableLists actually work when multiprocessing in python?
What I want to acchieve is, that a list with 10 to 20 thousand dicts inside is used in parallel. For that I am using multiprocessing. The problem with that is, that each process was using something ...
0
votes
0
answers
17
views
Unable to access data in shared memory created from 1 script to other script
I have two scripts one script is creating a space in shared memory and adding data in that shared memory other script is accessing the data in that memory. But I'm receiving error
FileNotFoundError: [...
1
vote
0
answers
49
views
Shared Memory vs Named Pipe comparison gone wrong: pipe is faster?
I have taken a course related to IPC in POSIX around 1.5 years ago and we made a benchmark to compare two communication method of 1.named pipes and 2.shared memory(with a circular buffer).
We create ...
0
votes
1
answer
94
views
How to give counter back to GUI from worker process with as little blocking of worker as possible and as little busy wait for gui as possible?
I have a gui that starts two seperate programms (that can be run parallel) in seperate processes.
I want to update the collective progess bar while the programms are running. So far i've been doing it ...
0
votes
0
answers
27
views
Python `ShareableList` is removed when "reading" process closes
I have a "main" process (write-only) in Python that generates a list of IDs which I would like to share with other independently created Python processes (read-only) on the same machine. I ...
0
votes
0
answers
100
views
How do I write to shared memory using a char pointer?
#define MIN_PID 300
#define MAX_PID 5000
#define SPROD_NAME "/sem_prod"
#define SCONS_NAME "/sem_cons"
std::vector<int> pidb = {};
int count = 0;
char* pid_manager::produce(...
1
vote
0
answers
96
views
Why doesn't Perl's `$shm->remove` from `IPC::SharedMem` remove the shared memory segment?
I wrote a simple web server that uses IPC::SharedMem (IPC::SharedMem->new(IPC_PRIVATE, $size, S_IRWXU)) and forks off processes to handle client requests.
When the server exited cleanly (and killed ...
0
votes
0
answers
57
views
Getting segmentation error while copying mp4 file to shared memory using memcpy()
I am writing a code to share data using shared memory, for txt data my code is working but for mp4 files it is not working sender is not able to send data.
EDIT: after changing buffer size as ...
1
vote
1
answer
82
views
Why does python shared memory implicitly unlinked on exit?
I wrote the following simple program to try shared memory in python:
#!/usr/bin/env python3.8
from multiprocessing import shared_memory
from time import sleep
PATH='/pth'
if __name__ == '__main__':
...
1
vote
0
answers
73
views
shared memory: coredump when attempting to write data for logging c++20
I have some questions regarding shared memory for logging. Before I start, I rarely post questions in this platform, please kindly forgive me if I may offend anyone here.
To give a rough intro, I have ...
0
votes
0
answers
73
views
Python Shared Memory Leakage
I ran the first example for the multiprocessing.shared_memory.SharedMemory class in the Python documentation website on Python 3.8. It worked as expected. But when I attached to the shared memory from ...
1
vote
1
answer
51
views
Shared memory not granted permission to parent process
I have a problem in my C code about shared memory.
I have a C code that doesn't work as I expected:
#include "sys/ipc.h"
#include "sys/shm.h"
#include "unistd.h"
#...
0
votes
1
answer
54
views
Server-Clients model using shared memory
I have the following program and can't make it to proper synchronization between clients and server.
After running , my program always ends up in collision between a client and a server trying to ...
2
votes
2
answers
97
views
Using the same variable between different Python Runs
I have a script like this
for epoch in range(num_epochs):
for bag in range(num_bags):
feats = pd.read_csv(f"feats_{bag}.csv")
... # some logic
As you can see, it ...
1
vote
0
answers
66
views
Is it possible to share memory between the main and the render process in Electron?
I'm working on an application that needs to transfer lots of binary data (chunks of 10-100 MB) as fast as possible, and the ipc api and/or http requests are simply not fast enough (~50-200 MB/s). The ...
-1
votes
1
answer
139
views
Allocating large shared memory leads to "Bus error (core dumped)"
When allocating a large amount of shared memory (60% of total 128GiB RAM, 3GiB used), I got "Bus error (core dumped)" after several seconds and the memory usage went up. But if I allocate ...
0
votes
0
answers
47
views
How to write Jagged array into shared memory with C/C++
I have the following jagged array (rank 6) which is output of a C/C++ code (this is achieved by splitting a rectangular 2D array)
{{{{{{1270529, 1, 1, 1, 1, 0, 1, 0.05, 1, 0., 0.}, {0., 0., 0., 0., 0.,...
0
votes
0
answers
30
views
Manage shared data in multi-thread-process applications
I have a web application in Python. Each API request is handled in a new thread. Therefore, I need to use mutexes to prevent race conditions.
However, some threads can spawn new processes. One thread ...
0
votes
1
answer
77
views
Variable number of SharedMemory objects managed by SharedMemoryManager in Python
I would like to have a variable number of SharedMemory objects (depending on the number of consumer processes in my producer-consumer app architecture) managed by SharedMemoryManager so I am creating
...
0
votes
0
answers
13
views
Nodejs worker threads shared object
I have an object in the main thread. What's the best way to allow a worker thread to be able to read that object? Note that the object changes multiple times per second.
1
vote
1
answer
51
views
Windows: Singleton in DLL from different EXEs seems shared
I'm trying to use a singleton in a DLL to globally configure logging. While everything works fine with one EXE using the DLL, when another EXE uses the same DLL at the same time, they seem to share ...
2
votes
1
answer
37
views
Is it necessary to close the SharedMemory at all times?
I noticed constantly closing and opening memory under heavy loads affects performance (example 1 get/set_shm).
Can I keep an open SharedMemory in different processes and keep a link to a numpy array ...
0
votes
0
answers
19
views
Boost.Interprocess. Using shared memory
I am considering using shared memory from Boost.Interprocess library to communicate between parent & child processes (running on the same machine). The C++ library I am developing is to be used on ...
1
vote
1
answer
47
views
Child process doesn't modify statistics in parent process
I'm working on a C application that involves multiple processes, both child processes and the parent process, and I'm trying to share statistics between them using shared memory and semaphores. ...
0
votes
0
answers
65
views
Shared memory error - cannot map an existing or recently created shared memory
I'm implementing a shared memory to do IPC between 2 processes in Linux Ubuntu.
The idea is to try to get to an existing shared memory. If not found, create it.
After that, adjust the shared memory to ...
0
votes
0
answers
58
views
How To Allocate Shared Memory in Specific NUMA Node?
I am trying to check the NUMA Effect on shared memory.
I want to allocate a shared memory(using shmget(), shmat()) in a specific NUMA node, for example in node 0. Other works will be done in another ...
1
vote
0
answers
57
views
SharedMemory in Python behaves weirdly (closes) on Debian
I'm trying to use shared memory as system-managed unique resource to ensure single instance and thought shared memory is a good option. Here's the code
import multiprocessing.shared_memory as ...
0
votes
0
answers
41
views
Shared memory, pipes, or sockets, with which one, data sharing between two proecesses can be faster and safer?
Inside our project (windows specific with VC++), we intended to share data between two processes. Indeed different approaches have been acquired to do the task, namely establishing pipes in between, ...
0
votes
0
answers
23
views
how to guarantee data avaliability between different block of shared memory
I used mmap to map files for data sharing bewteen processes, I then used another shared memory to notify the process when data is ready. a semaphore is created inside the shared memory.
after data is ...
0
votes
0
answers
39
views
how could i map virtual space in kernel to user space?
i want to shared a memeory space between user and kernel.
the first thing i want to do is allocating memory space in kernel, use kmalloc or vmalloc , and mapping the sapce to user for the second.
so i ...
2
votes
1
answer
240
views
Storing an std::vector in shared memory
I'm trying to share an std::vector between two processes by storing it in shared memory. The first process tries to add some data to the vector and the second process tries to read the data.
To create ...
0
votes
0
answers
153
views
How do I atomically change a value of a subslice in a slice
I have a byte slice in golang that contains binary data. Now in that slice there is a 32 bit integer that acts as a lock. The slice comes from mapped memory so another process is accessing the same ...
0
votes
1
answer
130
views
Named Pipe (Buffer?) or Shared Memory Files in Python
I have a question regarding named pipes or something similar in python and wether they can be made to work for my use case. I have something like the following code.
fn = path/to/my/file.ext
os.mkfifo(...
0
votes
0
answers
25
views
Does shared memory absolutely need manual locks?
I have a raspberry pi that fetches different sensor data over I2C and SPI. I need to do multiple stuff with the same data, like visualization, database, and control. I want to keep it lean and get the ...
4
votes
2
answers
82
views
Copying bytes object to shared memory list truncates trailing zeros
I'm trying to copy large bytes objects into a shareable list that's been initialized with bytes objects of large enough size. When the bytes object I'm copying over has trailing zeros (i.e. final ...
0
votes
0
answers
42
views
Linux 22 posix shared memory for ipc between python and c++/unreal sigbus errors
In an existing project I've started to help work on that has worked fine in the past, we're using posix shared memory for ipc between python and c++ code for Unreal Engine 4.27.
Recently the code ...
0
votes
0
answers
26
views
Difference between `multiprocessing.shared_memory` and anonymous mmap?
What is the technical difference between multiprocessing SharedMemory and an anonymous mmap? Is one better than the other?
0
votes
0
answers
81
views
How to use shared memory between processes on POSIX system
There are some processes use shared memory to do some synchronization. The struct in shared memory lists below:
struct Shm {
pthread_mutex_t mutex;
int a;
bool b;
double d;
};
Now ...
0
votes
0
answers
36
views
Restricting access of other processes when creating shared memory
In Windows, I would like to offer my software as a "data provider" so that other developers can read the data by using their own software, in same PC.
To achieve this I picked up the shared ...
-1
votes
1
answer
73
views
multiprocessing: Can a dict be shared between two Python shell?
I come from this post multiprocessing: How do I share a dict among multiple processes?
but I want something slightly different. In that post, a dict is shared between a parent process and its child ...
0
votes
0
answers
156
views
google-chrome constantly time out inside docker container even with "--disable-dev-shm-usage" hosted on fly.io
I am using chromiumoxide crate in rust to automate headless chrome inside docker container.
Here is the browser configuration:
let config = BrowserConfig::builder()
.chrome_executable(chrome_bin)
...
4
votes
1
answer
146
views
Named shared memory between C++ and python on Windows
I'm working on a C++ application that exchanges data with a Python script through named shared memory on Windows. I'm facing a problem where the Python script displays a completely black image as if ...
-2
votes
1
answer
66
views
Writing a string into shared memory from producer process with different character array sizes but same length string
I am trying to write a string into shared memory using a producer process as shown below.
int main(){
char str[30];
scanf("%[^\n]", str);
int shm_id = shmget((key_t)1234, sizeof(...
1
vote
0
answers
89
views
Linux shared memory queue
Is it possible to place
struct Data
{
std::queue<int> mydata;
}
in linux shared memory and perform queue operations from different programs? any alternative methods other than custom ...
0
votes
1
answer
2k
views
BufferError: memoryview has 1 exported buffer trying to close a shared memory where I put a dataframe of pyarrow data type values
I'm trying to put dataframes of different types in multiprocessing shared memory. When I'm using dataframes that contain python types I've no problem, but when I use pyarrow types I've problems ...
0
votes
0
answers
44
views
How can I my external app collect data from games (sim racing)?
I'm doing a project where I need some informations from the game like:
car model
tires
suspension....
So I'm developing an app that is based on cloud and I need to somehow transfer data from the ...