2

We have a PSGI script using UWSGI with multi processes and threads, looking for a fast solution to share editable hashes between these processed and threads. I did tests by using in-memory storage like Cache::Memcached::Fast and Redis.

There are fast but not fast enough, compared using just a locale hash

my %hash = ();

which is up to 10 times faster during our tests. Is there a way to share and editable variables in run time - like having sort of a local hash scenario - , e.g. that process 1 is updating

$hash{$id} = $new_val;

and process 2 has also that new value for $hash{$id}?

I also tried IPC::Shareable, MCE::Shared::Minidb or threads::shared (threads::shared: segment memory limit) which did work for the PSGI.

Is there a way to have FAST - editable - local variables between all processes on the same machine shared?

11
  • Why did not threads::shared work? Sep 8, 2019 at 9:51
  • It worked partial, this module is limited to the default/local segment memory limit. The hash is pretty big though (min. 100K elements; small value). threads::shared went out of memory when setting the hash :( Sep 8, 2019 at 10:17
  • "..100K elements.." How did you run out of memory? How much memory did your shared data consume? What error message did you get? I tried now with a shared variable of size 100M and I did not get any errors. Sep 8, 2019 at 10:44
  • Sorry, I meant IPC::Shareable which run out of memory. threads::shared did not work. The second process didn't have and values of %hash. threads::shared works in PSGI enviroments? (I guess there is a parallel/separate process instead of a thread) Sep 8, 2019 at 10:53
  • 1
    You're right. Redis performs here at 0.010856 sec. (testing over multi processors; memcached should perform even better than redis). Any solution to come close to a "locale hash performance" Sep 8, 2019 at 11:56

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.