1

We use phpredis Library to connect to our 64 node Redis cluster from our serving machines. Though we make use of persistent connections, as php doesn't reuse objects across requests, every request makes a CLUSTER SLOTS call to the Redis Cluster first, and then makes a data fetch. This is proving to be very costly as this increases CPU on both the API and Redis and also increases network usage for meta info (CLUSTER SLOTS) that could otherwise be cached. Basically, we want the Redis cluster connection object to be re-used across multiple requests, in the same php-fpm process. Any suggestions on how to do this?

UPDATE: I tried the following diff in cluster_library.c code but this seems to be causing random runtime exceptions in php.

index 3e532b7..b2cbf16 100644
--- a/cluster_library.c
+++ b/cluster_library.c
@@ -7,6 +7,10 @@
 #include <zend_exceptions.h>

 extern zend_class_entry *redis_cluster_exception_ce;
+int cache_count = 0;
+//Cache the cluster slots value for every n requests/calls, n being 100 for now
+int CACHE_COUNT_VAL = 10;
+clusterReply *permSlots=NULL;

 /* Debugging methods/
 static void cluster_dump_nodes(redisCluster *c) {
@@ -939,7 +943,18 @@ PHP_REDIS_API int cluster_map_keyspace(redisCluster *c TSRMLS_DC) {
         }

         // Parse out cluster nodes.  Flag mapped if we are valid
-        slots = cluster_get_slots(seed TSRMLS_CC);
+ if (permSlots && cache_count <= CACHE_COUNT_VAL) {
+         slots = permSlots;
+         cache_count++;
+ }
+ else {
+         slots = cluster_get_slots(seed TSRMLS_CC);
+ }
+
         if (slots) {
             mapped = !cluster_map_slots(c, slots);
             // Bin anything mapped, if we failed somewhere
@@ -951,8 +966,16 @@ PHP_REDIS_API int cluster_map_keyspace(redisCluster *c TSRMLS_DC) {
         if (mapped) break;
     } ZEND_HASH_FOREACH_END();

+    if((!permSlots && mapped && slots) ||
+             cache_count >= CACHE_COUNT_VAL) {
+ permSlots = slots;
+ cache_count = 0;
+    }
+
     // Clean up slots reply if we got one
-    if(slots) cluster_free_reply(slots, 1);
+    // if(slots) cluster_free_reply(slots, 1);

     // Throw an exception if we couldn't map
     if(!mapped) {

1 Answer 1

4

Developer of phpredis here. At present this is a limitation/flaw in phpredis as it has to issue a CLUSTER SLOTS command for each new request. As far as I know the only way to avoid this is by pointing phpredis to a proxy like Codis or Corvis but I don't have any personal experience using them.

That being said, I did implement experimental support for this feature in this branch. It uses PHP's persistent_list to cache slot information across requests.

As Redis Cluster adoption appears to be ramping up, I am going to try to get this into the mainline develop branch soon and perhaps into the next stable release!

Cheers, Mike

7
  • Thanks Mike, however, I tried a simplistic solution by changing cluster_map_keyspace command to not call cluster_get_slots(seed TSRMLS_CC); every time and cache the slots in a global variable. I also commented cluster_free_reply(slots,1) call for it to not free up slots memory. It works in a standalone scenario, but fails in php-fpm + nginx setup. Any thoughts on why this is happening?
    – sreeraag
    Oct 30, 2017 at 10:11
  • Yes, because the Zend framework provides its own allocator, which will clean up after you if you forget to free a variable. That's why you have to use things like persistent_list or similar in order to retain context between requests. You could try to allocate persistent memory (pemalloc) but I don't recommend it. :) Oct 30, 2017 at 15:17
  • Had no idea about this! Thanks Mike! But this works in a test script with a standalone php process but was failing in a nginx + phpfpm setup, which was strange. Why would you not recommend pemalloc? Also, I was thinking about serializing the cluster slots reply to a file and fetching from the file every time. Do you think this is a good idea? Or the branch that uses persistent_list, I don't think is caching CLUSTER SLOTS out of the box. How do I enable it ?
    – sreeraag
    Oct 31, 2017 at 3:27
  • 1
    Well the redisReply that phpredis retrieves from CLUSTER SLOTS is a multilevel structure with pointers and pointers to pointers, etc. You would have to do a "deep copy" of that and pemalloc(ptr, 1) everything to avoid seeing weird behavior. As far as my old branch it looks like creating with persistent connections is what enables caching. I'm going to revisit this functionality this week to get it into the mainline branch, as I think it's something lots of people will want. Oct 31, 2017 at 17:55
  • Thanks so much, Mike! Would love to see this functionality in the main branch. Btw, I tried enabling persistent connections with the dev branch and ran a test. But, the performance was similar to using it without persistent connections, so was not sure if the caching is actually working or not.
    – sreeraag
    Nov 2, 2017 at 5:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.