18

Using Magento's back-office, after saving a category which was linked to many products, only the first 1000 products (or 2000, or x000, depending on host configuration) are kept.

All other category/product links are deleted from the catalog_category_product table in the database.
This occurs even though no checkboxes have been unchecked in the products grid (Category Products tab).

No error message is displayed or logged.
Tracing data posted from the browser to the server doesn't reveal any missing product IDs, but even if you check more products on the back-office grid, when finally saving the category, the server only keeps the first x000 and removes other IDs...

6 Answers 6

27

Though some answers can be found on other websites, I thought it was worth sharing this on StackOverflow...

The source of the problem can be found in Mage_Adminhtml_Catalog_CategoryController, where the saveAction() method retrieves a big POSTed string (category_products, encoded like a query string), which is processed by PHP function parse_str():

if (isset($data['category_products']) && !$category->getProductsReadonly()) {
    $products = array();
    parse_str($data['category_products'], $products);
    $category->setPostedProducts($products);
}

Alas! As from version 5.3.9 of PHP, there is a new configuration setting called max_input_vars, which limits the number of input variables that may be accepted.
This limit is mainly applied to $_GET, $_POST and $_COOKIE superglobals, but is also used internally by the parse_str() function!
(See PHP manual)

Depending on the php.ini configuration of your host, the number of products linked to a category is therefore limited by this setting...

One solution is to increase the value of max_input_vars, in php.ini or in .htaccess:

<IfModule mod_php5.c>
    php_value max_input_vars 10000
</IfModule>

This can even be done more safely by changing this setting only for admin pages (adapt the LocationMatch pattern to your own back-office URL style):

<LocationMatch "mymagentostore/(index\.php/)?admin/">
    <IfModule mod_php5.c>
        php_value max_input_vars 10000
    </IfModule>
</LocationMatch>

(Source)

This only solves the problem until one of your categories reaches the new max number of products...

Another solution would be to fix Magento's code in order not to use the parse_str() function at this point.
For instance, in Mage_Adminhtml_Catalog_CategoryController, saveAction() method, replace:

parse_str($data['category_products'], $products);

with:

$cat_products_split = explode('&', $data['category_products']);
foreach($cat_products_split as $row) {
    $arr = array();
    parse_str($row, $arr); //This will always work
    list($k, $v) = each($arr);
    if (!empty($k) && !empty($v)) {
        $products[$k] = $v;
    }
}

(Source)

It's hard to decide which solution is the best, since the first one makes your server more vulnerable to attacks (as the max_input_vars is meant to mitigate the possibility of denial of service attacks which use hash collisions), and the second solution makes you modify a Magento core class, which can lead to further problems in future Magento upgrades...

Hope this is useful anyway, I struggled some time before I could find out why some categories kept loosing some products!

6
  • Do you see a problem with increasing this to greater than 10,000? We have some categories with over 31,000 products... May 15, 2013 at 16:29
  • Well, I guess you can raise the limit to any number (yet, understanding that this makes your server more vulnerable to some DoS attacks).
    – tpiazzapw
    May 28, 2013 at 9:56
  • For the second solution, couldn't you just override the controller from a custom module. This would make it upgrade safe since the core file remains unmodified.
    – russjman
    Jul 22, 2013 at 16:40
  • 1
    @russjman look here for a complete implementation of solution 2:magento.stackexchange.com/questions/6319/…
    – WonderLand
    Aug 2, 2013 at 11:08
  • @programmeurweb: your code for solution 2 is not working fine for me (don't know why but it was limiting the category size to 527 prod): I have 'fixed' your code ( it is almost equivalent )
    – WonderLand
    Aug 2, 2013 at 11:09
5

The best solution would be to use observer event "catalog_category_prepare_save". So you need to change config.xml of you module:

<config>
    <global>
        <events>
            ...
            <catalog_category_prepare_save>
                <observers>
                    <your_module>
                         <class>your_module/observer</class>
                         <method>onCatalogCategoryPrepareSave</method>
                    </your_module>
                </observers>
            </catalog_category_prepare_save>

Then add function to app/code/local/Your/Module/Model/Observer.php

<?php
class Your_Module_Model_Observer
{
    /**
     * Fix bug with saving only first 1000 products in a category
     * It's related to default php config 'max_input_vars' = 1000
     *
     * @param Varien_Event_Observer $observer
     */
     public function onCatalogCategoryPrepareSave(Varien_Event_Observer $observer)
     {
         $phpDefaultMaxInputVars = (int)ini_get('max_input_vars');
        /** @var Mage_Core_Controller_Request_Http $request */
        $request = $observer->getRequest();
        $dataProducts = $request->getPost('category_products');
        $dataProducts = preg_split('/&/', $dataProducts, -1, PREG_SPLIT_NO_EMPTY);
        /** @var Mage_Catalog_Model_Category $category */
        $category = $observer->getCategory();

        if ($dataProducts && !$category->getProductsReadonly() && count($dataProducts) > $phpDefaultMaxInputVars) {
            $products = array();
            foreach ($dataProducts as $_product) {
                $_product = trim($_product);
                if (preg_match('/([0-9]*)=([\-]?[0-9]*)/', $_product, $matches)) {
                    $products[$matches[1]] = $matches[2];
                }
            }
            $category->setPostedProducts($products);
        }
    }
}

(Source)

1

I found a working solution so I am sharing it over here !

Create a file name php5.ini in your magento root folder and in the first line just place this code

[PHP]
max_input_vars = 5000

Now if its not working you will need to add this line in you SERVER PHP,ini (/usr/local/lib/php.ini)

max_input_vars = 5000

Hope its helping.

0

<IfModule mod_php5.c> php_value max_input_vars 10000 </IfModule>

did the trick for me, I had the same problem as described above, Magento 1.4.1.1 only saved 1001 products in the category, but after putting the code above in my .htaccess the bug is gone.

0

The solution 2 from programmeurweb is bugged, on Magento 1.8.0.0 !

if (!empty($k) && !empty($v)) {
    $products[$k] = $v;
}

It should be:

if (!empty($k)) {
    $products[$k] = $v;
}

$v is used for 'position' and is not a checkbox value = 1 (as associate product to current category checkbox).

I've tested this already. Am I wrong ?

This reminds of never copy&paste code from net.

PS: I can't comment answers now.

0

LocationMatch is not permitted in .htaccess, it can be only placed in Apache httpd.conf

The prefered way is to create an Apache include and not modify httpd.conf directly as it would break on next updare/rebuild.

In my case I created an include file pre_virtualhost_global.conf and put this code inside:

<LocationMatch "mymagentostore/(index\.php/)?admin/">
<IfModule mod_php5.c>
    php_value max_input_vars 10000
</IfModule>

Afterwards of course, you need to restart apache. I used my WebHostManager so it was an easy process.

More info can be found at:

[http://docs.cpanel.net/twiki/bin/view/EasyApache3/WebHome#Custom Directives to http.conf][1]

I think this is the most elegant solution, doesn't touch Magento Core, applies to all sites on your server and also protects the frontend.

[1]: https://web.archive.org/web/20140804033608/http://docs.cpanel.net:80/twiki/bin/view/EasyApache3/WebHome?#Custom Directives to http.conf

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.