1

I was using N1QL to read data from my couchbase db and was encountering very bad performance. I'm working with views atm, but if anyone has an idea why this happens, I'm happy to know and maybe I'll go back to N1QL. While the pagination is very slow with 2M records (but works), the paginated search times out @ 2M records. Couchbase CE 4.1.0

Here is the Query:

public static function findByPage($recordsPerPage, $page) {
        $query = CouchbaseN1qlQuery::fromString('SELECT * FROM `public_portal` WHERE `collection`=$collection ORDER BY `_id` LIMIT $limit OFFSET $offset');
        $query->options['$collection'] = static::COLLECTION_NAME;       
        $query->options['$limit'] = $recordsPerPage;        
        $query->options['$offset'] = $recordsPerPage*($page-1);     
        return self::doQueryAndGetObjects($query);
    }

The indexes:

CREATE INDEX `public_portal_collection` ON `public_portal`(`collection`) USING GSI;

CREATE INDEX `public_portal_id` ON `public_portal`(`_id`) USING GSI;

My explain:

cbq> EXPLAIN SELECT * FROM `public_portal` WHERE `collection`="tree" ORDER BY `_id` LIMIT 24 OFFSET 24;
{
    "requestID": "ab6df326-8f33-48b6-84a4-c22ac394f803",
    "signature": "json",
    "results": [
        {
            "#operator": "Sequence",
            "~children": [
                {
                    "#operator": "Sequence",
                    "~children": [
                        {
                            "#operator": "IndexScan",
                            "index": "public_portal_collection",
                            "keyspace": "public_portal",
                            "namespace": "default",
                            "spans": [
                                {
                                    "Range": {
                                        "High": [
                                            "\"tree\""
                                        ],
                                        "Inclusion": 3,
                                        "Low": [
                                            "\"tree\""
                                        ]
                                    }
                                }
                            ],
                            "using": "gsi"
                        },
                        {
                            "#operator": "Parallel",
                            "~child": {
                                "#operator": "Sequence",
                                "~children": [
                                    {
                                        "#operator": "Fetch",
                                        "keyspace": "public_portal",
                                        "namespace": "default"
                                    },
                                    {
                                        "#operator": "Filter",
                                        "condition": "((`public_portal`.`collection`) = \"tree\")"
                                    },
                                    {
                                        "#operator": "InitialProject",
                                        "result_terms": [
                                            {
                                                "expr": "self",
                                                "star": true
                                            }
                                        ]
                                    }
                                ]
                            }
                        }
                    ]
                },
                {
                    "#operator": "Order",
                    "sort_terms": [
                        {
                            "expr": "(`public_portal`.`_id`)"
                        }
                    ]
                },
                {
                    "#operator": "Offset",
                    "expr": "24"
                },
                {
                    "#operator": "Limit",
                    "expr": "24"
                },
                {
                    "#operator": "FinalProject"
                }
            ]
        }
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "6.755603ms",
        "executionTime": "6.573912ms",
        "resultCount": 1,
        "resultSize": 2972
    }
}

This was done with 4000x5 records.

"Collection" is what I call "type".

1 Answer 1

1

The query uses order by and query engine needs to fetch all the records and sort before returning documents even though limit value is small, Due to that it taking time.

What type of time out you are seeing. Is it from indexer or query. Could you post the timeout message.

In 4.5.0 this type of queries performs much better.

1
  • For some reason it doesn't timeout anymore, it just returns nothing. It is a little faster without the ORDER BY but I need it. I'll wait for 4.5.0.
    – Mattia_98
    Jul 28, 2016 at 9:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.