1

So I have this object model:

string Name; // name of the person
int    Age; // age of the person
string CreatedBy; // operator who created person

My query sounds like this: all documents WHERE Age > 40 AND CreatedBy == 'callum' AND Name contains 'll'

CreatedBy is a necessary, scope of control.

Age is also a necessary (but isn't a security issue)

Name is where it can get fuzzy, because that is what the user is querying. Akin to sort of contains

The query below works for the first two parts:

"query": {
     "bool": {
         "must": [
            {
                "range": {
                   "age": {
                      "gt": 40
                   }
                }
            },
            {
                "match": {
                   "createdBy": "Callum"
                }   
            }
         ]
     }
   }

I tried adding a multi_match because ultimately it maybe a search across Name, Address and other bits of information. I couldn't make sense of where to fit it in.

In my, nested queries would be useful. So first filter out all irrelevant users, then filter out irrelevant ages. Then do some fuzzier matching on relevant fields.

2 Answers 2

0

So, the answer to this isn't straightforward.

First of all you need to create an Analyser for Compound Words.

So in the .NET client it looks like:

this.elasticClient.CreateIndex("customer", p => p
    .Settings(s => s
        .Analysis(a => a
            .TokenFilters(t => t
                .NGram("bigrams_filter", ng => ng
                    .MaxGram(2)
                    .MinGram(2)))
        .Analyzers(al => al
            .Custom("bigrams", l => l
                .Tokenizer("standard")
                .Filters("lowercase", "bigrams_filter"))))));

this.elasticClient.Map<Person>(m => m
    .Properties(props => props
    .String(s => s
        .Name(p => p.Name)
        .Index(FieldIndexOption.Analyzed)
        .Analyzer("bigrams"))
    .String(s => s
        .Name(p => p.CreatedBy)
        .NotAnalyzed())
    .Number(n => n
        .Name(p => p.Age))));

Which is a sort of direct translation of the the first link provided. This now means that all names will be broken into their bigram representation:

Callum

  1. ca
  2. al
  3. ll
  4. lu
  5. um

Then you need the actual query to take advantage of this. Now this is bit I like, because we've set up that index on the name column, it means that all term queries can have partial words in them, so take this for example (Sense query):

GET customer/_search
{
    "query": {
        "filtered": {
           "query": {
            "multi_match": {
               "query": "ll",
               "fields": ["name"]
            }
           },
           "filter": {
               "bool": {
                       "must": [
                          {
                         "range": {
                            "age": {
                               "gt": 40
                            }
                         }
                      },
                      {
                         "match": {
                            "createdBy": "Callum"
                         }
                      }
                   ]
               }
           }
        }
    }
}

Here, we have a filtered query. So the query is always the first to be run (can't find documentation yet to cite that, but I have read it), and this will be the partial terms match. Then we simply filter - which is done after the query - to get the subset of results we need.

Because the ngrams analyser is only set on name that is the only column that will be partially matched against. So CreatedBy won't and thus we get our security around the results.

8
  • There is one issue, I dont think there is any match filter! It is only query, and cannot be used within the filter context... You can use term filter. Jun 24, 2016 at 11:13
  • this worked by the way, test and clarified. This is for version 2.x right, docs Jun 24, 2016 at 11:22
  • and also the query is always the first to be run (can't find documentation yet to cite that, but I have read it) it is just the reverse, it will first filter out the document and then apply query on it.. elastic.co/blog/found-optimizing-elasticsearch-searches elastic.co/guide/en/elasticsearch/reference/current/… There wont be any advantage of using filter if elastic would have first performed query on 1000k document and then apply filter... Jun 24, 2016 at 11:22
  • But in 2.0.0-beta1. filtered is deprecated! So you must not be using elastic 2.x! Jun 24, 2016 at 11:24
  • this is weird because even in the doc, it is mentioned that you can use match within filter, match queries are high level full text query! Jun 24, 2016 at 11:26
-1

Basically what you can do is put the query into two blocks:

"query": {
         "filter":{
            "bool":
            {
                "must":[
                    {
                        "range": {
                           "age": {
                              "gt": 40
                           }
                        }
                    }
                ]
            }
          },
         "query":{
            "bool": {
                "must": [
                    {
                      "multi_match" : {
                        "query":    "ll", 
                        "fields": [ "createdBy", "Address","Name" ] ,
                        "fuzziness":2
                      }
                    }             
                ]
            }
          }
       }

What you can do is in filter you can use condtions to filter things out, on then with the filtered data you can apply you multi-match query. The main reason why I included age in filter is because you dont need to perform any kind of free text search, you just need to check with a static value, you can include more conditions within the must block of filter.

You can also look into this article, which might give you some overview.

https://googleweblight.com/?lite_url=https://www.elastic.co/blog/found-optimizing-elasticsearch-searches&ei=EBaRAJDx&lc=en-IN&s=1&m=75&host=www.google.co.in&ts=1465153335&sig=APY536wHUUfGEjoafiVIzGx2H77aieiymw

Hope it helps!

15
  • I think you missed the fact that the multi_match should search for a string containing ll Jun 23, 2016 at 15:31
  • So, did it solved what you asked for? You can see more about multi_match in this link elastic.co/guide/en/elasticsearch/reference/current/… Jun 23, 2016 at 15:35
  • no, it didn't, but I am a lot closer, I used your prompt about splitting into two blocks, and looked at filter and query Jun 23, 2016 at 15:36
  • wildcard has gotten me closer, however, I'm not sure how to perform that on multiple fields Jun 23, 2016 at 15:37
  • You want to a search while typing kind of thing? Jun 23, 2016 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.