2

I have a a scheme like this in Solr:

<?xml version="1.0" encoding="UTF-8"?>
<schema name="dovecot-fts" version="3.0">
  <fieldType name="ytext" class="solr.TextField" autoGeneratePhraseQueries="true" positionIncrementGap="100">
    <analyzer type="index">
      <tokenizer class="solr.StandardTokenizerFactory"/>
      <filter class="solr.EdgeNGramFilterFactory" minGramSize="3" maxGramSize="20"/>
      <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/>
      <filter class="solr.WordDelimiterGraphFilterFactory" catenateNumbers="1" generateNumberParts="1" splitOnCaseChange="1" generateWordParts="1" splitOnNumerics="1" catenateAll="1" catenateWords="1"/>
      <filter class="solr.FlattenGraphFilterFactory"/>
      <filter class="solr.LowerCaseFilterFactory"/>
      <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
      <filter class="solr.PorterStemFilterFactory"/>
    </analyzer>
    <analyzer type="query">
      <tokenizer class="solr.StandardTokenizerFactory"/>
      <filter class="solr.SynonymGraphFilterFactory" expand="true" ignoreCase="true" synonyms="synonyms.txt"/>
      <filter class="solr.FlattenGraphFilterFactory"/>
      <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/>
      <filter class="solr.WordDelimiterGraphFilterFactory" catenateNumbers="1" generateNumberParts="1" splitOnCaseChange="1" generateWordParts="1" splitOnNumerics="1" catenateAll="1" catenateWords="1"/>
      <filter class="solr.LowerCaseFilterFactory"/>
      <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
      <filter class="solr.PorterStemFilterFactory"/>
    </analyzer>
  </fieldType>
  <field name="hdr" type="ytext" indexed="true" stored="true"/>
  <uniqueKey>id</uniqueKey>
</schema>

And a record looks like this:

{
    "id": "339/9821f61c4fa04b62fa030s002df11e39/[email protected]",
    "hdr": "...................Some header information...................Date: Fri, 23 Sep 2022 15:24:43 +0300...................Some other header information..................."
}

Now, I want an additional field containing the date in the HDR field to sorting results. To achieve this I tried copying and manipulate hdr field with PatternTokenizerFactory and I can't.

  <fieldType name="ts" class="solr.TextField" positionIncrementGap="100">
    <analyzer>
      <tokenizer class="solr.PatternTokenizerFactory" pattern="Date:\s(\w{3},\s\d{2}\s\w{3,4}\s\d{4}\s\d{2}:\d{2}:\d{2}\s\+\d{4})" group="1"/>
    </analyzer>
  </fieldType>

  <field name="hdr" type="ytext" indexed="true" stored="true"/>
  <field name="received" type="ts" indexed="true" stored="false" required="false"/>
  <copyField source="hdr" dest="received"/>

So I'm waiting for your help, thanks.

1 Answer 1

0

In your schema, you are creating a copy field named received and copying the content of hdr into it. The received field has a specific text analysis but this doesn't change the value of the field. It only changes the way it is indexed. Moreover, you cannot sort documents using a text field.

To achieve your goal you need to transform the value of hdr and copy it into another field of type StrField or Date. It is important to use a non-text type if you want to sort using that field. You can use a custom Update Request Processor Chain: https://solr.apache.org/guide/solr/latest/configuration-guide/update-request-processors.html.

You must insert your custom update chain into the solrconfig.xml. Your best chance is to use the following request processors:

  • solr.CloneFieldUpdateProcessorFactory
  • solr.RegexReplaceProcessorFactory

This is an example that you can use as a starting point:

  <updateRequestProcessorChain name="extract_hdr_date">
    <processor class="solr.CloneFieldUpdateProcessorFactory">
      <str name="source">hdr</str>
      <str name="dest">hdr_date</str>
    </processor>
    <processor class="solr.RegexReplaceProcessorFactory">
        ....
    </processor>
    <processor class="solr.LogUpdateProcessorFactory" />
    <processor class="solr.RunUpdateProcessorFactory" />
  </updateRequestProcessorChain>

Given that you want to use that field for sorting, you might find it useful to enable docvalues on hdr_date: https://solr.apache.org/guide/solr/latest/indexing-guide/docvalues.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.