84

I am a little confused as to what the behaviour of the index and stored attibutes of the Solr fields is.

For example if I have the following in the Schema.xml

<field name="test1" type="text" indexed="false"
        stored="false" required="false" />

Will the field test1 be not stored in the Solr document even if I create a document with that field in it and set a value to that field and commit the document to Solr. As I have the stored=false attribute, does it mean that the value of the field is lost in Solr and not persisted?

5 Answers 5

151

That is correct. Typically you will want your field to be either indexed or stored or both. If you set both to false, that field will not be available in your Solr docs (either for searching or for displaying). See Alexandre's answer for the special cases when you will want to set both to false.

As stated here : indexed=true makes a field searchable (and sortable and facetable). For eg, if you have a field named test1 with indexed=true, then you can search it like q=test1:foo, where foo is the value you are searching for. If indexed=false for field test1 then that query will return no results, even if you have a document in Solr with test1's value being foo.

stored=true means you can retrieve the field when you search. If you want to explicitly retrieve the value of a field in your query, you will use the fl param in your query like fl=test1 (Default is fl=* meaning retrieve all stored fields). Only if stored=true for test1, the value will be returned. Else it will not be returned.

2
  • 2
    How can I favorite your answer? :) Commented Sep 14, 2017 at 23:06
  • 1
    I think you don't need to index for sorting, faceting. You can set docValues=true.
    – Gibbs
    Commented Apr 27, 2020 at 10:16
35

The main point of having both set to false is to explicitly skip that particular field.

For example, if you have a storing/indexing dynamicField mapping and you want to ignore one particular name that would otherwise fall under dynamicField's pattern.

Alternatively you could use dynamicField to ignore a whole set of fields with same prefix/suffix that comes from a 3rd party. For example, Tika will send you a whole bunch of metadata fields which you may just want to ignore. See this defined in Solr's example schema.xml and used in solrconfig.xml

In the later versions of Solr, you could also use IgnoreFieldUpdateProcessorFactory (see full list for others) instead, which will get rid of those fields even earlier in the indexing process.

6
  • 1
    Didn't realize this use case for a field that has both indexed and stored set to false.
    – arun
    Commented Mar 10, 2014 at 17:51
  • 5
    If you read example configurations line-by-line, you learn a lot of weird and wonderful things. Commented Mar 12, 2014 at 7:01
  • If i store index only and not field value than will it make impact on performance as i can always store field values in some other DB and return the data from there once Solr has given me search result. Will it help me to reduce the index file size and better performance
    – viren
    Commented Mar 31, 2017 at 11:10
  • 2
    Solr documentation allows in-place updates of a fields that are both set to false. Is there any sense for that? What practical sense of updating fields that are non-searchable and non-storable? Commented Jun 9, 2017 at 8:14
  • 2
    Because that particular example has docValues enabled, which stores content in a different way yet again. And you can return docValue even if stored is set to false. This is a new Solr functionality (6+) that was not present when the question above was answered. Commented Jun 9, 2017 at 18:33
15

Quoting from this response in the Solr's mail thread:

"indexed" and "stored" are independent, orthogonal attributes - you can use any of the four combinations of true and false. "indexed" is used for search or query, the "lookup" portion of processing a query request. Once the search/query/lookup is complete and a set of documents is selected, "stored" is the set of fields whose values are available for display or return with the Solr response.

Part of the reason for the separation is that Solr/Lucene "analyzes" or transforms the input data into a more efficient form for faster and more relevant search/lookup. Unfortunately, that analyzed/transformed data is frequently no longer suitable for display and human consumption. In other words the analysis/transformation is not bidirectional/reversible. Setting "stored=true" guarantees that the original data can be retrieved in its original form.

1
  • 5
    "Analyzed/transformed data is frequently no longer suitable for display and human consumption..." I wondered why we can't display field's value if it's set to index="true", but stored="false". You've cleared it up. Thank you!
    – AlexMelw
    Commented Mar 2, 2018 at 6:19
3

If both are false you loose your data in that field. If indexed true, the data are searchable but it can not be displayed. If you set stored true you will not be able to search on that field but it can be displayed (in this case you can write copyfield rule to copy the info from that field to the default searchable field). Both set as true -> you can search and display.

1
  • How is this different from the existing answers?
    – MatsLindh
    Commented Apr 19, 2018 at 6:54
0

indexed = true means that this field can be used in the search. For example, if I set the item field as follows and I try to perform the field in a search

<field name="item" type="text_general" uninvertible="true" indexed="false" stored="true"/>

fq = item: "Tennis" will mark an error.

stored = true means that this field can be retrieved in the list of fields displayed after a query. For example, if the item field is defined as follows

<field name="item" type="text_general" uninvertible="true" indexed="true" stored="false"/>

You will be able to search fq = item: "Tennis" correctly, but it will not return the item field in the results.

Regards

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.