1

I'm getting an "Immense Term" exception when trying to index some documents into elasticsearch; the field that's throwing the error uses the "standard" analyzer, which as I understand it should be capping terms at 255 and breaking them into tokens. Is there something I'm missing about that process? If not, what info can help solve the problem; A sample error:

java.lang.IllegalArgumentException: Document contains at least one immense term in field="raw_content" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[60, 33, 68, 79, 67, 84, 89, 80, 69, 32, 104, 116, 109, 108, 32, 80, 85, 66, 76, 73, 67, 32, 34, 45, 47, 47, 87, 51, 67, 47]...', original message: bytes can be at most 32766 in length; got 42807

The mapping for "raw_content" is:

      "raw_content" : {
        "type" : "string",
        "analyzer" : "standard"
      },
2
  • Definitely looks like it isn't being run through the analyzer. Not quite sure why that would be. Does the behavior change if you add "index" : "analyzed" to the mapping?
    – femtoRgon
    Dec 2, 2015 at 23:08
  • That sequence of bytes resolves to <!DOCTYPE html PUBLIC "-//W3C/ (= the first few bytes of your problematic content). Which version of ES are you using?
    – Val
    Dec 3, 2015 at 4:16

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.