17

I'm trying to use the latest version of Elastic Search library which is already available in the maven repos.

<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>7.0.0</version>
</dependency>

But not sure how can I use the 7th version with Spring Boot which imports 6.5. My maven dependency:

  <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
  </dependency>
7
  • Instead of spring-boot-starter-data-elasticsearch dependency add the elasticsearch (version 7.0.0) version in your pom.xml file and you can use it. Commented Apr 19, 2019 at 12:49
  • This answer might help. Replace the version number to your needs.
    – Nishant
    Commented Apr 20, 2019 at 1:57
  • 2
    for my case I switched to using RestHighLevelClient by elastic instead of the spring data elastic search Commented May 24, 2019 at 21:28
  • 1
    @SarvarNishonboev here is a link to the pull request on GitHub. We have to wait I guess... github.com/spring-projects/spring-data-elasticsearch/pull/284
    – tech4242
    Commented Jul 1, 2019 at 15:37
  • 1
    As far as I can see it is meant to be supported in Spring Data Elasticsearch 4.x release. Anybody has idea about the estimated release date of that version? Commented Sep 8, 2019 at 9:25

2 Answers 2

15

UPDATE

Spring Boot 2.3 is integrating spring-data-elasticsearch 4 so it will support ElasticSearch 7.x out of the box. It will be released soon but you can already try it:

plugins {
  id 'org.springframework.boot' version '2.3.0.RC1'
  id 'io.spring.dependency-management' version '1.0.9.RELEASE'
}

I have tested it positively and all of my test scenarios are passing so I would definitely recommend this way. I will keep the answer below for people that for some reasons can't upgrade to 2.3.

OLD WORKAROUND (previous versions original answer)

As we don`t really know when Spring Data Elastic Search 4.x is going to be released I am posting my way of integrating the current Spring Data Elastic Search 4.x and stable Spring Boot 2.1.7. It might work as a temporary workaround for you if you want to work with the Spring Repositories and the newest Elastic Search.

1) Force newest elastic search client in your dependencies (in my case: build.gradle)

dependencies {
    //Force spring-data to use the newest elastic-search client
    //this should removed as soon as spring-data-elasticsearch:4.0.0 is released!
    implementation('org.springframework.data:spring-data-elasticsearch:4.0.0.BUILD-SNAPSHOT') {
        exclude group: 'org.elasticsearch'
        exclude group: 'org.elasticsearch.plugin'
        exclude group: 'org.elasticsearch.client'
    }

    implementation('org.elasticsearch:elasticsearch:7.3.0') { force = true }
    implementation('org.elasticsearch.client:elasticsearch-rest-high-level-client:7.3.0') { force = true }
    implementation('org.elasticsearch.client:elasticsearch-rest-client:7.3.0') { force = true }
}

2) Disable the Elastic Search auto configuration and health check components as they become incompatible (you may later want to implement your own health check).

@SpringBootApplication(exclude = {ElasticsearchAutoConfiguration.class, ElasticSearchRestHealthIndicatorAutoConfiguration.class})
@EnableElasticsearchRepositories
public class SpringBootApp {

    public static void main(String[] args) {
        SpringApplication.run(SpringBootApp.class, args);
    }

}

3) As we disabled the auto configuration we need to initialize the ElasticsearchRestTemplate ourselves. We also need to do it to provide the custom MappingElasticsearchConverter to avoid class incompatibilities.

/**
 * Manual configuration to support the newest ElasticSearch that is currently not supported by {@link org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchAutoConfiguration}.
 *
 * @author aleksanderlech
 */
@Configuration
@EnableConfigurationProperties(ElasticsearchProperties.class)
public class ElasticSearchConfiguration {

    @Primary
    @Bean
    public ElasticsearchRestTemplate elasticsearchTemplate(ElasticsearchProperties configuration) {
        var nodes =  Stream.of(configuration.getClusterNodes().split(",")).map(HttpHost::create).toArray(HttpHost[]::new);
        var client = new RestHighLevelClient(RestClient.builder(nodes));
        var converter = new CustomElasticSearchConverter(new SimpleElasticsearchMappingContext(), createConversionService());
        return new ElasticsearchRestTemplate(client, converter, new DefaultResultMapper(converter));
    }

    private DefaultConversionService createConversionService() {
        var conversionService = new DefaultConversionService();
        conversionService.addConverter(new StringToLocalDateConverter());
        return conversionService;
    }
}

CustomElasticSearchConverter:

/**
 * Custom version of {@link MappingElasticsearchConverter} to support newest Spring Data Elasticsearch integration that supports ElasticSearch 7. Remove when Spring Data Elasticsearch 4.x is released.
 */
class CustomElasticSearchConverter extends MappingElasticsearchConverter {

    private CustomConversions conversions = new ElasticsearchCustomConversions(Collections.emptyList());

    CustomElasticSearchConverter(MappingContext<? extends ElasticsearchPersistentEntity<?>, ElasticsearchPersistentProperty> mappingContext) {
        super(mappingContext);
        setConversions(conversions);
    }

    CustomElasticSearchConverter(MappingContext<? extends ElasticsearchPersistentEntity<?>, ElasticsearchPersistentProperty> mappingContext, GenericConversionService conversionService) {
        super(mappingContext, conversionService);
        setConversions(conversions);
    }

    @Override
    protected <R> R readValue(@Nullable Object source, ElasticsearchPersistentProperty property,
                              TypeInformation<R> targetType) {

        if (source == null) {
            return null;
        }

        if (source instanceof List) {
            return readCollectionValue((List) source, property, targetType);
        }

        return super.readValue(source, property, targetType);
    }

    private Object readSimpleValue(@Nullable Object value, TypeInformation<?> targetType) {

        Class<?> target = targetType.getType();

        if (value == null || target == null || ClassUtils.isAssignableValue(target, value)) {
            return value;
        }

        if (conversions.hasCustomReadTarget(value.getClass(), target)) {
            return getConversionService().convert(value, target);
        }

        if (Enum.class.isAssignableFrom(target)) {
            return Enum.valueOf((Class<Enum>) target, value.toString());
        }

        return getConversionService().convert(value, target);
    }


    private <R> R readCollectionValue(@Nullable List<?> source, ElasticsearchPersistentProperty property,
                                      TypeInformation<R> targetType) {

        if (source == null) {
            return null;
        }

        Collection<Object> target = createCollectionForValue(targetType, source.size());

        for (Object value : source) {

            if (isSimpleType(value)) {
                target.add(
                        readSimpleValue(value, targetType.getComponentType() != null ? targetType.getComponentType() : targetType));
            } else {

                if (value instanceof List) {
                    target.add(readValue(value, property, property.getTypeInformation().getActualType()));
                } else {
                    target.add(readEntity(computeGenericValueTypeForRead(property, value), (Map) value));
                }
            }
        }

        return (R) target;
    }

    private Collection<Object> createCollectionForValue(TypeInformation<?> collectionTypeInformation, int size) {

        Class<?> collectionType = collectionTypeInformation.isCollectionLike()//
                ? collectionTypeInformation.getType() //
                : List.class;

        TypeInformation<?> componentType = collectionTypeInformation.getComponentType() != null //
                ? collectionTypeInformation.getComponentType() //
                : ClassTypeInformation.OBJECT;

        return collectionTypeInformation.getType().isArray() //
                ? new ArrayList<>(size) //
                : CollectionFactory.createCollection(collectionType, componentType.getType(), size);
    }

    private ElasticsearchPersistentEntity<?> computeGenericValueTypeForRead(ElasticsearchPersistentProperty property,
                                                                            Object value) {

        return ClassTypeInformation.OBJECT.equals(property.getTypeInformation().getActualType())
                ? getMappingContext().getRequiredPersistentEntity(value.getClass())
                : getMappingContext().getRequiredPersistentEntity(property.getTypeInformation().getActualType());
    }

    private boolean isSimpleType(Object value) {
        return isSimpleType(value.getClass());
    }

    private boolean isSimpleType(Class<?> type) {
        return conversions.isSimpleType(type);
    }

}
4
  • 1
    Hi Aleksander, thx, just tested your approach and it works fine for me. Previous issues with repositories findBy methods are gone. Nevertheless, one question: Do you just ignore the following warning?: [299 Elasticsearch-7.3.0-de777fa "[types removal] Specifying types in document index requests is deprecated or do you have a solution for that? Commented Nov 13, 2019 at 15:56
  • This does not work for me. Throws: "ClassNotFoundException: org.springframework.data.mapping.model.EntityInstantiators" Commented Apr 12, 2020 at 22:36
  • Probably you used a never snapshot, I am going to update my answer anyway as now its a better solution available Commented Apr 14, 2020 at 4:08
  • Unfortunately it doesn't work with org.springframework.data:spring-data-elasticsearch:4.0.0.RELEASE
    – Blink
    Commented Nov 13, 2020 at 13:32
0

If anyone is using Spring Boot 2.1.2 and Kotlin, the following code may help you. I just translated it from @Alexander Lech answer, with some small changes:

First change to Alexanders Answer:

@SpringBootApplication(exclude = [ElasticsearchAutoConfiguration::class, 
ElasticsearchDataAutoConfiguration::class])

I had to exclude ElasticsearchDataAutoConfiguration, to make it work.

Second: Since we use Kotlin, and the custom converter is a lot of code, perhaps this translation to Kotlin will help somebody:

class CustomElasticSearchConverter(mappingContext: MappingContext<out ElasticsearchPersistentEntity<*>, ElasticsearchPersistentProperty>, customConversionService: GenericConversionService?) : MappingElasticsearchConverter(mappingContext, customConversionService) {

    private val conversionsNew = ElasticsearchCustomConversions(emptyList<Any>())

    init {
        setConversions(conversionsNew)
    }

    override fun <R : Any?> readValue(source: Any?, property: ElasticsearchPersistentProperty, targetType: TypeInformation<R>): R? {
        if (source == null) {
            return null
        }

        if (source is Collection<*>) {
            return readCollectionValue(source, property, targetType) as R?;
        }

        return super.readValue(source, property, targetType);
    }

    private fun readCollectionValue(source: Collection<*>?, property: ElasticsearchPersistentProperty, targetType: TypeInformation<*>): Any? {

        if (source == null) {
            return null
        }

        val target = createCollectionForValue(targetType, source.size)

        for (value in source) {
            require(value != null) { "value must not be null" }

            if (isSimpleType(value)) {
                target.add(readSimpleValue(value, if (targetType.componentType != null) targetType.componentType!! else targetType))
            } else {
                if (value is MutableCollection<*>) {
                    target.add(readValue(value, property, property.typeInformation.actualType as TypeInformation<out Any>))
                } else {
                    @Suppress("UNCHECKED_CAST")
                    target.add(readEntity(computeGenericValueTypeForRead(property, value), value as MutableMap<String, Any>?))
                }
            }
        }

        return target
    }

    private fun readSimpleValue(value: Any?, targetType: TypeInformation<*>): Any? {

        val target = targetType.type

        @Suppress("SENSELESS_COMPARISON")
        if (value == null || target == null || ClassUtils.isAssignableValue(target, value)) {
            return value
        }

        if (conversionsNew.hasCustomReadTarget(value.javaClass, target)) {
            return conversionService.convert(value, target)
        }

        @Suppress("UNCHECKED_CAST")
        return when {
            Enum::class.java.isAssignableFrom(target) -> enumByName(target as Class<Enum<*>>, value.toString())
            else -> conversionService.convert(value, target)
        }
    }

    private fun enumByName(target: Class<Enum<*>>, name: String): Enum<*> {
        val enumValue = target.enumConstants.find { it.name == name }
        require(enumValue != null) { "no enum value found for name $name and targetClass $target" }
        return enumValue
    }

    private fun createCollectionForValue(collectionTypeInformation: TypeInformation<*>, size: Int): MutableCollection<Any?> {

        val collectionType = when {
            collectionTypeInformation.isCollectionLike -> collectionTypeInformation.type
            else -> MutableList::class.java
        }

        val componentType = when {
            collectionTypeInformation.componentType != null -> collectionTypeInformation.componentType
            else -> ClassTypeInformation.OBJECT
        }

        return when {
            collectionTypeInformation.type.isArray -> ArrayList(size)
            else -> CollectionFactory.createCollection(collectionType, componentType!!.type, size)
        }
    }

    private fun computeGenericValueTypeForRead(property: ElasticsearchPersistentProperty, value: Any): ElasticsearchPersistentEntity<*> {

        return when {
            ClassTypeInformation.OBJECT == property.typeInformation.actualType -> mappingContext.getRequiredPersistentEntity(value.javaClass)
            else -> mappingContext.getRequiredPersistentEntity(property.typeInformation.actualType!!)
        }
    }

    private fun isSimpleType(value: Any): Boolean {
        return isSimpleType(value.javaClass)
    }

    private fun isSimpleType(type: Class<*>): Boolean {
        return conversionsNew.isSimpleType(type)
    }

}

After this, problems with some repository queries where solved. Please also be aware not to use spring-boot-starter-data-elasticsearch but spring-data-elasticsearch:4.0.0.BUILD-SNAPSHOT. (This took me some time).

Yes, the code is ugly, but after spring-data-elasticsearch:4.0.0 is released, you can throw it away.

2
  • Hi , I am trying to use the code mentioned above but I am getting the following error 2019-11-26 14:33:45,287 ERROR [restartedMain] org.springframework.data.elasticsearch.repository.support.AbstractElasticsearchRepository : failed to load elasticsearch nodes : org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=mapper_parsing_exception, reason=No type specified for field [properties]]
    – sivacham
    Commented Nov 26, 2019 at 10:56
  • 1
    OK, first of all I noticed, that using 4.0.0-SNAPHOST is a bit unsable, I would recommend to use "4.0.0.DATAES-690-SNAPSHOT", because the 4.0.0-SNAPSHOT changes too often. Second, to your problem: It seams that you are using an uknown type in your @Document and you need to annotate it with @Field(type = FieldType.<AnyfieldType>). Perhaps that could help your mapping. Commented Nov 27, 2019 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.