Questions tagged [amazon-elasticsearch]

Amazon Elasticsearch Service is a managed Elasticsearch offering as part of Amazon Web Services.

amazon-elasticsearch
Filter by
Sorted by
Tagged with
108 votes
7 answers
67k views

Proper access policy for Amazon Elastic Search Cluster

I've recently started using the new Amazon Elasticsearch Service and I can't seem to figure out the access policy I need so that I can only access the services from my EC2 instances that have a ...
hartfordfive's user avatar
  • 1,275
53 votes
5 answers
52k views

How to access Kibana from Amazon elasticsearch service?

I created Amazon elasticsearch service and populated data into it using logstash, which has been installed on an EC2 instance. On the Amazon elasticservice console page, there will be a link to access ...
AV94's user avatar
  • 1,824
47 votes
5 answers
28k views

ValidationException: Before you can proceed, you must enable a service-linked role to give Amazon ES permissions to access your VPC

I am trying to create a VPC controlled Elastic Search Service on AWS. The problem is I keep getting the error when I run the following code: 'ValidationException: Before you can proceed, you must ...
Michael Young's user avatar
41 votes
3 answers
31k views

Multi tenancy in Elastic Search

We are planning to introduce Elastic search(AWS) for our Multi tenancy application. We have below options, Using One Index Per Tenant Using One Type Per Tenant All Tenants Share One Index with Custom ...
Selvakumar Ponnusamy's user avatar
35 votes
2 answers
2k views

Is there a way to manually set an ElasticSearch document id when inserting via AWS Kinesis Firehose?

I have an AWS Kinesis Firehose Stream set up to feed data into an AWS ElasticSearch cluster, and I can successfully insert documents by sending them to the Firehose Stream, which loads them into ...
Alex Coleman's user avatar
29 votes
5 answers
34k views

Permissions to access ElasticSearch from Lambda?

I'm trying to use Elasticsearch for data storage for a Lambda function connected to Alexa Skills Kit. The Lambda works alright without Elasticsearch but ES provides much-needed fuzzy matching. The ...
Keith's user avatar
  • 590
28 votes
5 answers
45k views

AWS Elasticsearch Service IAM Role based Access Policy

I have been struggling to figure out how to communicate with the Amazon ES service from my EC2 instances. The documentation clearly states that the Amazon ES service supports IAM User & Role ...
nackjicholson's user avatar
24 votes
2 answers
15k views

How do you handle Amazon Kinesis Record duplicates?

According to the Amazon Kinesis Streams documentation, a record can be delivered multiple times. The only way to be sure to process every record just once is to temporary store them in a database ...
Antonio's user avatar
  • 1,575
24 votes
1 answer
13k views

Can I install plugin on Amazon Elasticsearch Service?

How to install elasticsearch-river-mongodb plugin to Amazon Elasticsearch Service?
Jagdish Idhate's user avatar
15 votes
2 answers
20k views

Simplest way to get data from AWS mysql RDS to AWS Elasticsearch?

I have data in an AWS RDS, and I would like to pipe it over to an AWS ES instance, preferably updating once an hour, or similar. On my local machine, with a local mysql database and Elasticsearch ...
Sam Fen's user avatar
  • 5,144
15 votes
2 answers
11k views

Elastic Transport client on AWS Managed ElasticSearch

I am trying to use the AWS Managed ElasticSearch for my project. I have followed https://aws.amazon.com/blogs/aws/new-amazon-elasticsearch-service/ and i am able to start an instance and which is ...
Ysak's user avatar
  • 2,709
15 votes
4 answers
9k views

Does Spring Data Elasticsearch support Amazon Elasticsearch?

From the research I have done it seems that the two do not work together because of HTTP only support for Amazon Elasticsearch. Hoping someone can clarify if in fact it is not something that is ...
code's user avatar
  • 4,143
14 votes
3 answers
6k views

What exactly is the primary_term in elastic search?

After reading a lot of documentation, I understood that the primary_term along with the sequence_number are used for optimistic concurrency control in order to prevent an older version of a document ...
Stephanie's user avatar
  • 143
13 votes
3 answers
21k views

How to talk to aws elasticsearch service using elastic java client?

I have set up a elasticsearch server using AWS elasticsearch service (Not EC2). It gave me an endpoint https://xxx-xxxxxxxx.us-west-2.es.amazonaws.com/ and if I click this endpoint(Note that there is ...
Edmond's user avatar
  • 634
13 votes
2 answers
26k views

AWS S3 Event notification using Lambda function in Java

I am trying to use Lambda function for S3 Put event notification. My Lambda function should be called once I put/add any new JSON file in my S3 bucket. The challenge I have is there are not enough ...
NGR's user avatar
  • 1,240
13 votes
1 answer
3k views

AWS Elasticsearch service: Disable index auto creation (auto_create_index)

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html mentions that auto creation of index can be disabled by Automatic index creation can be disabled by setting action....
H1101's user avatar
  • 183
12 votes
4 answers
35k views

How to connect to AWS Elasticsearch?

I'm trying to connect to AWS Elasticsearch but I always get the following error: Exception in thread "main" org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes ...
Fabian Lurz's user avatar
  • 2,039
12 votes
2 answers
4k views

AWS Elasticsearch & VPC - configuring network access from my fixed IP

I am unable to access AWS Elasticsearch Kibana with a browser. I have set up an Elasticsearch instance within my VPC exactly as described here; https://aws.amazon.com/blogs/aws/amazon-elasticsearch-...
ericTbear's user avatar
  • 153
11 votes
2 answers
9k views

Sorry! Something went wrong during authentication between Kibana and Amazon Cognito

I'm using AWS ElasticSearch with Cognito and getting the following error when trying to login Sorry! Something went wrong during authentication between Kibana and Amazon Cognito. What happened? User:...
thepolina's user avatar
  • 1,264
11 votes
1 answer
508 views

Interacting with Amazon Elasticsearch Service using JAVA SDK

I am having trouble finding examples of how to interact with an amazon elasticsearch cluster using the amazon sdk. Can someone point me to examples? I've found javadocs for things like AWSCredentials ...
user1154644's user avatar
  • 4,581
9 votes
1 answer
3k views

What is the difference between AWS Elasticsearch and AWS CloudSearch?

When should I use AWS Elasticsearch over AWS CloudSearch and vice versa?
johnnyodonnell's user avatar
8 votes
3 answers
13k views

AWS ElasticSearchService index_create_block_exception

I'm trying to create a new index in AWS ElasticSearch cluster after increasing the cluster size and seeing index_create_block_exception. How can i rectify this? I tried searching but did not get exact ...
Abhishek Agarwal's user avatar
8 votes
5 answers
13k views

ElasticSearch error: 'The client noticed that the server is not a supported distribution of Elasticsearch'

New to ElasticSearch. I was following this guide to get things set up: https://john.soban.ski/boto3-ec2-to-amazon-elasticsearch.html I ran the "connect_to_es.py" script there, and oddly it ...
de1337ed's user avatar
  • 3,253
8 votes
2 answers
6k views

Amazon elasticsearch interpretation of FreeStorageSpace metrics

I have 6 instances of type m3.large.elasticsearch and storage type instance. I don't really get what does Average, Minimum, Maximum ..mean here? I am not getting any logs into my cluster right now ...
Karup's user avatar
  • 2,054
8 votes
1 answer
4k views

Install custom plugin for Kibana on AWS ElasticSearch Instance

I want to know if it is possible to add a custom plugin for Kibana running on an AWS instance as mentioned in this link. From the command line we can type, bin/kibana-plugin install some-plugin But, ...
A.I's user avatar
  • 1,438
7 votes
3 answers
5k views

AWS Elasticsearch and CORS

I'm trialing the AWS Elasticsearch service: https://aws.amazon.com/elasticsearch-service/ Very easy to setup. Basically just hit deploy. I unfortunately can't get any of the Elasticsearch GUI's to ...
Kong's user avatar
  • 9,116
7 votes
1 answer
4k views

Amazon's AWS.NodeHttpClient - no documentation :(

I'm writing a lambda function where I have to write logs to Amazon's ElasticSearch Service. Backend is Node.js. I came across this example on github: https://github.com/awslabs/amazon-elasticsearch-...
Nika Kasradze's user avatar
7 votes
4 answers
2k views

Unable to Discover data on Kibana 5

I've set up a test Amazon Elasticsearch Service, which uses Elastic and Kibana 5.1. I'm able to insert a test entry via curl: curl -XPOST "https://mytestservicedomain.amazonaws.com/testindex/...
rtorres's user avatar
  • 2,801
7 votes
0 answers
335 views

Neo4j and AWS ElasticSearch Service integration failed occasionally

I have been using the Neo4j ElasticSearch integration module (https://github.com/neo4j-contrib/neo4j-elasticsearch) on the local, development and staging server given that the elasticsearch service is ...
Kunal Dethe's user avatar
  • 1,254
6 votes
0 answers
2k views

Is there a way to restart (Stop and Start) AWS ElasticSearch?

Not sure why AWS ES does not allow the cluster to be stopped and started. I believe behind the scene its just a set of AWS EC2 instances that themselves could be easily stopped and started. This way ...
Rahul's user avatar
  • 143
5 votes
1 answer
2k views

Is there any difference between using synonyms_path and using synonyms when specifying synonym filter for Elasticsearch?

In term of performance, is there any difference between using synonyms_path and using synonyms when specifying synonym filter for Elasticsearch ? Please refer to the images for example. Note that in ...
laituan245's user avatar
5 votes
1 answer
2k views

AWS Elasticsearch Service Role Permissions

I'm trying to submit documents to an Elasticsearch index from a Python Lambda script running as a role which has the AmazonESFullAccess policy attached. I am creating the requests with the python ...
unclemeat's user avatar
  • 5,097
5 votes
1 answer
1k views

Add multiple domain access policy to AWS Elasticsearch Service (Static IP and Lambda ARN)

After setting up AWS Elasticsearch, I installed Logstash and Kibana proxy on a static IP server, and added this domain access policy on ES and it's working fine: { "Version": "2012-10-17", "...
Jubba Smail's user avatar
  • 1,221
5 votes
0 answers
523 views

Amazon Elasticsearch Service, since upgrade to Amazon Opensearch service is unusably slow

Amazon OpenSearch Service (successor to Amazon Elasticsearch Service), since the upgrade from Amazon Elasticsearch service, the stock dashboards app has been cripplingly slow. Login, as well as most ...
gbegley's user avatar
  • 2,659
4 votes
2 answers
2k views

Security token service exception while restoring snapshot from S3 to AWS managed elasticsearch

I have an AWS managed Elasticsearch Service (say smallES) which has an properly working S3 bucket attached to containing day wise rolling indices of last 1 year. I've created another AWS managed ES ...
DeshErBojhaa's user avatar
4 votes
2 answers
4k views

Unable to resolve AWSRequestSigningApacheInterceptor

Hi All i have downloaded the below code from AWS in order to index the data to my elasticsearch service but it has end up with error i am unable to build the code. package com.Lambda.dynamodb; ...
Mohan vel's user avatar
  • 545
4 votes
1 answer
3k views

Creating a valid signed request to an AWS ES instance from node.js

I'm trying to find an example of how to connect to an AWS ES instance in node.js, and then hit the ES cluster with a simple request. I'm attempting to use the elasticsearch node package, along with ...
GrantD71's user avatar
  • 1,837
4 votes
1 answer
5k views

How to handle domain status of Amazon elasticsearch service?

I created an elasticsearch cluster and uploaded 45 GB of data.There, I tried to changed the access policy of the domain. The domain status has been showing "Processing" for last 24 hours. Is there any ...
AV94's user avatar
  • 1,824
4 votes
0 answers
789 views

How can I add lemmatization to AWS Elastic Search

We used to use a lemmatizer with Sphinx search, but have now moved to AWS Elastic search. How can we add a lemmatizer to aws elastic search so when we search for "be", "are" is matched as well?
Paul Preibisch's user avatar
4 votes
0 answers
388 views

AWS elastic search access - using Presigned URL

I'm using presigned URL with the generated aws signature and I can able to get default search responses like _search. But, I'm unable to query it. Below is my presigned URL :- https://domain-endpoint/...
Pradheep Narendran P's user avatar
3 votes
3 answers
2k views

How can I set compatibility mode for Amazon OpenSearch using CloudFormation?

Since AWS has replaced ElasticSearch with OpenSearch, some clients have issues connecting to the OpenSearch Service. To avoid that, we can enable compatibility mode during the cluster creation. ...
icalvete's user avatar
  • 1,029
3 votes
2 answers
4k views

Global Search in Elastic Search

Working on Elasticsearch, my use case is very straight forward. When a user types in a search box I want to search all of my data set irrespective of field or column or any condition (search all data ...
Kiran's user avatar
  • 868
3 votes
3 answers
4k views

Stream data to amazon elasticsearch using logstash?

So I spinned up a 2 instance Amazon Elasticsearch cluster. I have installed the logstash-output-amazon_es plugin. This is my logstash configuration file : input { file { path => "/...
Karup's user avatar
  • 2,054
3 votes
1 answer
2k views

Constant 503 errors from Elasticsearch inside a VPC when using Kinesis Firehose

I am using the Amazon Kinesis Data Generator to send data to a test Kinesis Firehose Stream for indexing in an Elasticsearch Service cluster. The data generator sends a fairly basic json doc for ...
Jamie Buchanan's user avatar
3 votes
2 answers
3k views

AWS Kinesis Firehose is not sending data to Elasticsearch....IAM permissions?

So I have everything ready and working good except for the last step which sending data from Firehose to Elasticsearch. This is the error I get in Kinesis Firehose Elasticsearch service logs: Error ...
Data_sniffer's user avatar
3 votes
1 answer
4k views

Scripting in AWS elasticsearch

I read online about scripting in aws elacticsearch service. It said that AWS ES doesn't support dynamic scripting so I am writing aggregations using scripts stored in my disk. I wrote the following ...
sidd607's user avatar
  • 1,379
3 votes
3 answers
6k views

Is it possible to transfer data from Redshift to Elasticsearch?

I'm working on something related to Amazon elasticsearch service.For that,I need to get data from Amazon Redshift.The data to be tranfered is huge i.e. 100 GB.Is there any way to get it directly form ...
AV94's user avatar
  • 1,824
3 votes
2 answers
2k views

how to return all documents that has a subset of input array in OpenSearch

I have a document with given structure: { "name" : "WF1", "myIndex" : [3, 4, 5] } Lets say I have 4 of such records - { "name" : "WF1", "myIndex&...
Mohammad Adnan's user avatar
3 votes
1 answer
2k views

AWS Elasticsearch cluster disk space not balanced across data instances

Background I have an AWS managed Elascsearch v6.0 cluster that has 14 data instances. It has time based indices like data-2010-01, ..., data-2020-01. Problem Free storage space is very unbalanced ...
gondalez's user avatar
  • 1,274
3 votes
2 answers
2k views

How to store date range data in elastic search (aws) and search for a range?

I am trying to store hotel room availability in elasticsearch. And then I need to search rooms those are available from a date till another date. I have come up with two ways to store data for ...
JVK's user avatar
  • 3,852

1
2 3 4 5