How to restore elastic search snapshot another cluster? without repository-s3, repository-hdfs, repository-azure, repository-gcs.
4 Answers
Finally i found the solution.it work fine.please read carefully and do. if you have question contact me waruna94kithruwan@gmail.com.
I have two elastic search cluster.i want a migrate elastic_01 data to elastic_02. i mean elastic_01 snapshot restore to elastic_02.let's go.
Importent
- verify elastic_01 and elastic_02 has this folder "/home/snapshot/".
- if not exist create this folder first.
- set correct permission to this folder.
- please verify elastic_01 and elatic_02 versions same or match.
- [elasticsearch snapshot documentation]: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html
(01) set elastic_01 snapshot settings
$ curl -XPUT '/_snapshot/first_backup' -H 'Content-Type: application/json' -d '{ "type": "fs", "settings": { "location": "/home/snapshot/", "compress": true } }'
(2) add snapshot location to elasticsearch.yml (elastic_01) edit elasticsearch.yml file and add this code line and save.
$ path.repo: ["/home/snapshot/"]
(03) create snapshot (elastic_01)
$ curl -XPUT "/_snapshot/first_backup/snapshot_1?wait_for_completion=true"
(04) set elastic_02 snapshot settings
$ curl -XPUT '/_snapshot/first_backup' -H 'Content-Type: application/json' -d '{ "type": "fs", "settings": { "location": "/home/snapshot/", "compress": true } }'
(05) add snapshot location to elasticsearch.yml (elastic_02) edit elasticsearch.yml file and add this code line and save.
$ path.repo: ["/home/snapshot/"]
(06) create snapshot (elastic_02)
$ curl -XPUT "/_snapshot/first_backup/snapshot_1?wait_for_completion=true"
(07) copy elastic_01 snapshot to >>>> elastic_02
- delete elastic_02 snapshot folder content $ rm -rf /home/snapshot/*
- copy elastic_01 snapshot folder content to elastic_02 snapshot folder
(08) list snapshot
$ curl -XGET '/_snapshot/first_backup/_all?pretty'
- it will show backup indexes and snapshot related data
(09) restore elastic search snapshot
$ curl -XPOST "/_snapshot/first_backup/snapshot_1/_restore?wait_for_completion=true"
NOTE: We need to make parameter "include_global_state" to "true" to restore the template as per link " https://www.elastic.co/guide/en/elasticsearch/client/curator/current/option_include_gs.html"
curl -X POST "localhost:9200/_snapshot/my_backup/snapshot_1/_restore?pretty" -H 'Content-Type: application/json' -d' { "include_global_state": true } '
{ "accepted" : true }
This answer is wrt Elastic Search 7.14
. So, it is possible to host a snapshot repository on a NFS. Since, you would like to restore snapshot of one cluster to another, you would need to meet the following pre-requisites:
- The NFS should be accessible from both source and destination cluster.
- The version of the source and destination cluster should be the same. At most, the destination cluster can be 1 major version higher than the source cluster. Eg: you can restore a 5.x snaphot. to a 6.x cluster, but not a 7.x cluster.
- Ensure that the shared NFS directory is owned by uid:gid = 1000:0 (
elasticsearch
user), and appropriate permissions are given (chmod -R 777 <appropriate NFS directory>
aselasticsearch
user)
Now, I am detailing the steps that you could take to copy the data.
- Create a registry of type
fs
on the source cluster:
PUT http://10.29.61.189:9200/_snapshot/registry1 { "type": "fs", "settings": { "location": "/usr/share/elasticsearch/snapshotrepo", "compress": true } }
- Take a snapshot on the created registry:
PUT http://10.29.61.189:9200/_snapshot/registry1/snapshot_1?wait_for_completion=true { "indices": "employee,manager", "ignore_unavailable": true, "include_global_state": false, "metadata": { "taken_by": "binita", "taken_because": "test snapshot restore" } }
- Create a registry of type
url
on the destination cluster. Typeurl
will ensure that the same registry (in terms of the shared NFS path) will be read-only wrt the destination cluster. Destination cluster can only restore/read snapshot info, but can not write snapshots.
PUT http://10.29.59.165:9200/_snapshot/registry1
{ "type": "url", "settings": { "url": "file:/usr/share/elasticsearch/snapshotrepo" } }
- Restore the snapshot generated from source cluster (in step no 2) to the destination cluster.
POST http://10.29.59.165:9200/_snapshot/registry1/snapshot_1/_restore
For more info, refer : https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-restore-snapshot.html
Your idea is to first create a snapshot on nodeB, then delete its data and overwrite nodeA's data to this location?
But according to elastic's documentation, nodeB should mount the NFS directory in a read-only manner, so it does not have write permissions, such as using the type: url repository.
PUT _snapshot/local
{
"type": "url",
"settings": {
"url": "file:/home/esdata/snapshot"
}
}