Backing up Elasticsearch 7 to Azure container storage

What an exciting post….

So you’ve created an ElasticSearch 7 standalone cluster, in Azure, and wish to use the Azure Repository plugin to handle backups.

You’re also conscious about not exposing storage accounts on public IP addresses (if they need not be)….

So, given you’ve created a storage account within Azure called e.g. mysimplestorage and given it a private-link connection to the same virtual network your ElasticSearch Servers will be on … you should find it has a hostname like mysimplestorage.blob.core.windows.net (see the ‘Endpoints’ “tab” within the storage account. That ‘mysimplestorage‘ is needed below.

What else needs doing ?

Well,

a) you don’t want proxy settings in your elasticsearch.yml file (azure.client.default.proxy.host: …. and other things)

b) You probably do want to create a Managed Identity to attach to your virtual machines, which gives them access to to the storage account (I called mine ‘elasticsearch7’ as I have no imagination). Assign that role ‘owner’ for the given storage account (perhaps there is a finer grained permission?)

c) When creating new VMs for the cluster, you can use :

az vm identity assign -g MyResourceGroup -n NewServerName –identities elasticsearch7

d) Edit the Storage Account’s access control list, and add the role in

e) install the azure repository plugin on your elasticsearch nodes

/usr/share/elasticsearch/bin/elasticsearch-plugin install -b repository-azure

f) Add stuff to the ElasticSearch keystore –

echo mysimplestorage | /usr/share/elasticsearch/bin/elasticsearch-keystore add –stdin azure.client.default.account

echo “somekey” | /usr/share/elasticsearch/bin/elasticsearch-keystore add –stdin azure.client.default.key

chown elasticsearch:elasticsearch /etc/elasticsearch/elasticsearch.keystore

At this point, restart all your ElasticSearch nodes (for me they don’t seem to pickup keystore settings until a restart) and then go into Kibana and configure the backup routine.