Bring up ELK on Docker Swarm

Assuming there is a working Docker Swarm, this blog describes the steps to bring up an ELK stack on Docker Swarm.

First off, you need to decide if the official ELK Docker images on Docker Hub work for you; Or you would need to use custom images. If the official ones (Elasticsearch, Kibana, Logstash) serve the purpose, you may directly skip to service creation section; Otherwise you need to build the images on all individual nodes in the Swarm cluster or setup your own Docker registry.

Service Creation

All services should be created on the manager node in Swarm cluster. First create an Elasticsearch service called es-master, mapping a host dir /data/es to /usr/share/elasticsearch/data within the container. This also assumes an overlay network es is already existing.

docker service create \
               --network es \
               --name es-master \
               -p 9200:9200 \
               --mount type=bind,source=/data/es,destination=/usr/share/elasticsearch/data \
               elasticsearch

Create Kibana service called kibana, joining into es network. -e option points to es-master. The example command uses a custom Kibana image called kibana/plugin.

docker service create \
               --network es \
               --name kibana \
               -p 5601:5601 \
               -e ELASTICSEARCH_URL=http://es-master:9200 kibana/plugin

To verify the services,

docker service ls

ID            NAME       REPLICAS  IMAGE          COMMAND
5w8v5jksx7h5  kibana     1/1       kibana/plugin  
bpojoyb5wz16  es-master  1/1       elasticsearch  

To see on which node kibana is running,

docker service ps kibana

ID                         NAME      IMAGE          NODE          DESIRED STATE  CURRENT STATE           ERROR
39sadh4cfpqp0zwdh6mbh47er  kibana.1  kibana/plugin  indocgubt104  Running        Running 34 seconds ago  

To launch kibana in a browser, type node_IP:5601 in URL bar. Note that you can use either the IP address of manager node or the worker node actually runs kibana.