Thursday, 2 March 2017

Distributed, Dockerised Applications and Centralised Logging: Using Graylog



Written By: Priyanka Dive
In this demonstration, we will forward logs from a sample app which is dockerized  to a centralize graylog server.


Docker is buzz word nowadays ,everybody want to know more and more about it also merge this with other tools. Once you dockerized your application next step is management of logs , so we are going to look for open source centralized log management tool Graylog.


You can test on AWS EC2 instance like t2.small with Ubuntu 14.04 or on other test machine where we have following things installed.


Pre-requisites:
Git
Docker version 1.13.1
docker-compose version 1.11.1


We are merging two things Graylog and Docker-compose sample application.We can forward docker logs to graylog using gelf driver.
For Graylog setup, we can use this graylog image.We will go with sample example from official docker-compose site . Docker-compose will forward logs to graylog and we will be able to see docker logs in centralized location.


With some simple steps we can test this as follows,


1.Clone repo:
This repository contains docker-compose sample application and graylog compose file.
#git clone https://github.com/priyankadive/graylog-docker-compose.git
#cd graylog-docker-compose


Here, you will find two separate directories for test application and for graylog.


2.Start Graylog:
Make changes in PULIC_IP according to your need.If you want your graylog log data persistent then you need to create external volumes for storing data.


#mkdir /graylog/config
#cd /graylog/config
#wget https://raw.githubusercontent.com/Graylog2/graylog2-images/2.1/docker/config/graylog.conf
#wget https://raw.githubusercontent.com/Graylog2/graylog2-images/2.1/docker/config/log4j2.xml


Now, start graylog server with it’s dependencies.
#cd graylog
#docker-compose up -d


This will start graylog.
Visit http://PUBLIC_IP:9000/ default Login credentials are as follows:
Username : admin
Password : admin


You can change it using  $ echo -n yourpassword | shasum -a 256 Replace the output hash in graylog docker-compose.yml for  GRAYLOG_ROOT_PASSWORD_SHA2 .


3.Configure input to Graylog
We will configure input for out test application as follows:
-Go to Graylog URL http://PUBLIC_IP:9000 login.
-Go to System -> Inputs -> Select Input GELF UDP.
-Provide Title For Example. docker-compose
-Select Check Box for Global ,So it will listen to all.
-Specify Port For this Example Port 12201.
-Keep everything as default.
-Save the configuration.


Input will be running and listening on port 12201.You can change source name if you want.




4.Start sample application.
#cd graylog-docker-compose/composetest
#docker-compose up -d
Sample application will be started.


Check URL http://PUBLIC_IP:5000/ or http://localhost:5000/
It is simple python application which will display text as follows:
Hello World! I have been seen 1 times.


5.Check Logs in Graylog.
Visit URL : http://PUBLIC_IP:9000
In source you can see previously configured Input (docker-compose) from step 3.
Click on 'Show received messages' .


You will see log of sample application as follows.




Graylog is very useful for log management. You can search logs by time stamps.If you want to search logs from last 5 minutes,10 minutes, or specific time frame.You can select the time frame and search logs in it.
You can also save your search query for fast access.We can create dashboard for showing logs of particular application.


Conclusion

In this demonstration we have seen how we can centralize docker-compose application logs in graylog.

No comments:

Post a Comment

Amazon EKS - Kubernetes on AWS

By Komal Devgaonkar Amazon Elastic Container Service for Kubernetes (Amazon EKS), which is highly available and scalable AWS service....