Lecture 256: Hands-on: configuring logging with Logstash and Elasticsearch

Logging in microservices isn't just a trick to look professional — it's a vital tool. When your microservices start talking to each other, you need to be sure you can easily trace where something went wrong. Think of your code as a theatre of code, and logs as the backstage diary. The ELK stack (Elasticsearch, Logstash, Kibana) makes that diary readable, searchable, and easy to analyze.

Why ELK?

The more your apps communicate, the crazier log monitoring gets. ELK solves this with:

• Elasticsearch — a powerful search-optimized database.
• Logstash — a versatile log shipper.
• Kibana — a visualization layer for logs with pretty dashboards.

And the best part: instead of reading JSON outputs in your terminal you can "admire the charts" in Kibana. Sounds nice? Let's dive in!

Setup steps

Installing the ELK stack components

1. Elasticsearch:
   Download Elasticsearch and run it locally. Command to start after unpacking:

   ./bin/elasticsearch
   

   Make sure it's running by visiting http://localhost:9200 in your browser.
2. Logstash:
   Download Logstash and get it battle-ready. You'll need to configure Logstash with a config file — more on that shortly.
3. Kibana:
   Download Kibana. After you start Kibana it'll be available at http://localhost:5601.

Preparing the Spring Boot application

Create a Spring Boot app with a basic logging structure. Add the following dependencies to your pom.xml (or build.gradle):

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-log4j2</artifactId>
    </dependency>
</dependencies>

Why Log4j2? It's flexible and plays nicely with Logstash.

Project setup for logging

log4j2 configuration

Create a file log4j2.xml in src/main/resources. Example configuration:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <!-- Console appender -->
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} [%t] %-5level %logger{36} - %msg%n" />
        </Console>

        <!-- Logstash appender -->
        <Socket name="Logstash" host="localhost" port="5044">
            <JsonLayout complete="true" compact="true" eventEol="true" />
        </Socket>
    </Appenders>

    <Loggers>
        <Root level="info">
            <AppenderRef ref="Console" />
            <AppenderRef ref="Logstash" />
        </Root>
    </Loggers>
</Configuration>

Here we configured:

1. Logs are printed to the console.
2. Logs are sent to Logstash over a socket.

Logstash configuration

Create a file logstash.conf:

input {
  tcp {
    port => 5044
    codec => json
  }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "application-logs"
  }
  stdout {
    codec => rubydebug
  }
}

Here's what's happening:

• input — Logstash is waiting for data over TCP on port 5044.
• output — logs are stored in Elasticsearch in the application-logs index and printed to the console for curious eyes.

Start Logstash with this config file:

./bin/logstash -f logstash.conf

Verification

1. Writing code: In your Spring Boot app add a couple of logs:

   
   import org.slf4j.Logger;
   import org.slf4j.LoggerFactory;
   import org.springframework.web.bind.annotation.GetMapping;
   import org.springframework.web.bind.annotation.RestController;
   
   @RestController
   public class LogController {
       private final Logger logger = LoggerFactory.getLogger(LogController.class);
   
       @GetMapping("/log")
       public String generateLog() {
           logger.info("Hey, ELK! This is my first log.");
           logger.error("Houston, we have a problem!");
           return "Log sent";
       }
   }
   

2. Run: Start your Spring Boot app, open a browser, and navigate to:

   http://localhost:8080/log
   

3. Check in Kibana:
   • Go to Kibana (http://localhost:5601).
   • Open Management -> Index Patterns.
   • Set up an index pattern for application-logs.
   • Go to Discover, and you'll see all your lovely logs.

Useful tips

1. Always test the config with small logs before rolling it out to production. Broken logging in a real system is a pain.
2. Make sure your Logstash can handle the load. If it can't, use clustering or alternatives like Fluentd.
3. Set up index rotation in Elasticsearch so your clusters don't drown in gigabytes of data.

Now that you've completed this stage, you have a powerful tool not only for collecting logs but also for hunting down issues in your microservices. In the next lecture we'll look at monitoring metrics with Prometheus — for now, enjoy exploring your logs in ELK!