Monitoring Microservices (Spring Boot + Micrometer + Prometheus + Grafana)

  • 4.2/5
  • 5251
  • Jul 20, 2024

In this article, we will see how to use Micrometer, Actuator, Prometheus, and Grafana to collect, monitor, and alert about microservices' metrics.

Micrometer

Micrometer is a dimensional-first metrics collection facade that allows the application to time, count, and gauge its code with a vendor-neutral API.

It is the metrics collection facility included in Spring Boot 2's actuator.

Micrometer integration with Spring Boot enables a developer to choose one or more monitoring systems to use today, and change his mind later as the need changes without requiring a rewrite of the custom metrics instrumentation.

Spring Boot 2 autoconfigures quite a few metrics out of the box: JVM metrics (memory, GC statistics, thread utilization, classes), CPU usage, request latencies, uptime, Tomcat usage, HikariCP pool metrics, etc.

It provides a vendor-neutral metrics collection API and implementations for a variety of monitoring systems: Datadog, New Relic, JMX, Prometheus, Elasticsearch, CloudWatch, etc.

Spring Boot 2 configures a composite "MeterRegistry" to which any number of registry implementations can be added, allowing us to ship application metrics to more than one monitoring system.

The simplest form of the registry is "SimpleMeterRegistry," but "CompositeMeterRegistry" allows multiple registries to be added.

All a developer needs to do is include a vendor-specific micrometer dependency in the application.

Prometheus

Prometheus is a popular open-source time-series database for collecting and storing time-series data such as performance metrics.

Prometheus uses a pull-based approach for getting metrics. The application doesn't need to know where Prometheus is located or even be concerned if Prometheus has died.

Grafana

Grafana is a multi-platform, open-source analytics and interactive visualisation web application. It provides charts, graphs, and alerts for the web when connected to supported data sources.

Actuator

Actuator is used to expose operational information about the production-ready application—health, metrics, info, dump, env, etc. It uses HTTP endpoints or JMX beans to enable us to interact with it.

Code Changes

Let's create two microservices, "order-service" and "catalog-service," where "order-service" calls a REST API exposed by "catalog-service."

1) Include Micrometre and Actuator

1.1) Dependencies

The "micrometer-registry-prometheus" dependency allows the metrics collected by Micrometer to be exposed in a Prometheus way.

        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-registry-prometheus</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

1.2) Config

The "management.endpoints.web.exposure.include" property tells Spring Boot's Actuator which endpoints it should expose.

management.endpoints.web.exposure.include=health,info,prometheus

Now a number of metrics will be exposed automatically on the actuator endpoint: http://localhost:8081/actuator/prometheus.

2) Add a custom metric

Although the setup so far provides quite a few metrics out of the box, we can also add our own custom metrics.

Micrometer can publish different types of metrics: gauge, histogram, and timer.

Let's add a custom metric to "catalog-service." It will calculate how long the "getProduct(...)" methods take to execute and report that as a metric. We will be using the Micrometer "Timer" metric type.

2.1) Dependencies

In order to add a custom timer, we need to add the "spring-boot-starter-aop" dependency to the project.

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
        </dependency>

2.2) Code changes

If you register the "TimedAspect" bean in the spring context, it will allow Micrometer to add a timer to custom methods.

Next, we need to add the @Timed annotation to the method we want to time.

2.3) Test

In order to check if the custom metric is populating in the actuator response, let's hit the "catalog-service" API, i.e., http://localhost:8081/product/101, and check the actuator endpoint at http://localhost:8081/actuator/prometheus.

Pull metrics in Prometheus

One of the easiest ways to start Prometheus for testing is to run it as a Docker container: https://hub.docker.com/r/prom/prometheus.

1) Configure

Let's add rules in a "prometheus.yml" to scrape metrics from "order-service" (http://localhost:8080/actuator/prometheus) and "catalog-service" (http://localhost:8081/actuator/prometheus) every 5 seconds.

We have used "host.docker.internal" in place of "localhost" so that the "Prometheus" instance can connect to the Docker host from inside a Docker container.

2) Run

Next, we need to run Prometheus and pass this configuration to it.

docker run \
    -p 9090:9090 \
    -v ~/Desktop/prometheus.yml:/etc/prometheus/prometheus.yml \
    prom/prometheus
3) Observe

Let's observe the metrics in Prometheus by pointing to: http://localhost:9090 > Status > Targets.

Go to the "Graph" tab and search for the metric "process_cpu_usage":

Visualize in a Grafana Dashboard

Although Prometheus has a decent UI, Grafana's dashboards are more powerful.

One of the easiest ways to start Grafana for testing is to run it as a Docker container: https://hub.docker.com/r/grafana/grafana.

docker run -d --name=grafana -p 3000:3000 grafana/grafana

Let's point to Grafana: http://localhost:3000/

Use the default admin user credentials as admin/admin.

1) Configure

Let's add Prometheus as a data source: http://host.docker.internal:9090/

Click "Save" and check at the end of the screen; you should see a message that the data source has been connected.

2) Import a Predefined Dashboard

Since we have a Spring Boot app, we can take advantage of one of the pre-configured dashboards for Spring Boot.

Go to Dashboards > New Dashboard > Import.

We can either search for the dashboard or just enter the template id into the Import field (put the id as 12900) and click "load."

Choose Prometheus as the data source and click Import.

3) Visualize

Great! we can now have a complete Spring Boot dashboard:

Source code: GitHub

Index
How to Implement PostgreSQL Full-Text Search with Spring Boot

15 min

Spring's transaction management with the @Transactional annotation

9 min

Spring Boot Rest APIs with PostgreSQL (Spring Boot + Rest APIs)

15 min

Caching in Spring Boot (@Cacheable, @CacheEvict & @CachePut)

21 min

Declarative REST Client in Spring Boot (Spring 6 HTTP Interface)

13 min

Circuit Breaker in Spring Boot (Spring Cloud Circuit Breaker + Resilience4j)

12 min

Handling Concurrent Service Calls in a Spring Boot Application: Asynchronous vs. Reactive Approaches

11 min

Profiling a Spring Boot application with Pyroscope

7 min

Service discovery in Spring Boot (Spring Cloud + Netflix Eureka)

9 min

Dockerize Spring Boot app and Push image to DockerHub (Spring Boot + DockerHub)

4 min

Creating a Jenkins Pipeline for Spring Boot application

2 min

Monitoring Microservices (Spring Boot + Micrometer + Prometheus + Grafana)

7 min

Edge Server Pattern in Microservices (Spring Cloud Gateway)

7 min

Circuit Breaker Pattern in Microservices (Spring BOOT + Resilience4j)

4 min

Spring Cloud config server setup with Git

8 min

Distributed Tracing in Microservices (Spring Cloud Sleuth + Zipkin)

9 min

Circuit Breaker Pattern with Resilience4J in a Spring Boot Application

24 min

Deploying Spring Boot microservices on Kubernetes Cluster

12 min

Reactive programming in Java with Project Reactor

50 min

Spring Reactive with PostgreSQL (Spring Boot WebFlux + PostgreSQL)

13 min

Spring Reactive, Thymeleaf Hello World (Spring Webflux + Thymeleaf + JS/CSS)

9 min

Problem JSON (application/problem+json) in Spring WebFlux

15 min

Spring Boot Login/Logout (Spring Security + MySql + Thymeleaf)

21 min

Securing Server-to-Server Communication with "Spring Boot" & "OAuth 2"

18 min

Integrating Elasticsearch with a Spring Boot and PostgreSQL application

16 min

Sending Emails in Spring Boot via SMTP

7 min

How to create a basic Spring 6 project using Maven

5 min

Spring Boot, Thymeleaf Hello World (Spring Boot + Thymeleaf + JS/CSS)

9 min