Integrating Elasticsearch with a Spring Boot and PostgreSQL application

  • 4.4/5
  • 560
  • Nov 08, 2024

In this article, we will integrate Elasticsearch into a Spring Boot application to enable full-text search and automatic data updates based on PostgreSQL data. Specifically, we will cover the following:

1) How to integrate Elasticsearch into a Spring Boot application.

2) How to automatically add, update, and delete data in the Elasticsearch index based on operations on PostgreSQL data.

3) How to interact with Elasticsearch to perform CRUD operations and full-text searches using Spring Data Elasticsearch.

1) Add Dependencies

Add the required dependencies for Elasticsearch in your pom.xml.

2) Configure Elasticsearch

Ensure Elasticsearch is running on your local machine or adjust the URL if it’s hosted elsewhere.

3) Create an Elasticsearch/JPA Entity

This entity represents a row in PostgreSQL and a document in Elasticsearch. It will be indexed in Elasticsearch to enable efficient search queries.

This class represents an Article entity, which functions as both a row in a PostgreSQL database and a document in an Elasticsearch index. It is designed to be indexed in Elasticsearch, enabling efficient search capabilities. The class also uses an EntityListener to handle specific lifecycle events.

@Document(indexName = "articles"): This annotation indicates that Article is also an Elasticsearch document, which will be stored in an Elasticsearch index named articles.

@EntityListeners(ArticleEntityListener.class): This annotation registers an entity listener (ArticleEntityListener) to handle JPA entity lifecycle events such as persist or update.

4) Synchronizing PostgreSQL and Elasticsearch

The ArticleEntityListener class is an entity listener that listens to specific lifecycle events (persist, update, and remove) on the Article entity.

Its purpose is to synchronize changes between the PostgreSQL database and the Elasticsearch index by updating the Elasticsearch index whenever there is a change in the PostgreSQL data.

5) Create an Elasticsearch Repository

Define a repository for Elasticsearch using ElasticsearchRepository. This will allow you to perform searches directly on your Elasticsearch index.

6) Custom Elasticsearch Operations for Articles

The ArticleSearchOperations class provides custom methods for interacting with the Elasticsearch index specifically for Article entities. It uses ElasticsearchOperations to perform indexing, retrieval, updating, deletion, and various types of search queries on the Article documents.

7) PostgreSQL Repository for Article Entity

The ArticleRepository interface provides a way to interact with the PostgreSQL database for Article entities. It extends JpaRepository, enabling CRUD operations and query execution on Article records stored in the articles table.

8) Periodical Sync between PostgreSQL and Elasticsearch

The BatchSyncService class is a Spring service responsible for periodically synchronizing Article data between PostgreSQL and Elasticsearch. It runs on a fixed schedule, fetching data from the PostgreSQL database and updating the Elasticsearch index to ensure both are in sync.

9) REST Controller for Managing Articles in PostgreSQL

The ArticlePostgresRest class is a Spring Boot REST controller that provides CRUD operations for the Article entity, allowing interaction with PostgreSQL through HTTP requests. It exposes endpoints for creating, reading, updating, and deleting articles.

10) REST Controller for Elasticsearch Operations on Articles

The ArticleSearchOperationsRest class is a Spring Boot REST controller that exposes APIs for interacting with articles stored in Elasticsearch. It provides endpoints for indexing, retrieving, updating, and deleting articles, as well as performing full-text search queries.

11) REST Controller for Elasticsearch Operations Using Repository

The ArticleSearchRepositoryRest class is a Spring Boot REST controller that exposes APIs for managing articles in Elasticsearch using a repository-based approach. This class performs operations such as indexing, updating, retrieving, deleting articles, and performing search queries.

12) Application Entry Point

The Application class serves as the entry point for the Spring Boot application.

13) Test the Application

Navigate to http://localhost:8080/swagger-ui.html (or the configured URL for Swagger UI). You should now see an interactive interface to test the endpoints for the Article APIs.

14) More Info

In Spring Data Elasticsearch, there are several ways to interact with an Elasticsearch cluster:

1) ElasticsearchRepository: A high-level interface based on Spring Data repositories, which provides CRUD operations and query methods for Elasticsearch.

2) ElasticsearchOperations: A more generic interface for interacting with Elasticsearch. It offers a lower-level API than ElasticsearchRepository and provides flexibility to define custom methods, work with indices, mappings, and perform custom queries.

3) ElasticsearchRestTemplate (formerly ElasticsearchTemplate): A concrete implementation of ElasticsearchOperations using the REST client (since Spring Data Elasticsearch 4.0+). ElasticsearchRestTemplate is often used for more complex operations, such as bulk indexing, custom queries, and handling more advanced Elasticsearch configurations.

Source Code: GitHub

Index
How to Implement PostgreSQL Full-Text Search with Spring Boot

15 min

Spring's transaction management with the @Transactional annotation

9 min

Spring Boot Rest APIs with PostgreSQL (Spring Boot + Rest APIs)

15 min

Caching in Spring Boot (@Cacheable, @CacheEvict & @CachePut)

21 min

Declarative REST Client in Spring Boot (Spring 6 HTTP Interface)

13 min

Circuit Breaker in Spring Boot (Spring Cloud Circuit Breaker + Resilience4j)

12 min

Handling Concurrent Service Calls in a Spring Boot Application: Asynchronous vs. Reactive Approaches

11 min

Profiling a Spring Boot application with Pyroscope

7 min

Service discovery in Spring Boot (Spring Cloud + Netflix Eureka)

9 min

Dockerize Spring Boot app and Push image to DockerHub (Spring Boot + DockerHub)

4 min

Creating a Jenkins Pipeline for Spring Boot application

2 min

Monitoring Microservices (Spring Boot + Micrometer + Prometheus + Grafana)

7 min

Edge Server Pattern in Microservices (Spring Cloud Gateway)

7 min

Circuit Breaker Pattern in Microservices (Spring BOOT + Resilience4j)

4 min

Spring Cloud config server setup with Git

8 min

Distributed Tracing in Microservices (Spring Cloud Sleuth + Zipkin)

9 min

Circuit Breaker Pattern with Resilience4J in a Spring Boot Application

24 min

Deploying Spring Boot microservices on Kubernetes Cluster

12 min

Reactive programming in Java with Project Reactor

50 min

Spring Reactive with PostgreSQL (Spring Boot WebFlux + PostgreSQL)

13 min

Spring Reactive, Thymeleaf Hello World (Spring Webflux + Thymeleaf + JS/CSS)

9 min

Problem JSON (application/problem+json) in Spring WebFlux

15 min

Spring Boot Login/Logout (Spring Security + MySql + Thymeleaf)

21 min

Securing Server-to-Server Communication with "Spring Boot" & "OAuth 2"

18 min

Integrating Elasticsearch with a Spring Boot and PostgreSQL application

16 min

Sending Emails in Spring Boot via SMTP

7 min

How to create a basic Spring 6 project using Maven

5 min

Spring Boot, Thymeleaf Hello World (Spring Boot + Thymeleaf + JS/CSS)

9 min