Integrating Elasticsearch with a Spring Boot and PostgreSQL application

In this article, we will integrate Elasticsearch into a Spring Boot application to enable full-text search and automatic data updates based on PostgreSQL data. Specifically, we will cover the following:

1) How to integrate Elasticsearch into a Spring Boot application.

2) How to automatically add, update, and delete data in the Elasticsearch index based on operations on PostgreSQL data.

3) How to interact with Elasticsearch to perform CRUD operations and full-text searches using Spring Data Elasticsearch.

1) Add Dependencies

Add the required dependencies for Elasticsearch in your pom.xml.

2) Configure Elasticsearch

Ensure Elasticsearch is running on your local machine or adjust the URL if it’s hosted elsewhere.

3) Create an Elasticsearch/JPA Entity

This entity represents a row in PostgreSQL and a document in Elasticsearch. It will be indexed in Elasticsearch to enable efficient search queries.

This class represents an Article entity, which functions as both a row in a PostgreSQL database and a document in an Elasticsearch index. It is designed to be indexed in Elasticsearch, enabling efficient search capabilities. The class also uses an EntityListener to handle specific lifecycle events.

@Document(indexName = "articles"): This annotation indicates that Article is also an Elasticsearch document, which will be stored in an Elasticsearch index named articles.

@EntityListeners(ArticleEntityListener.class): This annotation registers an entity listener (ArticleEntityListener) to handle JPA entity lifecycle events such as persist or update.

4) Synchronizing PostgreSQL and Elasticsearch

The ArticleEntityListener class is an entity listener that listens to specific lifecycle events (persist, update, and remove) on the Article entity.

Its purpose is to synchronize changes between the PostgreSQL database and the Elasticsearch index by updating the Elasticsearch index whenever there is a change in the PostgreSQL data.

5) Create an Elasticsearch Repository

Define a repository for Elasticsearch using ElasticsearchRepository. This will allow you to perform searches directly on your Elasticsearch index.

6) Custom Elasticsearch Operations for Articles

The ArticleSearchOperations class provides custom methods for interacting with the Elasticsearch index specifically for Article entities. It uses ElasticsearchOperations to perform indexing, retrieval, updating, deletion, and various types of search queries on the Article documents.

7) PostgreSQL Repository for Article Entity

The ArticleRepository interface provides a way to interact with the PostgreSQL database for Article entities. It extends JpaRepository, enabling CRUD operations and query execution on Article records stored in the articles table.

8) Periodical Sync between PostgreSQL and Elasticsearch

The BatchSyncService class is a Spring service responsible for periodically synchronizing Article data between PostgreSQL and Elasticsearch. It runs on a fixed schedule, fetching data from the PostgreSQL database and updating the Elasticsearch index to ensure both are in sync.

9) REST Controller for Managing Articles in PostgreSQL

The ArticlePostgresRest class is a Spring Boot REST controller that provides CRUD operations for the Article entity, allowing interaction with PostgreSQL through HTTP requests. It exposes endpoints for creating, reading, updating, and deleting articles.

10) REST Controller for Elasticsearch Operations on Articles

The ArticleSearchOperationsRest class is a Spring Boot REST controller that exposes APIs for interacting with articles stored in Elasticsearch. It provides endpoints for indexing, retrieving, updating, and deleting articles, as well as performing full-text search queries.

11) REST Controller for Elasticsearch Operations Using Repository

The ArticleSearchRepositoryRest class is a Spring Boot REST controller that exposes APIs for managing articles in Elasticsearch using a repository-based approach. This class performs operations such as indexing, updating, retrieving, deleting articles, and performing search queries.

12) Application Entry Point

The Application class serves as the entry point for the Spring Boot application.

13) Test the Application

Navigate to http://localhost:8080/swagger-ui.html (or the configured URL for Swagger UI). You should now see an interactive interface to test the endpoints for the Article APIs.

14) More Info

In Spring Data Elasticsearch, there are several ways to interact with an Elasticsearch cluster:

1) ElasticsearchRepository: A high-level interface based on Spring Data repositories, which provides CRUD operations and query methods for Elasticsearch.

2) ElasticsearchOperations: A more generic interface for interacting with Elasticsearch. It offers a lower-level API than ElasticsearchRepository and provides flexibility to define custom methods, work with indices, mappings, and perform custom queries.

3) ElasticsearchRestTemplate (formerly ElasticsearchTemplate): A concrete implementation of ElasticsearchOperations using the REST client (since Spring Data Elasticsearch 4.0+). ElasticsearchRestTemplate is often used for more complex operations, such as bulk indexing, custom queries, and handling more advanced Elasticsearch configurations.

Source Code: GitHub