Micronaut (5 of 6): Postgres Integration

Lydtech
Microservices Event Driven Architectures Testing & Quality Ovalview Component Test Framework
Courses Case Studies Blog News Contact
Micronaut (5 of 6): Postgres Integration

Introduction

In this section the demo Micronaut application will be enhanced to use a Postgres database as the backing store. The choice on the selection of the Postgres client and data migration libraries are discussed, given their impact on the ability to build a native executable. The unit, integration and component tests are updated in line with the changes.

The companion application, written both in Kotlin and in Java, have been updated with Postgres support, and are available in these repos:

The source code for the companion application, in Kotlin and Java, is available here: [Kotlin version | Java version].

This is the fifth article in a six part series on the Micronaut framework.

  1. Framework Features
  2. Demo Application
  3. Testing
  4. Native Builds
  5. Postgres Integration (this article)
  6. Kafka Integration (coming soon)

Demo Application

The demo application has been updated to replace the in-memory Map store that was used for reading and writing item entities with a Postgres database. This means that the service class is now responsible for integrating with this external resource.

Figure 1: Reading and writing to Postgres

Figure 1: Reading and writing to Postgres

Postgres Client Library

Deciding on which client library to use in order for the application (the client) to call the database is a crucial one. As with any library selection a number of factors should always be carefully considered. These include the maturity and battle hardiness of the library, how well supported it is, how well documented, how quickly bugs are addressed, and the size of the community.

With these factors in mind, an obvious choice would be the micronaut-data-hibernate-jpa library. This library integrates the popular Hibernate JPA for ORM capabilities, allowing developers to use familiar JPA annotations and APIs with Micronaut Data. It ticks all the boxes on the list above. However there is a significant potential downside, and that is that Hibernate makes extensive use of reflection, dynamic class loading, and other features that are not natively supported. This means building native executables, if that is a requirement, becomes much more complex. It can be done, one step being to use a tool like native-image-agent to generate the necessary configuration files by analysing the application during runtime. But this is time consuming and error prone and can lead to unexpected errors at runtime (see the fourth article in the series, Micronaut: Native Builds on building native images).

It may be that there is no need to build a native executable, in which case this library would be a good choice. However for this demo the intention is to build and run a native executable. Another excellent choice that is compatible with GraalVM native images, and ticks the previous boxes, is micronaut-data-jdbc. This provides a high level abstraction for JDBC operations, supporting the repository pattern and automatic SQL generation.

Database Schema Migration Library

A similar decision is required on the database schema migration library to use. This library is used for creating and evolving the database schema and writing any necessary data. While this can be used in Production, these tasks are often done outside of the application by another process. However, in earlier environments such as Dev and QA, and very commonly in tests, data migration is usually an important consideration.

As with the Postgres client library there are multiple options, and a popular choice here would be to use micronaut-flyway, known for its ease of version controlling and applying schema changes and populating tables. However it has limited support for GraalVM native images. An alternative library is Liquibase, which is also a popular choice offering similar functionality to micronaut-flyway, but importantly is compatible with GraalVM native images. This then is the selected library for this demo.

Application Changes

Service

The design of the application, using the single responsibility principle, means that the actual changes required to swap in the Postgres database in place of the local Map are minimal, as the reads and writes are encapsulated in the service class. The ItemService [Kotlin | Java] uses a repository interface as its interface to the database, and an entity class to encapsulate the fields for the entity instance. The service is updated to inject the ItemRepository class:

@Singleton
class ItemService(private val itemRepository: ItemRepository)

The createItem method then uses the repository to persist the item:

fun createItem(request: CreateItemRequest): UUID {
   var item = Item(request.name)
   item = itemRepository.save(item)
   log.info("Item created with id: {}", item.id)
   return item.id as UUID
}

Repository

The ItemRepository interface [Kotlin | Java] is defined:

@JdbcRepository(dialect = Dialect.POSTGRES)
interface ItemRepository : CrudRepository<Item, UUID>

The interface is annotated with the micronaut-data-jdbc library @JdbcRepository annotation to specify that the repository is a JDBC repository and to define the SQL dialect to use. Specifying the SQL dialect ensures that the SQL queries generated by Micronaut Data can be optimised, and handle database specific behaviours such as pagination and date functions.

The repository extends the Micronaut CrudRepository interface and so inherits methods that are automatically generated for basic CRUD operations. This means the developer does not have to write these themselves. The entity type is defined, in this case the Item type, along with the primary key type, here UUID.

Entity

The entity class is a plain old Java object (POJO) that represents a table in the database. It is mapped with the Micronaut Data annotation @MappedEntity, and it defines the fields that comprise the entity. In the demo application the entity is Item [Kotlin | Java], and it defines the id field (a UUID), and the name:

@MappedEntity("ITEM")
data class Item(
   @Id
   @AutoPopulated
   @MappedProperty("ID")
   var id: UUID? = null,


   @MappedProperty("NAME")
   var name: String
) {
   constructor(name: String) : this(null, name)
}

The id field is marked as the primary key of the entity via the @Id annotation, and the @AutoPopulated annotation means it is automatically populated with a unique UUID when a new item instance is persisted.

Postgres Configuration

With the Map store being replaced by an external database, the application must be configured so it is able to connect to the database. The application.yml [Kotlin | Java] has the connection properties added accordingly:

datasources:
   default:
       url: jdbc:postgresql://localhost:5432/postgres
       driverClassName: org.postgresql.Driver
       username: postgres
       password: postgres
       dialect: POSTGRES

Database Schema Migration

The demo application uses Liquibase to create the schema on application startup. The configuration all lives under src/resources/ so is versioned along with the application code. The application.yml [Kotlin | Java] points to the root Liquibase configuration file to use:

liquibase:
   datasources:
       default:
           change-log: classpath:db/changelog.yml

The changelog.yml [Kotlin | Java] file lists out each data migration file to apply. In this case there is just a single migration:

databaseChangeLog:
 - include:
     file: changelog/V1.0.0-changelog.yml
     relativeToChangelogFile: true

The V1.0.0-changelog.yml [Kotlin | Java] defines the database schema. In this case it contains the schema definition for the ITEM table. The beginning of this configuration is shown here:

databaseChangeLog:
 - changeSet:
     id: create-table-item
     author: demo
     changes:
       - createTable:
           tableName: ITEM

Test Changes

Overview

The unit, integration, and component tests for the demo Micronaut application are covered in detail in the third article in the series, Micronaut: Testing. This section looks at what changes are required to support the addition of the Postgres database as the backend store.

Unit Tests

The ItemServiceTest unit test [Kotlin | Java] must be updated to mock the ItemRepository, as the scope of this test is to prove the service functionality. The repository is mocked in order to specify its behaviour when the service class uses it, to prove that the service itself behaves as expected.

The repository is mocked using the mockk mocking library, and is passed into the service constructor:

private lateinit var service: ItemService
val itemRepositoryMock = mockk<ItemRepository>()

@BeforeEach
fun setUp() {
   clearAllMocks()
   service = ItemService(itemRepositoryMock)
}

The updated testCreateItem() is shown here, with the repository mocked to return the saved Item entity. The test verifies that the mock was called exactly once:

@Test
fun testCreateItem() {
   val itemId = randomUUID()
   every { itemRepositoryMock.save(any(Item::class)) } returns TestDomainData.buildItem(itemId, randomAlphabetic(8))

   val request = TestRestData.buildCreateItemRequest(randomAlphabetic(8))
   val newItemId = service.createItem(request)


   assertEquals(itemId, newItemId)
   verify(exactly = 1) { itemRepositoryMock.save(any(Item::class)) }
}

Integration Tests

Integration tests typically treat the application as a black box, for example sending in requests to the application’s REST API and verifying the responses. This is the case with the example EndToEndIntegrationTest [Kotlin | Java] where the operations to create, retrieve, update and delete are tested by calling the REST API. There are therefore no changes required to the test code, as this does not interact with the datastore directly.

What is required is to add the configuration for a test database. As this is an integration test it is responsible for verifying that the application can successfully integrate with its dependent resources. To facilitate this, an in-memory database is configured. This is much faster than starting up a disk-based database, and is much simpler to setup and teardown. It also ensures each test runs in isolation without being affected by the state of a real database, and provides a consistent environment for the tests to run.

H2 is a lightweight, open-source relational database written in Java and is an excellent choice for use with integration tests. To configure it for the tests the application-test.yml [Kotlin | Java], which is used by the integration tests, is updated with the test database connection details:

datasources:
   default:
       url: jdbc:h2:mem:testdb;MODE=PostgreSQL;DB_CLOSE_ON_EXIT=FALSE
       driverClassName: org.h2.Driver
       username: sa
       password: password
       dialect: H2

The database mode is set to PostgreSQL to configure H2 to emulate the PostgreSQL dialect. This makes H2 more compatible with PostgreSQL, supporting its syntax, functions and behaviour. The dialect is set to H2 so that Micronaut Data generates SQL queries that are compatible with H2 to ensure they are correctly interpreted by the H2 database.

The integration test is annotated with @MicronautTest, which results in the Micronaut application context being started at the outset of the test, which includes configuring and starting the in-memory H2 database. When the application is called by the tests to perform CRUD operations on the item table, it now integrates with the database, and the tests verify this integration.

As before, the tests are run via gradle with the command:

./gradlew clean test

Component Tests

As with the integration tests, the component tests treat the system as a black box, again using the REST API to exercise the end to end application flows. There are therefore no code changes required to these tests either, but in this case the application will be using a real Postgres database instance to integrate with. Component tests therefore prove that a running instance of the application successfully integrates with a real database instance.

The Component Test Framework is configured to spin up a Postgres database in a lightweight Docker container using Testcontainers, along with the Docker container that contains the application under test.

Figure 2: Component testing a Micronaut application with Postgres

Figure 2: Component testing a Micronaut application with Postgres

To achieve this the gradle.properties [Kotlin | Java] is updated to specify that the Postgres database is required, along with its necessary configuration:

systemProp.postgres.enabled=true
systemProp.postgres.host.name=postgres
systemProp.postgres.port=5432
systemProp.postgres.database.name=test
systemProp.postgres.schema.name=public
systemProp.postgres.username=user
systemProp.postgres.password=password
systemProp.postgres.container.logging.enabled=false

As the EndToEndCT component test [Kotlin | Java] hooks into the Component Test Framework via the @ExtendWith(ComponentTestExtension::class) annotation (see more on this in the third article in the series, Micronaut: Testing), that is all that is required for the Docker container to be started.

The final configuration change is to tell the application under test, running in its own Docker container, how to connect to this instance. An application-component-test.yml [Kotlin | Java] properties file is provided in the src/main/test/ directory with this configuration:

datasources:
 default:
   url: jdbc:postgresql://postgres:5432/test
   username: user
   password: password

The application and Docker image are first built, either as a standard application jar:

./gradlew clean build
docker build -t ct/micronaut-postgres-kotlin:latest .

Or as a native application jar:

./gradlew clean nativeCompile
./gradlew dockerBuildNative

And the component tests are then run via the command:

./gradlew componentTest --rerun-tasks

Running The Application

With the suite of unit, integration, and component tests passing there is high confidence that the application will run successfully - indeed it already has run and integrated with a real Postgres instance using the component tests. To run the application the following steps can be taken. First the Postgres Docker image is started, and a docker-compose.yml [Kotlin | Java] file is provided that defines this container:

services:
   postgres:
       container_name: postgres
       image: postgres:14.1-alpine
       restart: always
       environment:
           - POSTGRES_USER=postgres
           - POSTGRES_PASSWORD=postgres
       ports:
           - '5432:5432'
       volumes:
           - postgres:/var/lib/postgresql/data

volumes:
   postgres: {}

This container is started with:

docker-compose up -d

The Micronaut application is then built and run, as previously described in the second article in the series, Micronaut: Demo Application, for the standard application, and in the fourth article in the series, Micronaut: Native Builds, for the native build of the application. For example, to run the native build:

./gradlew nativeCompile
./gradlew nativeRun

The REST endpoints can now be hit using curl, for example to create a new Item in the Postgres database (which returns the location header in the REST response), and then retrieve it:

curl -i -X POST localhost:9001/v1/items -H "Content-Type: application/json" -d '{"name": "test-item"}'
curl -i -X GET localhost:9001/v1/items/653d06f08faa89580090466e

Summary

The Micronaut framework provides robust support for integrating an application with a Postgres database. Leveraging popular client libraries, developers can apply annotations directly to the code to achieve this integration. This approach both simplifies the development and eliminates the need for writing boilerplate code. When selecting the client library one important factor to consider is whether the application needs to support native image generation. In such cases, selecting a library that avoids the use of reflection is crucial to ensure compatibility.

Source Code

There are two flavours of the Micronaut application, one in Kotlin and one in Java, and the source code is available here:

Kotlin version: https://github.com/lydtechconsulting/micronaut-postgres-kotlin/tree/v1.0.0
Java version: https://github.com/lydtechconsulting/micronaut-postgres-java/tree/v1.0.0

Next... Kafka Integration

In the final article in the series (coming soon) the example Micronaut applications will be enhanced to integrate with a Kafka message broker.

View this article on our Medium Publication.

Related articles