Blog

Lydtech
Microservices Event Driven Architectures Testing & Quality Ovalview Component Test Framework
Courses Case Studies Blog News Contact
                             
Micronaut (1 of 6): Framework Features

In-depth guide to using Micronaut - part one uncovers the main framework features
Application Logging & Alerting With Graylog (1 of 2): Introduction

An overview of application logging and alerting with Graylog.
KafkaJS: Integration Testing With Testcontainers

Demonstrating integration testing a KafkaJS Typescript application using Testcontainers and Docker.
Monitoring & Alerting: Prometheus, Grafana & Alertmanager - Part 1: Introduction

Setting up a monitoring and alerting stack consisting of Prometheus, Grafana and Alertmanager.
Kafka Streams Windowing - Overview

An Overview of the windowing types for Kafka Streams
Simple! We’ll just dead-letter it

Exploring the implications of using Dead Letter Topics (DLT) in the context of Apache Kafka for failed messages.
Kafka on Kubernetes - Part 1: Introduction to Kubernetes

Demonstrating Kafka on Kubernetes (Part 1 of 3): providing an introduction to Kubernetes, minikube and kubectl.
Authentication and Authorisation using OIDC and OAuth 2. Part 1: A Primer

An introduction into the world of Authentication and Authorisation
Size Matters

Why thinking small throughout the software development lifecycle can result in big returns.
Schemaless Database Considerations

Schemaless database considerations and the impact on application code.
Build Only What You Need - Part 1 - Development

We review the reasons we try to add functionality beyond what is required, and the costs of doing this.
Kafka Poison Pill

What is a poison pill in the context of Apache Kafka? What are the effects and how can they be prevented?
Automated Testing: Don't Put It Off

The costs of delaying automated testing.
Kafka Consumer Auto Offset Reset

Configuring from where in a topic partition a new consumer should first consume messages.
The Hidden Cost of AWS Lambdas

Some observations on running PoC & early stage applications in AWS.
Should You Draw It? (Yes You Should!)

Good development starts before a single line of code is written.
Integrating Flink with Kafka

Demonstrating connecting a Flink application to Kafka to consume, process, and produce Kafka events.
Kafka Connect: CDC with Debezium and MongoDB

Demonstrating Change Data Capture using Kafka Connect with Debezium and MongoDB.
Kafka Consumer Group Rebalance - The Next-Gen Protocol With Kafka 4.0

The changes and improvements of the next generation consumer group rebalance protocol with Kafka 4.0.
KafkaJS: Integration Testing With Testcontainers

Demonstrating integration testing a KafkaJS Typescript application using Testcontainers and Docker.
Kafka Streams Windowing - Spin it up with Kubernetes

Deploying the kafka streams windowing apps in Kubernetes
Kafka Streams Windowing - Tumbling Windows

A closer look at Tumbling Windows
Kafka Streams Windowing - Sliding Windows

A closer look at Sliding Windows
Kafka Streams Windowing - Overview

An Overview of the windowing types for Kafka Streams
Kafka Streams Windowing - Hopping Windows

A closer look at Hopping Windows
Kafka SASL Authentication: Spring Boot Demo

Demonstrating how to configure and use the SASL authentication mechanism to connect to Kafka.
Kafka Message Compression (2 of 2): Spring Boot Demo

Demonstrating sending compressed messages to Kafka with Spring Boot.
Kafka Message Compression (1 of 2)

The impact and effectiveness of message compression when sending batches to Kafka.
Kafka Chaos Testing With Conduktor Gateway

Automating Kafka chaos testing using Conduktor Gateway.
Simple! We’ll just dead-letter it

Exploring the implications of using Dead Letter Topics (DLT) in the context of Apache Kafka for failed messages.
Kafka on Kubernetes - Part 3: Spring Boot Demo

Demonstrating Kafka on Kubernetes (Part 3 of 3): deploying a Spring Boot application to Kubernetes that consumes and produces events from Kafka.
Kafka on Kubernetes - Part 2: Deploying Kafka

Demonstrating Kafka on Kubernetes (Part 2 of 3): stepping through the deployment of Kafka and Zookeeper to Kubernetes.
Kafka on Kubernetes - Part 1: Introduction to Kubernetes

Demonstrating Kafka on Kubernetes (Part 1 of 3): providing an introduction to Kubernetes, minikube and kubectl.
Kafka Keys, Partitions and Message Ordering

Uncovering how Kafka guarantees message ordering based on the relationship between message keys and topic partitions.
Kafka JSON Serialization

Demonstrating serialization and deserialization of JSON formatted Kafka messages using Spring Boot with Spring Kafka.
Kafka Connect: Transactional Outbox With Debezium: Spring Boot Demo

Demonstrating the Transactional Outbox pattern using Kafka Connect with Debezium for Change Data Capture.
Kafka Consumer Non-Blocking Retry Pattern

Demonstrating a generic non-blocking consumer retry pattern.
Kafka Consumer Non-Blocking Retry: Spring Retry Topics

Using Spring Kafka retry topics for non-blocking consumer retry.
Kafka Monitoring & Metrics: With Confluent Control Center

How to monitor a Kafka cluster using Confluent Control Center to provide a view on the key broker, topic, consumer and producer metrics.
Kafka Monitoring & Metrics: With Conduktor

How to monitor a Kafka cluster using Conduktor to provide a view on the key broker and topic metrics.
Kafka Replication & Min In-Sync Replicas

Looking at how Kafka replication provides high availability and fault tolerance, and the impact of min-sync replicas.
Kafka Schema Registry & Avro: Component Testing

Component testing an application with a Dockerised Kafka and Schema Registry, using Testcontainers and the Component Test Framework.
Kafka Schema Registry & Avro: Integration Testing

Integration testing the application using Spring Boot test, with the embedded Kafka broker and a wiremocked Schema Registry.
Kafka Poison Pill

What is a poison pill in the context of Apache Kafka? What are the effects and how can they be prevented?
Conduktor Platform & Kafka Component Testing

Using the Conduktor Platform as an aid to component testing a Kafka based application.
Kafka Schema Registry & Avro: Spring Boot Demo (2 of 2)

A deep dive into the Spring Boot application demo code that integrates with Kafka Schema Registry and uses Avro Serialization.
Kafka Schema Registry & Avro: Spring Boot Demo (1 of 2)

Showcasing a Spring Boot application integrating with Kafka Schema Registry and using Avro Serialization.
Kafka Schema Registry & Avro: Introduction

An introduction to using the Schema Registry with Kafka, and Apache Avro for message serialization.
Kafka Message Batch Consumer Retry

Uncovering the consumer retry behaviour when failures occur in the middle of processing a batch of messages.
Kafka Consume & Produce: At-Least-Once Delivery

Understanding the meaning and ramifications of the at-least-once delivery guarantees provided by Kafka.
Kafka Streams: Transactions & Exactly-Once Messaging

Achieving exactly-once messaging in a Kafka Streams application by utilising Kafka Transactions.
Kafka Topics: Fat Pipe Or Thin Pipe?

Investigating whether a topic should be used for more than one event type.
Kafka Consume & Produce: Testing

Comprehensive guide to unit, integration, and component testing a Kafka Spring Boot application.
Kafka Consume & Produce: Spring Boot Demo

Demonstrating Kafka message consume and produce with a Spring Boot application.
Kafka Streams: Testing

The tools available to comprehensively test a Kafka Streams application - before it reaches QA.
Kafka Streams: State Store

Understanding the role and usage of the state store in a stateful Kafka Streams application.
Kafka Streams: Spring Boot Demo

Showcasing Kafka Streams processing with a Spring Boot application.
Securing Kafka with Mutual TLS and ACLs

Securing a Kafka installation using a combination of mutual TLS and ACLs.
Kafka Streams: Introduction

A look at the usage and characteristics of the Kafka Streams API.
Kafka Consumer Group Rebalance - Part 2 of 2

The role and impact of consumer group rebalances.
Kafka Consumer Group Rebalance - Part 1 of 2

The role and impact of consumer group rebalances.
Kafka Producer Message Batching

Increase throughput by batching messages sent to the broker.
Kafka Producer Configuration

Key factors to consider when configuring the Kafka Producer.
Kafka Consumer Auto Offset Reset

Configuring from where in a topic partition a new consumer should first consume messages.
Kafka Transactions: Part 2 - Spring Boot Demo

Spring Boot application demonstrating exactly-once messaging with Kafka Transactions.
Kafka Transactions: Part 1 - Exactly-Once Messaging

Exploring using Kafka Transcations to achieve exactly-once messaging.
Kafka Idempotent Consumer With DynamoDB

Compare and contrast DynamoDB with RDMS as the deduplication backing store.
Kafka Unclean Leader Election

Trading off availability for durability with unclean leader election.
Kafka Producer Acks

Configuring Producer message acknowledgements trading off performance for durability.
Kafka Idempotent Consumer & Transactional Outbox

Kafka resiliency with the Idempotent Consumer pattern and the Transactional Outbox pattern utilising CDC.
Kafka Idempotent Producer

Utilising the Idempotent Producer to avoid duplicate messages.
Kafka Deduplication Patterns - Part 2 of 2

Analysing the effect of applying different deduplication patterns when processing Kafka messages.
Kafka Deduplication Patterns - Part 1 of 2

Analysing the effect of applying different deduplication patterns when processing Kafka messages.
Kafka Consumer Retry

Recovering from transient errors using stateless and stateful retry.
Kafka Client: Apache Kafka vs KafkaJS

Understanding the role and importance of the Kafka client library.
Micronaut (3 of 6): Testing

In-depth guide to using Micronaut - part three demonstrates unit, integration and component testing the example Micronaut application
Micronaut (2 of 6): Demo Application

In-depth guide to using Micronaut - part two steps through an example Micronaut application written in Kotlin and Java
Micronaut (1 of 6): Framework Features

In-depth guide to using Micronaut - part one uncovers the main framework features
Lydtech Reference Projects

Lydtech's collection of reference projects available on our Github.
Three Key Principles For A Successful IT Project

Keep it simple, draw it out, test it early: three key principles that will contribute towards the success of an IT project.
Schemaless Database Considerations

Schemaless database considerations and the impact on application code.
Build Only What You Need - Part 2 - Product Features

From a Product Team perspective we examine why teams deliver features that do not provide client value, and the costs incurred.
Build Only What You Need - Part 1 - Development

We review the reasons we try to add functionality beyond what is required, and the costs of doing this.
The Hidden Cost of AWS Lambdas

Some observations on running PoC & early stage applications in AWS.
Flyway in Serverless Architectures

How to use Flyway in your serverless architectures.
Failover and Circuit Breaker with Resilience4j

Using the Resilience4j library for failover and circuit breaker.
Exploring JHipster

Building full-stack apps using JHipster.
Microbenchmarking with JMH

Using the JMH Microbenchmarking library for performance testing your code.