Single Message Transformations - The Swiss Army Knife of Kafka Connect
Do you remember Angus "Mac" MacGyver? The always creative protagonist of the popular 80ies/90ies TV show, who could solve about any problem with nothing more than a Swiss Army knife, duct tape, shoe strings and a paper clip?
The single message transformations (SMTs) of Kafka Connect are almost as versatile as MacGyver’s Swiss Army knife:
How to change the timezone or format of date/time message fields?
How to change the topic a specific message gets sent to?
How to filter out specific records?
SMTs can be the answer to these and many other questions that come up in the context of Kafka Connect. Applied to source or sink connectors, SMTs allow to modify Kafka records before they are sent to Kafka, or after they are consumed from a topic, respectively.
The Emitter Parameter Pattern for Flexible SPI Contracts
For libraries and frameworks it’s a common requirement to make specific aspects customizeable via service provider interfaces (SPIs): contracts to be implemented by the application developer, which then are invoked by framework code, adding new or replacing existing functionality.
Often times, the method implementations of such an SPI need to return value(s) to the framework. An alternative to return values are "emitter parameters": passed by the framework to the SPI method, they offer an API for receiving value(s) via method calls. Certainly not revolutionary or even a new idea, I find myself using emitter parameters more and more in libraries and frameworks I work on. Hence I’d like to discuss some advantages I perceive about the emitter parameter pattern.
Plug-in Architectures With Layrry and the Java Module System
Making applications extensible with some form of plug-ins is a very common pattern in software design: based on well-defined APIs provided by the application core, plug-ins can customize an application’s behavior and provide new functionality. Examples include desktop applications like IDEs or web browsers, build tools such as Apache Maven or Gradle, as well as server-side applications such as Apache Kafka Connect, a runtime for Kafka connectors plug-ins.
In this post I’m going to explore how the Java Platform Module System's notion of module layers can be leveraged for implementing plug-in architectures on the JVM. We’ll also discuss how Layrry, a launcher and runtime for layered Java applications, can help with this task.
Introducing Layrry: A Launcher and API for Modularized Java Applications
One of the biggest changes in recent Java versions has been the introduction of the module system in Java 9. It allows to organize Java applications and their dependencies in strongly encapsulated modules, utilizing explicit and well-defined module APIs and relationships.
In this post I’m going to introduce the Layrry open-source project, a launcher and Java API for executing modularized Java applications. Layrry helps Java developers to assemble modularized applications from dependencies using their Maven coordinates and execute them using module layers. Layers go beyond the capabilities of the "flat" module path specified via the --module-path parameter of the java command, e.g. allowing to use multiple versions of one module within one and the same application.
Reworking Git Branches with git filter-branch
Within Debezium, the project I’m working on at Red Hat, we recently encountered an "interesting" situation where we had to resolve a rather difficult merge conflict. As others where interested in how we addressed the issue, and also for our own future reference, I’m going to give a quick run down of the problem we encountered and how we solved it.
Monitoring REST APIs with Custom JDK Flight Recorder Events
The JDK Flight Recorder (JFR) is an invaluable tool for gaining deep insights into the performance characteristics of Java applications. Open-sourced in JDK 11, JFR provides a low-overhead framework for collecting events from Java applications, the JVM and the operating system.
In this blog post we’re going to explore how custom, application-specific JFR events can be used to monitor a REST API, allowing to track request counts, identify long-running requests and more. We’ll also discuss how the JFR Event Streaming API new in Java 14 can be used to export live events, making them available for monitoring and alerting via tools such as Prometheus and Grafana.
Enforcing Java Record Invariants With Bean Validation
Record types are one of the most awaited features in Java 14; they promise to "provide a compact syntax for declaring classes which are transparent holders for shallowly immutable data". One example where records should be beneficial are data transfer objects (DTOs), as e.g. found in the remoting layer of enterprise applications. Typically, certain rules should be applied to the attributes of such DTO, e.g. in terms of allowed values. The goal of this blog post is to explore how such invariants can be enforced on record types, using annotation-based constraints as provided by the Bean Validation API.
Using Java 13 Text Blocks (Only) for Your Tests
When Java 9 was introduced in 2017, it was the last major version published under the old release scheme. Since then, a six month release cadence has been adopted. This means developers don’t have to wait years for new APIs and language features, but they can get their hands onto the latest additions twice a year. In this post I’d like to describe how you can try out new language features such as Java 13 text blocks in the test code of your project, while keeping your main code still compatible with older Java versions.
Quarkus Qute – A Test Ride
One of the long-awaited features in Quarkus was support for server-side templating: until recently, Quarkus supported only client-side web frameworks which obtain there data by calling a REST API on the backend. This has changed with Quarkus 1.1: it comes with a brand-new template engine named Qute, which allows to build web applications using server-side templates.
Automatically Deploying a Hugo Website via GitHub Actions
As a software engineer, I like to automate tedious tasks as much as possible. The deployment of this website is no exception: it is built using the Hugo static site generator and hosted on GitHub Pages; so wouldn’t it be nice if the rendered website would automatically be published whenever an update is pushed to its source code repository?