Home>Articles>Building Spring Boot application for Apache Kafka Admin operations
kafka logo programing developer
Articles

Building Spring Boot application for Apache Kafka Admin operations

1. About this article

Apache Kafka is a distributed streaming platform which is for building real-time streaming data pipelines and applications. Managing Kafka cluster is not an easy task and sometimes confusing. In this article, I will show you how you can implement an application which is capable to do Admin operations in Kafka.

2. Technology

In this example we use:

  • Spring Tool Suite 3
  • Spring Boot Framework
  • Zookeeper (version 3.5.6)
  • Apache Kafka (2.11 – 2.3.0)
  • Java 8

In the next part, I will introduce in details how to implement our solution.

3. Step by step implementation

3A. Running Kafka and ZooKeeper

1. You can change the default Zookeeper port in zoo.cfg file (Default port 2181).

2. Open a new cmd, and type zkserver. If the command is not recognized, you should double check the System Environment Variables setup or navigate in cmd to the zookeeper-3.5.6/bin directory and type zkserver there.

 

Congratulations, your ZooKeeper is up and running on port 2181

Thumbs up green emoji

3. Open cmd and navigate to the Kafka home directory.

4. Now type .\bin\windows\kafka-server-start.bat .\config\server.properties and press Enter.

NOTE: If you use any other port than 2181 to your ZooKeeper than go the Kafka_HOME/config directory look for the entry “zookeeper.connect=localhost:2181” and change it to the desired port.

 

3B. Creating Project

Create a new Maven project in your IDE.

Have the following dependencies added to your pom.xml:

 

</dependency>

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-web</artifactId>

</dependency>

<dependency>

<groupId>io.swagger.core.v3</groupId>

<artifactId>swagger-annotations</artifactId>

<version>2.0.10</version>

</dependency>

<dependency>

<groupId>io.springfox</groupId>

<artifactId>springfox-swagger2</artifactId>

<version>2.9.2</version>

</dependency>

<dependency>

<groupId>io.springfox</groupId>

<artifactId>springfox-swagger-ui</artifactId>

<version>2.9.2</version>

</dependency>

<dependency>

<groupId>org.springframework.kafka</groupId>

<artifactId>spring-kafka</artifactId>

<version>2.3.1.RELEASE</version>

</dependency>

<dependency>

<groupId>org.apache.kafka</groupId>

<artifactId>kafka_2.12</artifactId>

<version>2.3.0</version>

</dependency>

 

Project Structure should look like this:

 

In application.properties include “server.port=<port_number>” with the desired port number.

First, we have to make the foundation of our application with the App.java:

 

package com.technaura;


import
org.springframework.boot.SpringApplication;

import org.springframework.boot.autoconfigure.SpringBootApplication;

import springfox.documentation.swagger2.annotations.EnableSwagger2;

 

@SpringBootApplication

@EnableSwagger2

public class App {

              public static void main(String[] args) {

SpringApplication.run(App.class, args);

}

}

 

3C. The Service

As the next step, we implement the service methods for the Kafka operations. These methods will directly invoke the AdminZkClient methods.

Have the following imports for KafkaService:

 

import java.util.Properties;

import org.apache.kafka.common.utils.Time;

import org.springframework.stereotype.Service;

import kafka.admin.RackAwareMode;

import kafka.zk.AdminZkClient;

import kafka.zk.KafkaZkClient;

import scala.Option;

 

The key class of this solution is AdminZkClient class which contains multiple methods to operate with Kafka. In this article I will show how to create topic.

First, you should have the annotation for your class:

 

 

 

Now we have our client. If we wan’t to do operations in kafka, we simply just invoke the AdminZkClient’s methods.

 

For creating topic we introduce the following method:

 


 

3D. The Controller

As next step we implement the controller methods to invoke the service methods in these endpoints.

Have the following imports for KafkaController:

 

Then you should annotate your controller class:

 

4. Let’s start it up

So now that we finished coding, it’s time to try out our work.

Run your application from IDE as Spring Boot Application or build it with Maven into a jar file than run it.

 

When it is done, open your browser and visit:

http://localhost:7775/swagger-ui.html

 

You will see this:

 

Open down kafka-controller and then the endpoint. Click on “Try it out” give the topic name in the appearing text box then click Execute.

 

Now if you visit your Kafka log directory, a new folder appeared with the name of the newly created topic with a partition number which should be 0 by default.

 

Congratulations, you successfully implemented a Kafka controlling operation!

Thumbs up green emoji

Written by Miklós Takács, Software Developer at Technaura Systems GmbH.

 

For further articles, please read further:

Kafka Streams

Kafka SSL Client Authentication in Multi-Tenancy Architecture

Deploying Spring Boot Applications

Building web applications with Spring Boot and Kotlin

Spring Boot Tutorial – Bootstrap a Simple Application