Spring Cloud Config Series Part 3: Zookeeper Backend

In this part of the series, we take a look at how we can use Spring Cloud Config with Zookeeper to manage our configuration.

Introduction

I’ve always found it hard to define what Zookeeper is. This is the definition given in its website

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

In short, the way I think about Zookeeper, is basically a system that stores information and makes sure it’s consistent and synchronized across distributed systems. What kind of information you store in it, is up to you. You could, for example, store some information which will help with Leadership Election or implement Distributed Transactions. In our case, we are going to use it to store our application configuration.

Using Zookeeper as the backend

Zookeeper stores its data in something called ZNodes. You can think of these ZNodes as a filesystem. The way a ZNode is identified is very similar to files as well, so for example, you can have a ZNode identified by /config/ZookeeperSampleApplication/server.port which can contain a value like 8081.

The / separates ZNodes, and these ZNodes can have values and children ZNodes at the same time, so /config can also have a value as well as be the parent of the ZookeeperSampleApplication ZNode.

In order to set the data into Zookeeper, you’ll need to use the Zookeeper client, or try finding a User Interface available. In our example, we will use Exhibitor, which was created by Netflix and then released as Open Source.

Structure of the ZNodes in Spring Cloud Config

The same way that your runtime properties were structured in property files in the Git backend example, with Zookeeper, this configuration will be structured in ZNodes. Let’s take a look at how they are divided.

Assuming you have an application named ZookeeperSampleApplication, and you have UAT and PROD environments, then your ZNodes containing configuration (like folders) would look something like this:

/config/ZookeeperSampleApplication,PROD/
/config/ZookeeperSampleApplication,UAT/
/config/ZookeeperSampleApplication/
/config/application,PROD/
/config/application,UAT/
/config/application/

Let’s take a closer look:

To be more specific, let us look at a more concrete example. If you wanted to define the logging level (configured in Spring Boot by setting the property logging.level.ROOT) to be different per environment for your application, as well as defining defaults for both your application (when the profile is not matched) and any other application, your ZNode tree would look like this:

Exhibitor ZNode Tree

You can see that each of the ZNodes containing properties has a property logging.level.ROOT defined, and for the selected ZNode, which configures the UAT profile for our application, the value is DEBUG (look at the “Data as String” field)

Architecture Overview

The Architecture for this setup is a bit simplified compared to the one we saw in the previous post for the Git setup.

Zookeeper Setup

Spring Cloud Config will add a ZokeeperPropertySource to your application which will use Curator to communicate with Zookeeper. Then you just need to setup the Zookeeper cluster and tell your application where it lives.

Implementation Details

In order to use this setup, assuming that your applications are already Spring Boot apps, you should do at least the following things:

  1. Your client applications will need to import the Spring Cloud Config Zookeeper Client maven dependency.
    <dependency>
       <groupId>org.springframework.cloud</groupId>
       <artifactId>spring-cloud-starter-zookeeper-config</artifactId>
    </dependency>
    
  2. Define a file named bootstrap.properties in the root of your classpath, which will need to include the following properties:
    # The name of the application
    spring.application.name: ZookeeperSampleApplication
    # The connection string to for your Zookeeper instance.
    spring.cloud.zookeeper.connectString: localhost:2181
    

    This file will tell Spring Cloud to load up the remote Zookeeper configuration on the bootstrap phase of your application startup, before anything else happens. Once the Zookeper configuration is loaded, the library will identify which ZNodes apply to the current scenario (application name and profiles), and will add a PropertySource to the environment with the highest priority to make sure that Zookeeper properties are looked up before any other sources such as property files or system properties.

You can find the example code in this Github repository and play around with it.

Detecting Configuration Changes

Zookeeper comes with built in support for receiving events when ZNodes change. Spring Cloud Config will hook into this events and will add listeners to the ZNodes it cares about. When these ZNodes are added, removed or updated, it will refresh the context.

Remember that, even though the context will be refreshed automatically, you still need to make sure that the way you read your configuration will support the refresh automatically, otherwise you may need to annotate the beans that use the configuration with @RefreshScope. For more information on how Spring Cloud manages the configuration and the refresh events, take a look at the first post of the series.

This automatic refresh mechanism can be enabled or disabled by setting the property spring.cloud.zookeeper.config.watcher.enabled which is true by default. As with any Spring Cloud Config application, as long as you have the Spring Boot Actuators enabled, you can send a POST /refresh request to your application, which will refresh the context.

Conclusion

We do have Zookeeper instances in my current project, and in the past I’ve seen how some teams in my current company would write quite complex code to integrate with it to manage their configuration, so I was always a bit hesitant about integrating with it.

Then, after Spring Cloud Config Zookeeper came out, and took a look at how easy it was to configure it, we immediately started using it. Having a library that will manage the ZNode structure for you as well as listen to events and act accordingly is a huge addition to any project that already uses Zookeeper.

Having said that, I think Zookeeper has a few drawbacks as a configuration management system:

Overall, I think that from the infrastructure point of view, Zookeeper is quite a complex solution, but on the other hand, from an application point of view, with a library like Spring Cloud Config Zookeeper, it becomes a very interesting option.

Tags