Building Microservices in OSGi with the Apache Karaf Framework

The microservice paradigm has been a topic of heavy discussion and interest in recent years. To make the matter even more involved developers and architects have different understanding on the core principles of microservices and the need to adopt a microservice-based architecture.

Taking the decision to build a new system or modernize an existing on as a set of microservices brings further the question of choosing the right framework for the purpose. In the Java ecosystem these could be the Netflix OSS, Spring Cloud, Eclipse Microprofile, Oracle’s Helidon framework, Wildfly Swarm or even Vert.x framework to name a few.

Since the Spring and JavaEE frameworks provide mechanisms for adopting a microservice-based architecture, the very fundamental building blocks of creating one with OSGi are already well established by the core and compendium specifications.

Even though many might argue that OSGi is an alternative to microservices, considering that it splits a typical Java application into a set of modules, the idea of using OSGi in combination with microservices is in fact not new and has been already successfully applied.

We are bundled with the task of architect a complex system. So complex that it is even required that the separate microservices be split into separate self-container modules internally, enabling each one to evolve over time without having to bring down the entire microservice. Such a system would be represented at a very high level schematically by the following diagram:

Examples of such complex systems that might be implemented in that manner are:

• an IoT system (where each device integration is an OSGi microservice split into modules that implement the different functions of the device)
• complex supply chain (i.e. flight cargo supply chain system)
• a complex factory process (i.e. for an automotive or aeroplane factory)

In this article we will demonstrate how we can get the best of both worlds and build a microservice-based system on top of the Apache Karaf framework (using default Apache Felix OSGi runtime through Karaf) that addresses this latter scenario.

OSGi 101

OSGi (Open Services Gateway initiative) provides a set of specifications for building module systems in Java. There are several notorious frameworks implementing the OSGi framework such as Eclipse Equinox and Apache Felix. It is introduced as JSR 8 and JSR 291 to the Java platform.

An OSGi framework/runtime makes use of the Java class loading mechanism in order to implement a container for modular units (bundles). Many application servers are implemented using OSGi as a basis.

An OSGi bundle is a just a JAR file that contains source code, bundle metadata and resources. A bundle may provide various services and components to the OSGi runtime. An OSGi runtime allows for bundles to be installed, started, stopped, updated and uninstalled without requiring a reboot.

The OSGi Core spec defines a layered architecture that determines what is supported by the runtime. Each layer defines a particular functionality supported by the runtime and the bundles.

Bundles may export packages for use by other bundles, or import packages exported by other bundles. This dependency mechanism is referred to as wire protocol and is provided by the module layer of OSGi.

Bundles may publish services to the runtime and use already published services from the runtime. This dependency mechanism is provided by the service layer of OSGi.

The MANIFEST.MF file of the bundle’s JAR file describes the metadata of the bundle.

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: Sample
Bundle-SymbolicName: com.exoscale.sample
Bundle-Version: 1.0.0.qualifier
Bundle-Activator: com.exoscale.sample.Activator
Bundle-Vendor: Exoscale
Require-Bundle: com.exoscale.otherbundle
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
Service-Component: OSGI-INF/service.xml
Import-Package: com.exoscale.services.example;version="3.0.0“
Export-Package: com.sample.utils

Microservices From the OSGi Perspective

One can find different definition of microservices but all of them boil down to the notion of an architectural style for development of loosely-coupled, independently deployable units of work that are centred around a business capability (or a tightly-knit group thereof). They should be able to communicate with lightweight protocols and can be developed, tested and scaled independently.

This implies that microservices relate directly to modularization of the system. In that sense we can think of OSGi as microservice-oriented framework running in a JVM environment. To be more precise lets reveal how do the different microservice concepts map to the OSGi world:

However, if we span outside the premises of the OSGi environment, we get into additional challenges related to all of the above and including additional considerations such as load balancing, auto-scaling, self-healing, distributed tracing, resilience, fault tolerance and back-pressure to name a few.

All of these can be enabled by means of the Remote Service and Remote Service Admin specification defined by the OSGi compendium specification.

To be more precise the Remote Service Admin spec defines a pluggable management agent called Topology Manager that can fulfil the different concepts mentioned that characterize the interaction between remote OSGi services (effectively the communication channel between the different OSGi runtimes that form the set of microservices).

A Microservice-Based OSGi System

Let’s demonstrate how the discussed architecture applies to a simple system for car manufacturing.

For the purpose of simplicity we will limit the system to the production of the main parts. The system consists of three microservices which are distinct OSGi runtimes with different modules related to the particular microservice:

• body – includes separate modules for the production of the different body parts such as hood, bumper, pillars and spoilers;
• doors – includes separate modules for the production of door components such as handles, locks and hinges;
• windows - includes separate modules for the production of window components such as glasses and glass regulators.

The system works in a supply-chain manner, whereby we first build the body of the car, then the doors and finally the windows, before they are finally assembled.

Moreover, since each particular phase of the supply chain is modular, we can upgrade or replace entire modules that build certain parts (such as the hood) without really interacting with the full supply-chain and bringing the system out of operations.

The different microservices gather configuration from a central configuration microservice deployed in a separate OSGi container having just a single configuration module. The system is illustrated schematically on the following diagram:

Each distinct microservice has an assembly module that is responsible to assemble the parts for the component represented by the microservice.

Remote services provide a way to expose standard OSGi services to external applications using transport mechanisms such as SOAP or REST webservices, Apache Aries Remote Service Admin, r-osgi (Remote Services for OSGi) and others.

Two of the most notorious frameworks that provide implementation of the remote service and remote service admin specifications are Apache CXF and the one provided by ECF (Eclipse Communication Framework).

We will be using the Apache CXF distributed OSGi subproject that implements a REST provider for the Aries Remote Service Admin. We will be deploying our application in a Karaf environment using Felix OSGi runtime (Karaf also provides support for running an Equinox OSGi runtime).

Note that Karaf provides an alternative mechanism for creating a distributed OSGi runtime by means of the Karaf Cellar runtime.

First download latest Karaf version from the Karaf website.

Instead of downloading the CXF DOSGi distribution Karaf comes with a feature that installs it directly in Karaf.

We will demonstrate the sample implementation of the presented system by creating the car body assembly service that communicates with the config services by retrieving configuration data, and writes back completion status after a successful assembly completion.

Since we will need to two separate Karaf runtimes for the deployment of the services we will simulate the scenario by running the runtimes locally on different HTTP ports (9991 and 9992).

Unzip the Karaf distribution at two different locations, create an etc/org.ops4j.pax.web.cfg file in the two installations. In the first distribution specify the following in the created configuration file:

org.osgi.service.http.port=9991

And for the second specify port 9992:

org.osgi.service.http.port=9992

Start the two Karaf runtime with an interactive console as follows:

bin/karaf.bat

After the runtimes are started install the CXF DOSGi feature (with the CXF JAX-RS provider) and Jackson JSON provider on both of them as follows:

feature:repo-add cxf-dosgi
feature:install cxf-dosgi-provider-rs
feature:repo-add mvn:org.code-house.jackson/features/2.8.11/xml/features
feature:install jackson-jaxrs-json-provider
feature:install jackson-module-jaxb-annotations

Now let’s implement and deploy the services one by one and run an example scenario that demonstrates that our services interact as expected.

The Configuration Service

The configuration service API is provided by a config.api module that is shared across the different Karaf runtimes that need to interact with the configuration service.

The Maven configuration that provides the build information for the module is available here.

We use the Maven bundle plugin to generate the module metadata (in the MANIFST.MF file of the module). In order to represent a configuration entry we are going to use a simple POJO provided by the com.exoscale.carassembly.config.api.model.Config class:

public class Config {

    private String key;

    private Object value;

    public String getKey() {
        return key;
    }

    public void setKey(String key) {
        this.key = key;
    }

    public Object getValue() {
        return value;
    }

    public void setValue(Object value) {
        this.value = value;
    }
}

The configuration service API is provided by the com.exoscale.carassembly.config.api.ConfigService class:

@Path("config")
public interface ConfigService {

    @GET
    @Consumes({ "application/json", "application/xml" })
    @Produces({ "application/json", "application/xml" })
    public Config get(@QueryParam("key") String key);

    @DELETE
    @Consumes({ "application/json", "application/xml" })
    @Produces({ "application/json", "application/xml" })
    public Config remove(@QueryParam("key") String key);

    @POST
    @Consumes({ "application/json", "application/xml" })
    @Produces({ "application/json", "application/xml" })
    public Config add(Config config);

}

Build the module as follows:

mvn package

And then install it and start it as follows in the two Karaf runtimes (through the Karaf consoles):

install file:///D:/config.api/target/config.api-1.0.0-SNAPSHOT.jar
start com.exoscale.carassembly.config.api

Alternatively Karaf gives you a bundle ID you can use to start the bundle with the start command.

You can verify the bundle is started by investigating runtime modules with the list command.

The Configuration Service Implementation

The configuration service implementation is provided by a config module that is deployed only on the first Karaf instance (running on port 9991).

The Maven configuration that provides the build information for the module is available here.

The com.exoscale.carassembly.config.GeneralConfigService provides the implementation of the configuration service (full implementation available here ):

@Component(service = ConfigService.class, immediate = true, property = //
{ //
        "service.exported.interfaces=*", //
        "service.exported.configs=org.apache.cxf.rs", //
        "org.apache.cxf.rs.address=/api", //
        "cxf.bus.prop.skip.default.json.provider.registration=true"//,
} //
)
public class GeneralConfigService implements ConfigService, IntentsProvider {

    ...

    // the CXF intents provide a way to register additional capabilities for the OSGI service
    // in this particular case we register an intent that supplies the Jackson JSON provider 
    // so that we can use JSON for marshalling and unmarshalling
    @Override
    public List<?> getIntents() {
        return Arrays.asList(new JacksonJaxbJsonProvider());
    }
}

The significant part of the implementation is the single compile-time @Component declarative service annotation that is used to generate the service descriptor XML file under the OSGI-INF directory of the module.

The annotation is an easier way to define OSGi service metadata than writing service XML file. In fact it generates the files for use. In this particular case that is OSGI-INF/com.exoscale.carassembly.config.GeneralConfigService.xml. It also provides additional attributes used by the OSGi Remote Service feature (in our case CXF DOSGi):

• service.exported.interfaces – specifies a list of the service interfaces to be exposed remotely (the * value encompasses all interfaces implemented by the service implementation);
• service.exported.configs – specifies how are the services going to exposed remotely, in our example we specify REST services (as provided by the CXF JAX-RS implementation);
• org.apache.cxf.rs.address – this is the root path under which the services will be exposed remotely (in our case that is going to be localhost:9991/cxf/api);
• cxf.bus.prop.skip.default.json.provider.registration – this property is needed in order to tell CXF not to use the default JSON provider. Rather than that a Jackson JSON provider is registered (as a CXF intent) by specifying the org.apache.cxf.dosgi.common.api.IntentsProvider interface on the service, implementation and overriding the getIntents() method.

Build and deploy the module on the Karaf instance running on port 9991.

the Car Body Assembly Service

The car body assembly service is provided by the body.assembly module that has the following Maven metadata: The Maven configuration that provides the build information for the module is available [here]https://github.com/exoscale-labs/Apache_Karaf_Microservices_article/tree/master/sources/body.assembly/pom.xml).

The body assembly service is provided by the com.exoscale.carassembly.body.assembly.services.BodyAssemblyEndpoint interface:

@Path("assembly")
public interface BodyAssemblyEndpoint {

    @POST
    public void assemble();
}

And the body assembly service implementation is provided by the following implementation (which also exposes the body assembly service as a remote one):

@Component(service = BodyAssemblyEndpoint.class, immediate = true, property = //
{ //
        "service.exported.interfaces=*", //
        "service.exported.configs=org.apache.cxf.rs", //
        "org.apache.cxf.rs.address=/api", "cxf.bus.prop.skip.default.json.provider.registration=true" } //
)
public class BodyAssembly implements BodyAssemblyEndpoint, IntentsProvider {

    private final static Logger LOGGER = LoggerFactory.getLogger(BodyAssembly.class);

    public void assemble() {
        ConfigService configService = setupConfigClient();
        LOGGER.info("Assembling body with color: " + configService.get("color").getValue());
        assembleBodyParts();
        assembleDoors();
        assembleWindows();

        Config resultConfig = new Config();
        resultConfig.setKey("assemblyFinised");
        resultConfig.setValue("true");
        configService.add(resultConfig);
    }

    private void assembleDoors() {
        // call the doors assembly service ...
    }

    private void assembleWindows() {
        // call the windows assembly service ...
    }

    private void assembleBodyParts() {
        // assemble the body parts by triggering the various body part services
    }

    private ConfigService setupConfigClient() {
        // we register Jackson JSON provider for use by the JAX-RS client so that it can use JSON for marhsalling and unmarshalling
        LinkedList providers = new LinkedList<>();
        providers.add(new JacksonJaxbJsonProvider());

        // we use the CXF JAR-RS client factory to create a client for the ConfigService
        ConfigService configService = JAXRSClientFactory.create("http://localhost:9991/cxf/api", ConfigService.class,
                providers);
        return configService;
    }

    @Override
    public List<?> getIntents() {
        return Arrays.asList(new JacksonJaxbJsonProvider());
    }
}

Build and deploy the body assembly service on the second Karaf instance running on port 9992.

Testing the System

In order to make sure that the modules are started correctly without any exception you can run the following command on both Karaf instances to display the log entries:

log:display

We are going to first create an entry in the distributed configuration service that is going to specify the color of the car body, then trigger the body assembly service that writes status back to the configuration service, and finally review the status from the configuration service:

curl -X POST http://localhost:9991/cxf/api/config --data "{\"key\":\"color\",\"value\":\"blue\"}" --header "Content-Type: application/json"
curl -X POST http://localhost:9992/cxf/api/assembly --header "Accept: text/plain"
curl -X GET http://localhost:9991/cxf/api/config?key=assemblyFinised --header "Content-Type: application/json"

You should see the following result from the car body assembly:

{"key":"assemblyFinised","value":"true"}

Distributed Service Discovery With Zookeeper

In the demonstrated setup we used a JAR-RS client for interacting with the configuration service.

It will be way better to just inject the configuration service as a regular OSGi service and let Karaf take care of the service discovery out of the box.

This is possible by setting up Zookeeper for service discovery which is quite straight-forward.

We need to download Zookeeper and create a configuration for it (the zoo_sample.cfg file that comes with the Zookeeper installation can just be copied to a zoo.cfg file as a basic test configuration). We can start it by running:

bin\zkServer.cmd

In addition we need to install the following zookeeper CXF features in the Karaf runtimes:

feature:repo-add cxf-dosgi 1.7.0 
feature:install cxf-dosgi-discovery-distributed cxf-dosgi-zookeeper-server
For each of the runtimes Zookeeper configuration can be specified by creating a {KARAF_INSTALL}/etc/org.apache.cxf.dosgi.discovery.zookeeper.cfg file with the following contents:
zookeeper.host=localhost
zookeeper.port=2181

Then simply restart the Karaf instances and you are all set for distributed service discovery through Zookeeper.

Summary

We have implemented a complex style of architecture that is suitable for large scale systems using the OSGi framework in conjunction with the microservice paradigm.

While our example is simple, it models an architecture that can span hundreds of microservices with multiple modules on each.

OSGi remote services provided out of the box distributed service discovery.

Implementing concepts such as load balancing, fault tolerance and any of the others already mentioned can be done similarly to how we’ve introduced a centralized configuration service in the architecture.