The secrets of evolvable software

TL;DR: Modularization is the key to evolvable software. This blog post sheds some light on different aspects of modularization of software systems.

Motivation

The IT business is very dynamic. Changes happen everytime and everywhere. Programming languages, frameworks, tools, infrastructure – it feels like almost everything is changing all the time. For enterprises this is a challenge as it constantly requires modernization and maintenance. Ok, this is part of the nature of software systems today you might say, so why should we care?

Because from a business perspective it would be nice to have systems which do not require expensive rewrites now and then for cost reasons. Instead it would be better to have software that is able to evolve over time. Of cource maintenance is always required for example to get security patches. But if the software is build with evolvability in mind it can be adapted to future needs more easily (without major reimplementation). Evolvable software is structured in a way that enables change. It is one of the core values of Clean Code. Simply put evolvability is an important trait of modern software systems.

How can we get there? Let’s have a look at several aspects and good practices to create evolvable software.

Design

The key of evolvable software is in its design. And because the business domain is much more stable than technologies, the business is a good basis for design considerations. Domain Driven Design (DDD) means exactly that. Instead of using technical concepts such as message brokers, databases and so on we use business concepts such as products, sales or invoicing to organize our software system.

Especially important is the strategic design and in particular the concept of Bounded Contexts which is the central design pattern in DDD. Bounded Contexts foster modularity already at the design level. Domain design is carried out together with the experts from the particular domains and can be documented ideally with a graphical notation such as UML.

Codebase

Based on the domain design the codebase can be created and organized. I often see systems in which design model and code are decoupled which makes it difficult to understand the codebase. Moreover code is often structured based on technical concepts such as controllers, entities and so on. Although technical terms can be used at a lower level, the main structuring concept should be the domain. It is important to keep the design structure from the business domain at the code level to foster maintainability and evolvability.

Modularity is of utmost importance when it comes to evolvable software systems. This has to do with the fact that even the best programmers in the world are not able to understand software that is too large and complex. Modularity helps to chunk systems into smaller parts that are more likely to be understood.

Assume a Java based system. What are the options for modularity. At the language level we have packages. Actually packages are intended to structure a codebase into logical units. By keeping classes package private and exposing dedicated interfaces you can enforce encapsulation of your modules. Unfortunately it does not properly work with subpackages, but if a module is not too large this is not neccessarily a problem. By enforcing architectural constraints on your codebase with tools such as ArchUnit you can even relax encapsulation while keeping the codebase clean. If you want versioned artifacts, you can create libaries for instance with maven or gradle. If you need more independence put the libraries into separate repositories.

Services and APIs

Is that already a service? If the module has a dedicated interface contract I would say yes. But that is more of a philosophical question 😉

If you want to expose the service in an interoperable way, just add an HTTPS-Endpoint with REST- or RPC-style. No matter whether you choose a code- or contract-first approach, always go for an API-first aproach, as this gives you a better an more though-out structure which is usually closer to the business domain. My personal favourite is contract-first, for instance based on OpenAPI, because it is technology agnostic and opens the way for alternative implementation languages. Do you remember? We are talking about evolvable software. Even programming languages are changing over time. And even asynchronous interfaces and message broker based services deserve an API-contract, which can be created using AsyncAPI.

Frontend Components

Discussions about services most of the time happen on the server side in the form of service oriented architecture or microservices. But user interfaces can be modularized as well. Most modern UI frameworks such as Angular or React have a component model on board. UI components have interfaces as well. Those interfaces comprise everything that communicates with the outside world of the component such as attributes, events, cookie/local/session storage and window messages that should be properly documented. Modular distribution can be achieved by creating npm packages. If you want to increase interoperability and freedom of technology implement ui components as web components.

Containers

Containerization is a strong trend for good reasons. But over the last years the industry has learned that it is not always the best option to deploy each service or module as a separate unit. Deploying and running a lot of small units requires more infrastructure which can be costly to implement and maintain. In the past the pendulum swang from monoliths to microservices architectures which are both extreme in its implementations. Today so called moduliths are combining modularity with monolithic deployment models. Starting with a modulith can be a good option, evolving it to a microservice-based deployment model if and only if required. You can deploy a modulith standalone or using Docker and Kubernetes the same way as you would do with a single service if this is the runtime of your choice. Anyway it is important to find a proper granularity of modules. Self contained systems aligned to the business domains are often a good option in this regard. This also helps to keep cognitive load and responsibility manageable at the team level. This is again a matter of design.  

Limits

What happens when the programming Plattform changes completely, let’s say from .NET to JEE or from Java to Python? Even then you can save parts of your investments. Design outcomes and standards-based artifacts such as OpenAPI contracts are platform agnostic and can be reused. And as the business domain is usually quiet stable and not connected to changes in technology , you have a good chance to reuse your design outcomes as is.

Summary

Modularization is essential when it comes to creating evolvable software. Technologies are powerful enough to do it. Modular software is primarily a matter of design and the will to invest some effort into it. From the many projects that I have seen in my career, I am convinced that it is one of the most important things to do when creating professional software systems.

Bank11 Success Story

In 2016 PLEUS Consulting supported Bank11 in the development of their brand new sales financing system VICTOR 3.0.

Success Story Bank11 is a credit institution that specializes in sales financing. In 2016 the bank decided to replace their existing software with something new. To be able to meet the challenging requirements in terms of quality, customer satisfaction and process efficiency they decided to build their own solution.

The front-ends were developed using modern web technologies such as Javascript, HTML5, CSS and Angular. For the backend Java Enterprise (JEE) and a Sustainable Service Design approach was utilized to design and build a backend with a high degree of reuse and scalability. The service landscape was established using Domain Driven Design principles.

On the technical side, PLEUS Consulting supported the teams as Master Developer and Architecture Owner. In the area of agile techniques, PLEUS Consulting supported the development teams as Scrum Master and Agile Coach. Although not 100% tension free, the combination of those roles worked quite well. With these roles the bank received thorough support in the areas of technology and methodology.

From the beginning we tried to align technology and business as much as possible, creating a people centered architecture. Central to the strategy were BPMN process models, graphical business rules and visual service contracts. In order to create appealing front-ends for the car dealers and the back office of the bank we worked closely with user interface specialists which were members of the cross functional teams. Web stack technologies allowed us to create individual and great looking front-ends. Agile frameworks such as Scrum organized the development teams and created valuable software together with the customer within a short period of time.

The project has shown that with a combination of modern technologies and agile approaches a very short concept to market cycle can be achieved, creating competitive advantages. It also demonstrates that it is possible to establish an agile culture in rather traditional business domains.

You can read more details about the project in the official success story. If you want to find out more come to watch my talks at JAX 2017 in Mainz.

Sustainable Microservices with Spring Boot

In my article series about Sustainable Service (SSD) design I described a design and implementation approach to develop services with technical decoupling to improve reuse.

At the level of IT infrastructure sustainable means that service implementations can be used in different technical environments without a major rewrite. Technical decoupling is a key factor to achieve that. In the second part of the article I have provided an example on how to implement SSD with a JEE stack. Part of this example is a  calculator service which performs simple arithmetic operations.

Due to the technical decoupling, services can be moved with little effort, for example from a JEE Server to other runtimes like Apache Karaf (OSGi) or Spring Boot, just to name a few.

In this blog post I would like to demonstrate how to move the Calculator JEE example from Wildfly JEE Container to Spring Boot. The main difference in the Spring Boot deployment is the fact that each service is bundled with its own HTTP server. The deployment unit is not a JEE EAR which is deployed on a JEE Server, but a so-called über-jar which includes the complete HTTP infrastructure. The über-jar just requires a Java runtime and no additional infrastructure. This kind of deployment creates a high level of service autonomy which is often used in Microservice architectures.

Let me tell you a little story:

Assume Peter is an IT professional who is working on a fictitious software project for a large insurance company. One day a colleague, let’s call him Max, from another project enters Peterʼs office and starts the following conversation:

Max: I’ve heard you’ve implemented some very useful services. I saw them on your service repository Wiki and think we could use some of them in our new project.

Peter: Yes, that’s right. I am glad we’ve created something valuable.

Max: But…I also heard that you are using a full blown JEE Applicationserver to run your services.

Peter: Yes, this is the best runtime for our project, as it helps us to manage centralized deployment. Each service is deployed in its own EAR file, which gives us great flexibility.

Max: For our project we decided to use Spring Boot and deploy each service together with its own HTTP Server. I guess we can’t use your service without a major rewrite then?

Peter: You don’t have to rewrite the services because we’ve implemented them based on Sustainable Service Design.

Max: Sounds great, could you please show me what we have to do to run your services?

Peter: Of course. Let’s start by downloading the Calculator Example, which demonstrates how to build SSD-Services for JEE. First, build the example like so:

mvn clean install

Peter: Now you have the following maven artifacts (jars) in your local maven repository.

net.pleus.services.calculator:calculator_api
net.pleus.services.calculator:calculator_impl

Peter: You can use this artifacts without any modifications.

Max: Ok, I can see that api and impl form the core service. Our project decided to just use JSON/HTTP as the protocol for service interaction. I saw that in the orignal example REST, EJB and SOAP are provided.

Peter: No problem, just add the bindings when you need them. With SSD you can add additional bindings any time. So you can start with the bare minimum and expand later. This gives you great flexibility. Add the JSON/HTTP-binding first.

Max: In the original example I saw that it was called REST-Binding.

Peter: Although the original calculator example uses the term REST-Binding, I prefer to call it JSON/HTTP-Binding because it better describes what it is. An SSD-Service can manage multiple resources (nouns) and can support arbitrary operations (verbs). This representation is very well suited to modelling the real world (the domain) which is important for proper service design and reuse. If you really have the requirement to create REST-Style APIs, you can do it in the respective binding. But be aware that in this case you create a variation of your service contract (subcontract) which also relies on HTTP-Verbs instead of the verbs in the primary contract. Although it is possible, I would not recommend it. Ok, let’s not digress but move on with Spring Boot.

Max: Ok, please show me how to create the bootable service.

Peter: Sure, start with the following Maven-POM , which is based on the tutorial Building a RESTful Web Service with Spring Boot.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.3.5.RELEASE</version>
    </parent>

    <groupId>net.pleus.services.calculator</groupId>
    <artifactId>calculator_boot</artifactId>
    <version>1.0-SNAPSHOT</version>
    <name>Services :: calculator :: boot</name>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>
        <dependency>
            <groupId>net.pleus.services.calculator</groupId>        
            <artifactId>calculator_api</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>net.pleus.services.calculator</groupId>        
            <artifactId>calculator_impl</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
    </dependencies>
    
    <properties>
        <java.version>1.8</java.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

Peter: From lines 25-34 you can see the existing calculator jars. The rest is required to create a minimal Spring Boot über-jar. Now we create a JSONHTTP-Binding using Spring MVC.

@RestController
@RequestMapping(value = "/services/calculator/rest/api", method= RequestMethod.POST)
public class CalculatorJSONHTTPBinding {

    @Autowired
    private Calculator service;

    @RequestMapping(value = "/performcalculations", method= RequestMethod.POST)
    @ResponseBody
    public PerformCalculationsResponse performCalculations(@RequestBody PerformCalculationsRequest request) {
        return service.performCalculations(request);
    }
}

Peter: To add this binding we need some boilerplate code. First an application class.

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        ApplicationContext ctx = SpringApplication.run(Application.class, args);
    }
}

Peter: And second a little factory to create a service instance, so that it can be injected using @Autowired in the binding.

@Configuration
public class Factory{
    @Bean public Calculator createCalculator(){
        return new CalculatorImpl();
    }
}

Peter: That’s all. Build it and run the über-jar with the following command.

mvn clean install
java -jar target/calculator_boot-1.0-SNAPSHOT.jar

Max: Wow, that’s all? Can you prove that it works?

Peter: Of course. For example fire up SOAP-UI and send this request to the service at http://localhost:8080/services/calculator/rest/api/performcalculations

{
	"correlationid":"4711",
	"calculations": {
	 "value": [
	   {
	     "operation": "ADD",
	     "inputs": {
	       "value": ["1","2"]
	     }
	   }]
	}
}

Peter: This is what you get.

{
   "correlationid": "4711",
   "calculations": {"value": [{"result": 3}]}
}

Max: Can I reuse the existing binding from the original example instead of the one we created?

Peter: Yes, but it is technically coupled to JAX-RS. If you want to use it please read the blog Using JAX-RS With Spring Boot Instead of MVC.

Max: I saw that the original example contains a handy Java client to access the service. Can I reuse it?

Peter: You mean net.pleus.services.calculator:calculator_binding_rest_client. Yes, you can use it as it is. And it makes sense, as it gives you a nice fluent Java-API to access the service. First add the following Maven artifact to your pom.xml.

        
            net.pleus.services.calculator        
            calculator_binding_rest_client
            1.0-SNAPSHOT
        

Peter: Now you can use the Java client in your tests like this.

@Before
public void setUp() throws Exception {
  client = new CalculatorClient("localhost",port);
}
    
@Test
public void add() throws Exception {

  // Create calculation inputs
  Calculation calculation = new Calculation()
   .withOperation(Operation.DIVIDE)
   .withInputs(new ArrayOfInt()
     .withValue(9,3,2)
   );
				 
  // Create request
  PerformCalculationsRequest request = new PerformCalculationsRequest()
    .withCorrelationid(UUID.randomUUID().toString())
    .withCalculations(new ArrayOfCalculation().withValue(calculation));
		
  // Call service	
  PerformCalculationsResponse response = client.performCalculations(request);

  // Check correlation
  Assert.assertEquals(request.getCorrelationid(), response.getCorrelationid());
		
  // Check result
  Assert.assertEquals(new BigDecimal(1.5), response.getCalculations().getValue().get(0).getResult());	
}

Peter: When you run the test Spring Boots starts the HTTP Server and calls the service.

Max: It seems that I can easily run your service and even use the Java client within Spring Boot. We have a sustainable service and a lightweight runtime. Perfect! That saves us a lot of time and money. Maybe we should try to evolve the service together? This way we could create further value for other projects.

Peter: Good idea! If you want to try the example I’ve packaged it for download. For convenience it also contains identical copies from the original calculator example.
Feel free to use it as you like.