The secrets of evolvable software

TL;DR: Modularization is the key to evolvable software. This blog post sheds some light on different aspects of modularization of software systems.

Motivation

The IT business is very dynamic. Changes happen everytime and everywhere. Programming languages, frameworks, tools, infrastructure – it feels like almost everything is changing all the time. For enterprises this is a challenge as it constantly requires modernization and maintenance. Ok, this is part of the nature of software systems today you might say, so why should we care?

Because from a business perspective it would be nice to have systems which do not require expensive rewrites now and then for cost reasons. Instead it would be better to have software that is able to evolve over time. Of cource maintenance is always required for example to get security patches. But if the software is build with evolvability in mind it can be adapted to future needs more easily (without major reimplementation). Evolvable software is structured in a way that enables change. It is one of the core values of Clean Code. Simply put evolvability is an important trait of modern software systems.

How can we get there? Let’s have a look at several aspects and good practices to create evolvable software.

Design

The key of evolvable software is in its design. And because the business domain is much more stable than technologies, the business is a good basis for design considerations. Domain Driven Design (DDD) means exactly that. Instead of using technical concepts such as message brokers, databases and so on we use business concepts such as products, sales or invoicing to organize our software system.

Especially important is the strategic design and in particular the concept of Bounded Contexts which is the central design pattern in DDD. Bounded Contexts foster modularity already at the design level. Domain design is carried out together with the experts from the particular domains and can be documented ideally with a graphical notation such as UML.

Codebase

Based on the domain design the codebase can be created and organized. I often see systems in which design model and code are decoupled which makes it difficult to understand the codebase. Moreover code is often structured based on technical concepts such as controllers, entities and so on. Although technical terms can be used at a lower level, the main structuring concept should be the domain. It is important to keep the design structure from the business domain at the code level to foster maintainability and evolvability.

Modularity is of utmost importance when it comes to evolvable software systems. This has to do with the fact that even the best programmers in the world are not able to understand software that is too large and complex. Modularity helps to chunk systems into smaller parts that are more likely to be understood.

Assume a Java based system. What are the options for modularity. At the language level we have packages. Actually packages are intended to structure a codebase into logical units. By keeping classes package private and exposing dedicated interfaces you can enforce encapsulation of your modules. Unfortunately it does not properly work with subpackages, but if a module is not too large this is not neccessarily a problem. By enforcing architectural constraints on your codebase with tools such as ArchUnit you can even relax encapsulation while keeping the codebase clean. If you want versioned artifacts, you can create libaries for instance with maven or gradle. If you need more independence put the libraries into separate repositories.

Services and APIs

Is that already a service? If the module has a dedicated interface contract I would say yes. But that is more of a philosophical question 😉

If you want to expose the service in an interoperable way, just add an HTTPS-Endpoint with REST- or RPC-style. No matter whether you choose a code- or contract-first approach, always go for an API-first aproach, as this gives you a better an more though-out structure which is usually closer to the business domain. My personal favourite is contract-first, for instance based on OpenAPI, because it is technology agnostic and opens the way for alternative implementation languages. Do you remember? We are talking about evolvable software. Even programming languages are changing over time. And even asynchronous interfaces and message broker based services deserve an API-contract, which can be created using AsyncAPI.

Frontend Components

Discussions about services most of the time happen on the server side in the form of service oriented architecture or microservices. But user interfaces can be modularized as well. Most modern UI frameworks such as Angular or React have a component model on board. UI components have interfaces as well. Those interfaces comprise everything that communicates with the outside world of the component such as attributes, events, cookie/local/session storage and window messages that should be properly documented. Modular distribution can be achieved by creating npm packages. If you want to increase interoperability and freedom of technology implement ui components as web components.

Containers

Containerization is a strong trend for good reasons. But over the last years the industry has learned that it is not always the best option to deploy each service or module as a separate unit. Deploying and running a lot of small units requires more infrastructure which can be costly to implement and maintain. In the past the pendulum swang from monoliths to microservices architectures which are both extreme in its implementations. Today so called moduliths are combining modularity with monolithic deployment models. Starting with a modulith can be a good option, evolving it to a microservice-based deployment model if and only if required. You can deploy a modulith standalone or using Docker and Kubernetes the same way as you would do with a single service if this is the runtime of your choice. Anyway it is important to find a proper granularity of modules. Self contained systems aligned to the business domains are often a good option in this regard. This also helps to keep cognitive load and responsibility manageable at the team level. This is again a matter of design.  

Limits

What happens when the programming Plattform changes completely, let’s say from .NET to JEE or from Java to Python? Even then you can save parts of your investments. Design outcomes and standards-based artifacts such as OpenAPI contracts are platform agnostic and can be reused. And as the business domain is usually quiet stable and not connected to changes in technology , you have a good chance to reuse your design outcomes as is.

Summary

Modularization is essential when it comes to creating evolvable software. Technologies are powerful enough to do it. Modular software is primarily a matter of design and the will to invest some effort into it. From the many projects that I have seen in my career, I am convinced that it is one of the most important things to do when creating professional software systems.

Cars, Software and Competitiveness

It is often said that software development should be like building cars.

But cars are very different from modern software systems.
Since its invention more that 100 years ago a car is a very stable concept.
This works because the purpose for which it is build (driving on a more or less solid ground) is stable as well.

Software on the other hand needs to be much more flexible because the context changes frequently.
Some examples are markets, organizations, products and the internet.
The concept of software is much more abstract than the concept of a car.

More and more people realize that agility paired with quality is a key success factor.
That’s why a short concept to market cycle is needed.

Automotive is in fact a bad role model. You can see that every day.
The market is demanding for efficient cars with a low carbon footprint.
But the vendors fail to deliver. Why? Because they are too inflexible.

Let us not make the same fault.
In fact the methods and tools are available to establish a fast concept to market cycle.
Some examples are Scrum, Silverlight SketchFlow and Business driven SOA

If the companies have the heart to try those new approaches, they might be with rewarded with competitive advantage and flexibility.

Biztalk Technology Specialist

I passed the Microsoft Biztalk Server certification (MCTS) with flying colors (985).

An MCTS is:

“A Microsoft Certified Technology Specialist in Microsoft BizTalk Server 2006 (MCTS: BizTalk Server) has a deep and broad understanding of the design and development of distributed applications that use BizTalk Server 2006. The credential holder has also demonstrated expertise in deploying and managing a BizTalk Server 2006 solution and can create a BizTalk orchestration, integrate business rules and human workflow services, manage business processes, troubleshoot BizTalk solutions, and consume and publish Web services.”

I knew that before, but now it is official. 😉

Now I an entitled to deliver the official Biztalk MOC courses (2033 and 2934) in addition to the regular development courses.

How to enable request logging for pax-web-service OSGi bundle

Due to lack of documentation enabling request logging for the pax-web-service OSGi bundle can be time consuming task.
This is how it can be achieved:

1. Create a OSGi fragment bundle using Maven 2 like this: pom.xml
2. Add jetty.xml to src/main/resources directory of your project. Example: jetty.xml
3. Install the bundle into your OSGI environment.

This approach works with pax-web-service-0.6.0.jar and Eclipse Equinox 3.4.0.

Evolution of Languages

After working a while with Groovy in a project I have the impression that it is the better Java.
Mainly because it introduces nice productivity features such as closures, runtime compilation, loose typing, operator overloading just to name a few.
These features are available in C# for instance as lambda expression, extensions methods or type inference. And runtime compilation and operator overloading are even part of the .NET-Platform since version 1.
But while the C# language evolves from version to version in a logical way, in the Java world a new language is created. It is called a scripting language. Why that? Every language that can be compiled at runtime can be used as a scripting language.
I think it would be better to add those productivity features to the next version of Java without further cluttering the Java platform. This would help developers and decision makers to choose the right technology.

The Future of Java

In his newsletter What’s Next? Life After J2EE David Chappel writes down some interesting thoughts about the future of Java.

“…. Given this reality, what will the dominant Java application platform look like in the next few years? One possibility is that no leading choice will emerge, that JEE5 and Open SOA and various open source technologies will all gain a reasonable number of adherents. While technology fans might rejoice at this, enterprise decision makers

QuickModeler 2.1 released

When Visual Studio 2005 was released, I thought that there wouldn’t be the need for a reverse engineering class designer anymore. So I did not migrate QuickModeler to VS.NET 2005. But in the recent months I got many requests for this add-in. This is probably due to the fact that Visual Studio Express does not contain a class designer. So I decided to migrate it and finally it