The Future of ServiceMix

In the ServiceMix forum there is an interesting debate going on about the Future of ServiceMix. I’ve collected some quotes from the thread. I know it is sometimes dangerous to cite people without giving the full context, but I think it helps to get a rough idea about the direction in which ServiceMix is heading. If you are interested in more details please read to original post.

Quotes:

“The idea is really to make ServiceMix the best possible container for deploying Camel based integration.”
 
“This also means that there’s really not much to reuse from smx4, so smx5 would have its own new and dedicated svn area.”
 
“Dropping JBI is already a great step in that direction.”
 
“Additional things that could be pushed inside ServiceMix 5 would be a Tomcat based container deployment option (for those that don’t need OSGi), a new manual similar to what we have in Karaf (maybe reusing parts of it).”
 
“For advanced use cases, we should definitely guide users towards Karaf, but for a set of basic things, just using Tomcat and simple WARs may be the simplest thing that works for them!”
 
“To build the added value I’ve been talking about, we don’t really need the NMR imho.  I’m convinced we can do it directly on top of camel without the need for another layer, mostly using Camel interceptors.   That way, OSGi is not really needed anymore, though it’s still the best way to deploy Camel routes  using Karaf and would remain the main and standlone ServiceMix distribution.”
 
“Providing a way to easily deploy Activiti would definitely be a good thing. I don’t think it should part of the default distribution, as I’d like to keep it as light as possible though.”
 
Here is my personal summary: 
  1. ServiceMix 5 will primarily focus on Camel.
  2. JBI is finally dead.
  3. Karaf and OSGi will represent the primary runtime.
  4. Tomcat will act as an alternative container.
  5. Activiti will not be integrated.

Consequences:

If you develop for ServiceMix 4 today, focus on OSGi and Camel instead of  JBI and ServixMix features since those are the stable parts of ServiceMix.
If you have ServiceMix 3 and JBI solutions in place think about migration towards Karaf and Camel. If you plan to integrate Activiti, make sure the required functionality is not already covered by Camel to avoid unnecessary complexity (see blog post Combining Activiti and Camel).

Combining Activiti and Camel?

It seems that that ServiceMix/Camel and Activiti are growing together as you can read in the blog post Supersize Activiti with Mule ESB and Apache Camel and Deploy Activiti as OSGi Bundles

Activiti has its roots in business process management, whereas Camel comes from the area of enterprise integration. Both allow to describe and automate processes. Activiti uses BPMN, a graphical modeling language with an XML representation for persistence and execution. Camel uses a textual DSL which can also be expressed in XML. Grahpical modeling is also possible with the Fuse IDE.

BPMN example:

BPMN process
BPMN process

The BPMN XML representation is not intended to be written directly and can be hard to read.

Camel example:

from("file://start")
.choice()
  .when(header("goldCustomer").isEqualTo("true"))
    .to("bean:processor?method=grantDiscount")
    .to("direct:out")
  .otherwise()
    .to("direct:out");

from("direct:out")
.to("file://end");

Basically both examples are identical. The Camel example is more explicit and shows more technical details which are hidden in the BPMN example.

Especially non technical users tend to think that a graphical process notation is easier to understand because it looks better. It might be true for simple processes like the one above. But for more complex processes textual DSLs are often easier to write and understand.

Activiti and Camel overlap in many areas. Does it make sense to use them side by side anyway? I think yes, because neither Activiti nor Camel is complete in terms of features required by a fully featured SOA/BPM runtime. Actually they complement each other very well in some areas.

Unique features added by Camel:

  • Connectivity / Adaptors to connect to external systems using a variety of different protocols 
  • Predefined EIP patterns for message routing

Unique features added by Activiti:

  • Support for long running stateful processes
  • Human workflow integration

In order to avoid unnecessary complexity it helps to recall the difference between mediation- and business processes which I descibed in my blog post from 2006.

For mediation processes Camel is appropriate in most cases, even long running processes can be executed as long as they are stateless. But if you need long running stateful processes and human interaction Activiti is an interesting option. 

Both Camel and Activiti processes can be modeled using prescriptive and analytical BPMN. The software designer has the responsibility to decide which technology is appropriate for implementing the process. If this is not done right you will end up with unnecessary complexity, limited agility and maintainability.

It would be ideal to have an Open Source SOA/BPM Platform that enables end to end process automation without technology boundaries. As long as this is not available, combining Activiti and Camel is an interesting option. Proper solution design and differentiation of process types is key to create sustainable solutions.

Core Values of Great Software

Over the last decade I have seen many software solutions. Some excellent and some really bad ones. But most solutions were somewhere in the middle. They just did what they were supposed to do.

But is this really good enough? I don’t think so. Especially today where time to market is critical and the budgets are limited, functioning software is the bare minimum. On top of that other core values are getting more and more important.

  • Functionality – This is obvious.  The software must do what people expect it to do in a reliable manner.
  • Evolvability – This is also known as sustainability. A value that is important in many areas of our daily life. And also in software development. It means that we don’t want to reinvent the wheel over and over again. We don’t want to produce waste. Software must be flexible so that it can be reassembled even if technologies change. Investements have to be saved. Evolvability can be primarily achieved through proper software design. Especially service-oriented approaches are promising here.
  • Production efficiency – This is the most underestimated value. It means that just work is not enough, rather it must be quick and easy to produce. For a long time IT is has a reputation of being complicated and expensive. As a reaction to that companies are outsourcing to cheaper countries or applying more flexibile development processes auch as Scrum to improve efficiency. Especially the latter has proven to be successful. Choosing the right software is also a good way to come closer to that goal. For example frameworks should not only assessed by its functionality (often done by developers) but also on the ease of use and developing performance.

Although those values are important for every software project, they are especially important in todays SOA/BPM undertakings as they promise to increase agility and time to market. This can only be achieved with software solutions that are flexible and easy to use. Ideally even for business people without a developer background.

Speaking at Agility & User Experience Expert Forum

I am glad to be invited to speak at the Agility & User Experience Expert Forum. The conference organized by Accelsis and the Fraunhofer Institute for Experimental Software Engineering is going to take place on 16.June 2011 in Frankfurt/Germany.

The conference is packed with lots of real world examples showing how innovative approaches in the areas of usability and agility can improve corporate software development.

In my presentations I am going to show how agile techniques can be used in fixed price contexts (Yes, that is possible!) and how to achieve real operational agility beyond just Scrum.

I look forward having inspiring discussions and hope to see you there.
You can register here.

Activiti and the Open Source Reference Platform

Now that Activity published the first release (version 5.0) of their Open Source BPMN 2.0 engine and toolset, it was time to integrate it in our Open Source Reference Platform for SOA (OSRP).

Despite some initial weaknesses, such as BPMN error handling, Actitivi is very promising if you are looking for a lightweight, Open Source BPM engine. Especially the features in the areas IT/Business collaboration and human task  integration are worth a closer look. The modularity makes it easy to integrate the engine with other products.

Using the OSRP one can now compare the BPMN and BPEL approaches to BPM, understand how Activiti can be combined with other Open Source SOA technologies such as Apache ServiceMix ESB, and learn about the strenghts and limitations of the product.

Schema first contract design

One of the most important artifacts in Service-oriented Architecture (SOA) is the service contract. The contract defines how service producers and consumers interact with each other. Most people agree that a contract first approach is the preferred way of creating service contracts. WSDL based service contracts are very common. In many projects they are generated by the respective frameworks, such as Apache CXF or Microsoft WCF. This is dangerous, as the developer looses control over the contract. This can result in missing interoperability and limited reusabilty expecially in cross platform scenarios.
From what I see, most developers know how to work with their favourite IDE, but only a few know how to create a proper service contract from scratch. I think that is caused by the tool vendors, which in most cases have better support for code based approaches.
In this post I would like to show how to create a portable contract from scratch and share some best practices.

The domain model

The first thing to think about is the domain or entity model that the service should expose. The model should be modeled in XML Schema. XML Schema is standard, portable and supported on all platforms. In fact it is a good idea to use XML Schema as the single source of model information.

The following listing defines the entity Product:
 



  
    
      
        
          Name of the product
        
      
      
        
          Quantity on stock
        
      
    
  

The entity Product has two attributes name and quantity. Please note that the xml elements (line 10 and 15) are abbreviated. This helps to reduce the message size on the wire, expecially important in WANs. The downside is obviously reduced readability. That is not really a problem as the contract should be documented anyway using the standard XML Schema annotation elements as shown in line 11 to 13.
This is the result:

Wouldn’t it be nice to have a common base class in order to create a more generic service contract?
Luckily XML Schema allows that. Let’s create another schema BaseTypes.xsd in which we define the base types.



  
    
      
      
    
  

  
    
      
      
      
      
    
  

  
    
      
        
          Unique key
        
      
      
        
          Status of the entity
        
      
    
  

This base type Entity (line 24) defines common attributes for all entities. In XML Schema one can define enumerations as well (line 8 and 15). The enumeration EntityStatus defines the state of the entity. This is important in scenarios in which the entities are detached from the database. By marking the entities, it is easy to implement change tracking on the consumer side and just return the changed or deleted parts to the server. This saves network and processing time. EntityType is used later in our service interface to request different types of entities.
We can now import the base types into Types.xsd and change the Product type:





  
    
      
        
          
            
              Name of the product
            
          
          
            
              Quantity on stock
            
          
        
      
    
  

You can see the import in line 9. In line 12 and 13 the base type is added to the Product type.

This is the result:
Entity with base class

Now we have our model completely defined in XML Schema. By using imports and type inheritance one can create very modular entity models. Those models can be reused without any change by interface definitions such as WSDL and WADL for SOAP or REST services. Moreover the code models (Java, C#, etc.) can be fully generated from the schema. Every good framework has a generator tool on board. For instance Apache Axis/wsdl2java, Apache CXF/wsd2java, Silverlight/slsvcutil, .NET/svcutil, etc. The schema can be stored in a service repository in order to increase reusability amongst the organisation.

The messages
Every service operation receives a message and returns a message to the sender. Those messages should also be defined using XML Schema. The following snippet shows the message definitions. The fault messages are ommitted for the sake of simplification.


    
      
    
  

  
    
      
    
  

    
        
            This represents a request to read business entities
        
        
            
                
                    
                        Type of the entities to read
                    
                
                
                    
                        Unique keys of the entities to read
                    
                
            
        
    

    
        
            This represents the read response
        
        
            
                
                    
                        List of entities
                    
                
            
        
    

Those are our messages:

ReadEntitiesRequest and ReadEntitiesResponse are bulk operations that allow to read multiple entities with one call. As you can see ReadEntitiesResponse does not return Products but Entities. Do you remember? This is our base class type for all entities. By using the base class, the message can return arbitrary entity types in the same message. This is an example for a strongly typed but yet extensible interface.

The schemas can be reused in WSDL files:



  
  
  
  
      
  
  
    
  
  
  
    
      
      
    
  

  
        
    
      
      
        
      
      
        
      
    
  
  
    
      
    
  

The type information and message definitions are completely externalized to individual schemas. As a result the WSDL is small. The imported types (line 4) are used to define the WSDL messages (line 7 and 10). A WADL file would look like this:



  
    
    
  

  
    
      
        

            
              
                This is the product id
              
            
          
        
        
          
        
      
    
  

The types are imported (line 8 and 9) and used to define the return message (25).
By using the respective tool, such as wsdl2java or svcutil, we can completely generate the model and messages for the client and server on JEE, .NET and other platforms.

Summary
As you can see it is not too difficult to create schema based service contracts in a platform agnostic way. If you do, you will be rewarded with a great level of control, portability and reusability. Powerful features of XML Schema, such as inheritance, allow to create robust and extensible service contracts, which are the basis for interoperability. And if you want, you can even use an XML editor. 😉

Forms Over Data vs. Domain Driven Design

A common challenge in Service-oriented architecture (SOA) is the design of service contracts. Beside many variations we see two basic types of contracts.

Forms over data (FOD) is an approach in which the forms in the user interface directly reflects the data structures in a database. The service-layer, if present, primarily offers create, read, update and delete (CRUD) operations for the individual database entities.

FOD is fostered by technologies such as Microsoft Access, Oracle Forms, .NET RIA Services. In JEE it is recognized as an anti pattern addressed by design (anti-)patterns such as Face Off or Session Facade.
FOD is tempting because large parts of the service-layer can be generated from an existing data model. By using the right tools, developing an FOD-application can be quick and easy.

Domain driven design (DDD) is an approach that utilizes a service-layer whose data objects are inspired by the business domain. The database structures are hidden behind a service-layer and not known by the service consumer, for instance the user interface. The domain model is usually modeled using XML Schema. DDD is typically used in SOA environments.

DDD-Pros

  1. Portable domain model – If modeled in XML-Schema the domain object can be generated on JEE, .NET and other platforms.
  2. Business alignment – A business oriented domain model can be better understood by business users, thus increasing agility.
  3. Higher degree of reusability – Services representing business concepts are more likely to be reused, for instance in BPM environments.
  4. Decoupling of user interface and database – Data structures can be changed (up to a certain degree, of course) without affecting the clients. Changes can be handled in the service layer.
  5. Maintenance – Proper domain driven services are designed to last. Paired with a versioning approach service maintenance can be simplified.

FOD-Pros

  1. Development effort – FOD is often supported out-of-the-box in many products, so that no using it would require more knowledge and development effort.

Although there are always areas in which one or the other approach makes more sense, you can clearly see my preference. I think forms over data can be particularly useful for small applications and prototypes. Especially with existing database models. For large enterprise applications one should prefer a domain driven service design in combination with a sound SOA approach.