Sustainable Service Design

We all know the idea of sustainability from our daily life. But is it possible to apply this idea to software development? I think yes.
Sustainable Service Design is a practical approach to design and implement software services with a great level of reuse both at technical and business levels. It is based on the following four principles:

  1. Technology-agnostic service definition
  2. Unified request/response
  3. Consequent contract first
  4. Technology bindings
00_cover_0 If you want to know more, please read my latest article about sustainable service design which has been publised in issue 2.2015 of Javamagazin (German).The interview can be found on the jaxenter site.
In the second part in issue 3.2015 I am showing how to implement a sustainable approach using JEE and JBoss Wildfly.

 

BiPRO takes SOA to the next level!

The advocates of Service Orientation always pointed out that SOA comprises 50% technology and 50% business. At BiPRO this vision becomes true.
BiPRO is a standardization organization for the insurance industry. BiPRO has members from the entire market including well known insurance companies and agents. By standardizing services at the business level using proven technical standards, BiPRO really takes the SOA idea to the next level. BiPRO conformant services are reusable and highly interoperable. Development is contract first, based on open standards such as WSDL and XSD. It is really impressive to see what BiPRO already achieved.

On 25.6.-26.6. June BiPRO-Day 2014 is going to take place in Düsseldorf Germany.

BiPRO-Tag2014_Signet_col

At the event I am going to give a live coding session. In particular I am going to show how to implement and secure BiPRO services using current Web Service standards such as JAX-WS, SAML, WS-Security and WS-SecureConversation. Attendees can see how the implementation works with JBoss Wildfly, Apache CXF and .NET. Moreover they can take a peek into the future of BiPRO standards in the area of security.
I hope to see you there!

How to Link WSDL-Services To BPMN-Processes

Since BPMN2.0 it is not only possible to design processes but to also execute them using a process engine. The process flow has a appropriate visual representation in the standard. But executable processes are mostly data driven. They interact with external services and exchange data with them. In addition to that processes maintain their own internal state. So a common requirement is to model the process internal state and connect to external services using the service data representation. BPMN is capable to include data definitions based on WSDL and XML Schemas, although the capabilities of the tools (that I know) to visualize data are somewhat limited.

In this blogpost I would like to show you how data looks like in BPMN and how a process can be linked in a standardized way to existing services based on WSDL and XSD.

The process is as simple as possible. The service is based on a BiPRO service description. The BiPRO is a standardization organisation in the German insurance market that standardizes processes and services at a technical and business level.

WSDLtoBPMN

Below you see a simplyfied version in plain BPMN (when you import the bpmn below you will only see the events and tasks).


<?xml version="1.0" encoding="UTF-8"?>
<definitions id="definitions" xmlns="http://www.omg.org/spec/BPMN/20100524/MODEL"
  xmlns:bpmn="http://schema.omg.org/spec/BPMN/2.0"
  xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI"  
  xmlns:dc="http://www.omg.org/spec/DD/20100524/DC"
  xmlns:di="http://www.omg.org/spec/DD/20100524/DI"  
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  targetNamespace="http://www.pleus.net/example"
  xmlns:tns="http://www.pleus.net/example"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
  xmlns:nachrichten="http://www.bipro.net/namespace/nachrichten"
  xmlns:bipro="http://www.bipro.net/namespace"
  xsi:schemaLocation="http://www.omg.org/spec/BPMN/20100524/MODEL http://bpmn.sourceforge.net/schemas/BPMN20.xsd">
  
  <!-- WSDL Import -->
  <import importType="http://schemas.xmlsoap.org/wsdl/"
          location="KompsitService_2.4.3.1.1.wsdl"
          namespace="http://www.bipro.net/namespace" />

   <!-- Item definition. Link to the external WSDL/XSD structure. structureRef: QName of input element -->
   <itemDefinition id="getQuoteRequestItem" structureRef="nachrichten:getQuote" />
   <itemDefinition id="getQuoteResponseItem" structureRef="nachrichten:getQuoteResponse" />

   <!-- Message definitions. Link to the item definition. Can be visualized by using DI -->
   <message id="getQuoteRequestMessage" itemRef="tns:getQuoteRequestItem" />
   <message id="getQuoteResponseMessage" itemRef="tns:getQuoteResponseItem" />

   <!-- Interface definition. implementationRef = QName of WSDL Port Type -->
   <interface name="Komposit Interface" implementationRef="bipro:KompositServicePortType">
      <!-- Operation: implementationRef = QName of WSDL Operation -->
      <operation id="getQuoteOperation" name="getQuote Operation" implementationRef="bipro:getQuote">
         <!-- Links to the message definitions --> 
         <inMessageRef>tns:getQuoteRequestMessage</inMessageRef>
         <outMessageRef>tns:getQuoteResponseMessage</outMessageRef>
      </operation>
   </interface>
 
  <process id="servicecall">
 
   <!-- Datasources and targets for the service call (process state). Can be visualized by using DI and dataObjectReferences -->
   <dataObject id="dataInputOfProcess" name="Input for webservice" itemSubjectRef="xs:string"/>
   <dataObject id="dataOutputOfProcess" name="Output for webservice" itemSubjectRef="xs:string"/>
  
   <!-- Process start -->
   <startEvent id="start" />
 
   <sequenceFlow id="flow1" sourceRef="start" targetRef="initScript" />
 
   <!-- Initialization of process data -->
   <scriptTask id="initScript" scriptFormat="groovy" name="Initialize process">
     <script>
       def temp = "2.4.3.1.1"
       execution.setVariable("dataInputOfProcess", temp)
     </script>
   </scriptTask>
 
   <sequenceFlow id="flow2" sourceRef="initScript" targetRef="webService" />
 
   <!-- Web Service call -->
   <serviceTask id="webService" name="Call getQuote" implementation="##WebService" operationRef="tns:getQuoteOperation">
       
       <!-- Defines the inputs and outputs and links to item definitions -->
       <ioSpecification>
          <dataInput itemSubjectRef="tns:getQuoteRequestItem" id="dataInputOfServiceTask" />
          <dataOutput itemSubjectRef="tns:getQuoteResponseItem" id="dataOutputOfServiceTask" />
          <inputSet>
             <dataInputRefs>dataInputOfServiceTask</dataInputRefs>
          </inputSet>
          <outputSet>
             <dataOutputRefs>dataOutputOfServiceTask</dataOutputRefs>
          </outputSet>
       </ioSpecification>
       
       <!-- Defines the mapping between process data and service input -->
       <dataInputAssociation>
          <sourceRef>dataInputOfProcess</sourceRef>
          <targetRef>dataInputOfServiceTask</targetRef>
          <assignment>
            <from>
             bpmn:getDataObject('dataInputOfProcess')
            </from>
            <to>
             bpmn:getDataInput('dataInputOfServiceTask')/BiPROVersion/
            </to>
          </assignment>
       </dataInputAssociation>

       <!-- Defines the mapping between process data and service output -->
       <dataOutputAssociation>
          <sourceRef>dataOutputOfServiceTask</sourceRef>
          <targetRef>dataOutputOfProcess</targetRef>
          <assignment>
            <from>
             bpmn:getDataOutput('dataOutputOfServiceTask')/BiPROVersion/
            </from>
            <to>
             bpmn:getDataObject('dataOutputOfProcess')
            </to>
          </assignment>
       </dataOutputAssociation>

   </serviceTask>
 
   <sequenceFlow id="flow3" sourceRef="webService" targetRef="end" />
   
   <!-- Process end -->
   <endEvent id="end" />
 
  </process>

Now let’s look at the example step-by-step.

1. Import the service: Line 16-18 imports the WSDL file that includes the types and messages used by the external service that we want to call from the process.

2. Define the items: Line 21-22 defines items that act as links to the types defined in the imported WSDL and XSD files.

3. Define the messages: Line 25-26 defines messages to be used in the interface definition that we see in the next step. Messages can be visualized by modeling tools provided that DI Information is present in the model.

4. Define the interface: The interface is the equivalent to the WSDL port type in BPMN. It is defined in line 29-36. So far we have itemDefinitions that link to the XSD-messages and an interface that links to the WSDL-port type. The inMessageRef and outMessageRef elements use the messages defined in step 3 to indirectly reference the itemDefinitions.

5. Define the process variables: The process maintains state. This state is defined in the form of dataObjects in line 41-42. Please note that the links to the external service are defined outside the process (which begins in line 38). The dataObjects are defined inside the process as they represent the data that is maintained by the respective process instances. By the way, when importing the process in a modeling tool, dataObjects are not visualized. To visualize dataObjects as a paper sheet, dataObjectReferences can be used. In this simple example we just use a string as input and output which transports a version information send to the BiPRO service and back. In a more complex senario this could be a type from an imported XSD.

6. Initialize the process: A simple BPMN script task (line 50-55) is used to initialize the dataObject dataInputOfProcess. It just sets the version to 2.4.3.1.1.

7. Link the serviceTask: The most complex part is the serviceTask (line 60-102). The operationRef attribute (line 60) links to the operation which is part of the interface definition (line 31). This is the web service operation to be called when the serviceTask is executed. The serviceTask comprises the elements ioSpecification (line 63-72), dataInputAssociation(line 75-86) and dataOutputAssociation (line 89-100). ioSpecification can be seen as a logical port that describes the service input and output from the perspective of the service. The itemSubjectRef attribute on the dataInput and dataOutput elements (line 64-65) link to the itemDefinitions (line 21-22) and as such to the data structures in the WSDL files. The id together with the inputSet and outputSet (line 66-71) define the connection points the serviceTask offers for sending and receiving data. dataInputAssociation and dataOutputAssociation map the dataObjects (process data) to the connection points or in other words to the request and response structures of the service (service data).

When the serviceTask webService is called, the process data from the dataObject dataInputOfProcess is copied to the web service request message nachrichten:getQuote/BIPROVersion. Then the service is called. After the service call finished, the version is copied from the response message nachrichten:getQuoteResponse/BIPROVersion to the dataObject dataOutputOfProcess.

This blogpost has shown how to link WSDL/XSD based services to BPMN processes in a standardized way.
Even if automated process execution is not in focus, it can be important to unambiguously link services to processes to create an integrated view of the entire process including its data.

Web Service Security on BiPRO Day

BiPRO At the upcoming BiPRO day on 11.June 2013
I am going to give a presentation introducing the most important standards in the area of web service security. The aim is to show the purpose of the standards and how they work together to create secure and interoperable message based web service solutions.

About BiPRO day:
“Einmal im Jahr treffen sich die Mitglieder des BiPRO e.V. sowie Interessierte aus der Versicherungs- und Finanzdienstleistungsbranche zum BiPRO-Tag. Dabei stehen aktuelle und zukünftige Themenfelder der Prozessoptimierung allgemein und des BiPRO e.V. im Speziellen im Vordergrund. Dazu zählen Vorträge und Präsentationen aus laufenden und bevorstehenden Projekten, die Vorstellung neuer Normen sowie Berichte über Norm-Implementierungen bei den Mitgliedsunternehmen des Vereins.”

Integrated Process Management with Open Source

If you ever tried to create an execution environment to automate business- or integration processes based on Open Source products, you know that this is not an easy task. Although Open Source products like Activiti or Apache Camel are of high quality, they do not run with production grade quality out-of-the-box. For serious usage scenarios typically a lot of work is required to integrate those products into a sound platform. This fact hinders companies to use those great products and turn to closed source alternatives from Oracle, Appian or Inubit, just to name a few.

Now there is an interesting alternative called oparo. oparo is an integrated process automation platform based on rock solid Open Source products. oparo is not limited to BPMN processes only. It rather focuses on the entire process spanning business, workflow, mediation and integration.

The platform does all the plumbing required to turn single products such as Activiti, Apache Camel, Apache ActiveMQ, Lucene/Solr, etc. into a platform that can be used out-of-the box. Even better, oparo is entirely ASF2.0 licensed (today and tomorrow) which offers broad usage options and does not involve any hidden costs for enterprise features.
oparo shields the process engineer (the guy who analyses and automates processes) as much as possible from low level technical tasks such as connecting and transforming Camel and Activiti message payloads. It offers a unified development approach for the process engineer to focus on business functionality instead of technical plumbing. Moreover it comprises additional valuable services such as process flow tracking, humantask integration or a registry. Due to oparos service binding approach, those services can be easily integrated in existing IT landscapes using almost any technology (e.g. .NET, JEE, HTML5/JS/CSS). The runtime is scalable (in terms of technology and licenses), the set up is automated and the whole platform is based on proven standards.

If that sounds promising, you can give it a try. You can find more information and a downloadable jumpstart distribution at oparo – the efficient process platform (German only)

Business Process Evolution and Versioning

(Automated) business processes evolve over time! And they usually evolve faster than IT systems do.
So how can business process changes be delivered to the users quickly?

Let’s look at an example:
Assume we have a process for vacation planning for the staff of a large company. Initially the process was automated based on the knowledge of the human resource department. After 2 months new insights require a process change. The process should be optimized to speed up the decison whether vacation is granted or not. The process has evolved and the changes have to be put in place as soon as possible. This is a common situation and actually one of the promises of business process management is: Deliver business value fast.

Sounds simple, but how can we deliver the changed process?

There are serveral options to put the changed process in place:

Option 1: Parallel
The changed process coexists with the initial one for a period of time. Existing process instances must continue with the inital process definition.

Example: Users of the process are gradually trained to use the changed process. Some departments can still use the initial process, some use the new one. The process is triggered by IT systems as well. Those systems should have a smooth upgrade path.

Action: Create a new version of the process and deploy it in parallel to the one already in place.

|--- Startable V1 -------->
|--- Instances V1 -------->
                 |--- Startable V2 --------->
                 |--- Instances V2  -------->

Option 2: Merge
The changed process replaces the initial one. Existing process instances must continue using the changed process definition.

Example: Law changes render invalid the initial process. As of now all processes, including already running instances, must run with the latest process definition.

Action: Create a new version of the process and migrate existing instances to the new process definition.

|--- Startable V1 ------|--- Startable V2 --------->
|--- Instances V1 ------|--- Instances V1 + V2 ---->

Option 3: Phase Out
The changed process replaces the initial one. Existing process instances must continue with the inital process definition.

Example: Process analysis caused the process to be optimized, so that it can be executed in less time. All users should immediately use the changed process.
To keep effort low, already running process instances should continue running with the inital process definition.

Action: Create a new version of the process and deploy it in addition to the one already in place. Prevent the initial process version to be started by disabling the start events.

|--- Startable V1 --------|
|--- Instances V1 --------------------|
                          |--- Startable V2 --------->
                          |--- Instances V2  -------->

Be aware of endpoints:
If process versions are provided in parallel like in scenario 1 and 3 and connected to technical endpoints, for instance filedrops or web services, those endpoints might collide. Changing the structure of an endpoint, for instance the message payload, might cause incompatibility as well. In those cases (which are likely to happen) the endpoints must be versioned. Alternatively a dispatching mechanism can be used to route messages to the appropriate process version.

As you can see versioning is am important concept for process evolution. Which strategy to use depends on the process and the particular business requirements. The options introduced in this blog post might help to take the right decision. Make sure your process platform supports the options you need.

Agile Process Management with Open Source

Are you interested to know how to combine process management, agility and Open Source software? Then the roadshow Agile Process Management with Open Source is for you. It is going to take place in several German cities during autumn 2012. I am going to present ways to achive efficiency in the area of process automation using proven Open Source technologies paired with agile approaches. In times where CIOs have to think twice before they spend IT budget, undoubtely an interesting topic to talk about. It have some interesting ideas to share and hope for inspiring discussions.

Scrum and Silverlight in Reinsurance

Today I would like to share a success story of a project I accompanied as Scrum Coach and Solution Architect from analysis to production.

Main success factors were:

  • Scrum (Agile Development)
  • Cross functional Team
  • Service-oriented Design (SOA)
  • Silverlight RIA

You can read more in the case study Modernization of business partner management.

Combining Groovy and XSLT for Data Transformation

In the blog post Beautiful Transformations with Groovy I described how easy it is to create data transformations with Groovy. But sometimes organisations invested massively in XSLT transformation and want to reuse their existing XSLT templates. Read on for an an example that shows how to do that.

Assume we want to transform the following XML file to HTML:


  
    Germany
    Fast and nice
  
  
    Spain
    Large and handy
  
  
    Italy
    Small and cheap
  

Lets further assume the result should look like this:

Does it make sense? I don’t know, but that’s not really important. 😉

We have the following XSLT template to perform the transformation:

All you need is a Groovy script like the one below to transform the xml file to html using the given xslt.

// Load xslt
def xslt= new File("template.xsl").getText()

// Create transformer
def transformer = TransformerFactory.newInstance().newTransformer(new StreamSource(new StringReader(xslt)))

// Load xml
def xml= new File("cars.xml").getText()

// Set output file
def html = new FileOutputStream("output.html")

// Perform transformation
transformer.transform(new StreamSource(new StringReader(xml)), new StreamResult(html))

This is self-explanatory, isn’t it?
As XSLT is somewhat limited when it comes to more complex transformations, it can be extended by custom processors which can we implemented in Java or Groovy. A custom processor in Groovy can be implemented like this:

public class AgeProcessor{
    public def process(ExpressionContext context,int n){
        return "Age: " + (2012 - n) + " years";
    }
}

The processor is hooked up to the XSLT using the expressions in line 3 and 28 of the above XSLT file.
The examples above show how to reuse existing XSLT in Groovy. Are you interested to see the same same transformation in pure Groovy? (sorry, I could not resist ;-))
Here is the code:

// Load xml
def cars = new XmlSlurper().parse(new File("cars.xml"))

// Set output file
def writer = new FileWriter("output.html")

// Perform transformation
def builder = new MarkupBuilder (writer);
builder.html(xmlns:"http://www.w3.org/1999/xhtml") {
    head {
        title "Cars collection"
    }
    body {
        h1("Cars")
        ul(){
            cars.car.each{car ->
                li(car.@name.toString() + "," + car.country + "," + car.description + ", Age: " + (2012 - car.@year.toInteger()) + " years")
            }
        }
    }
 }

It is shorter and self-contained. It is also more intuitive and therefore easier to maintain. But if you have the requirement to support XSLT in Groovy you now know how to do that.

Beautiful Transformations with Groovy

Data transformations are the daily business in ETL and ESB scenarios. If you have a service- or business process boundary it is very likely that data has to be transformed between different representation.
Typical integration scenarios have to deal with a huge amount of different formats (flat file, xml, csv, json, even binary). To make things even worse they tend to change from time to time. Therefore it is important to have tools that are flexible and easy to understand. I found Groovy to be very handy for the transformation job. Let’s have a look at a typical example. Assume we have a flat file in the following format:

Id Product     Amount
 1 Cheese      15
 2 Sausage     2
 3 Bread       3
 4 Butter      4

What we want is an xml representation like this:


  
    
    Cheese
    15
  
  
    
    Sausage
    2
  
  
    
    Bread
    3
  
  
    
    Butter
    4
  

The groovy script below does the job. The great thing is not only that is works but also that its easy to develop and maintain. A developer can see the source and target structure at the same time.

def reader = new StringReader(flatfile);
def writer = new StringWriter();
def builder = new groovy.xml.MarkupBuilder(writer)

builder.order(){
    reader.eachLine() {
        def line = it
        item(){  
            id(id:line.getAt(0..1).trim())
            name(line.getAt(3..14).trim())
            amount(line.substring(15).trim())
        }
    }
}

println writer.toString();

Thanks to closures an the MarkupBuilder class the transformation can be written in a fluent and intuitive way. Typical other transformations are as simple as that, making Groovy a first choice for data transformations.