Year: 2010

Activiti and the Open Source Reference Platform

Now that Activity published the first release (version 5.0) of their Open Source BPMN 2.0 engine and toolset, it was time to integrate it in our Open Source Reference Platform for SOA (OSRP).

Despite some initial weaknesses, such as BPMN error handling, Actitivi is very promising if you are looking for a lightweight, Open Source BPM engine. Especially the features in the areas IT/Business collaboration and human task  integration are worth a closer look. The modularity makes it easy to integrate the engine with other products.

Using the OSRP one can now compare the BPMN and BPEL approaches to BPM, understand how Activiti can be combined with other Open Source SOA technologies such as Apache ServiceMix ESB, and learn about the strenghts and limitations of the product.

Scrum in Practice at Fraunhofer Fokus

Im am going to give a presentation about Scrum at the Fraunhofer Focus in Berlin at 2.November 2010. The event is called Scrum in practice. In addition to the principles of Scrum I am going to show how Scrum is applied in real projects and how to deal with typical challenges.
I would be glad to see you there. You can register here.

Create Silverlight Apps with Notepad

Did you know that you can write simple Silverlight applications using just a text editor without using an IDE or compiler?
Although it is recommended to use silverlight.js to start your silverlight application in a robust way, you don’t even need that. Just create a html file with a text editor of your choice and embed the XAML code.

The absolute minimum is this:

This is a plain html file. The Silverlight plugin is hosted using the object tag. The XAML content is not stored in a xap file, but in a local script of type text/xaml. Inside this script block any XAML can be placed.
The script can also use embedded storyboards to automatically triggers animations.
A more complex example which creates a rotating rectangle with text and a logo is shown here:

The script comprises of the visual elements Border, Grid, TextBlock and Image and four Storyboards which control the animation.
Inline XAML can be used to create nice looking splashscreens or single file demos.
Try it. Just copy the file above to a html file and open it. Provided that you have the Silverlight plugin installed you can see the animation.

Screenshots:

Live demo

If you want to let Silverlight interact with Javacript, just add another script block at the top of the page:

In order to call it, add the event handler to the Grid in XAML like this:

<Grid x:Name="Frame" Background="SteelBlue" Cursor="Hand" MouseLeftButtonDown="nav"> 
...

That’s it!

Schema first contract design

One of the most important artifacts in Service-oriented Architecture (SOA) is the service contract. The contract defines how service producers and consumers interact with each other. Most people agree that a contract first approach is the preferred way of creating service contracts. WSDL based service contracts are very common. In many projects they are generated by the respective frameworks, such as Apache CXF or Microsoft WCF. This is dangerous, as the developer looses control over the contract. This can result in missing interoperability and limited reusabilty expecially in cross platform scenarios.
From what I see, most developers know how to work with their favourite IDE, but only a few know how to create a proper service contract from scratch. I think that is caused by the tool vendors, which in most cases have better support for code based approaches.
In this post I would like to show how to create a portable contract from scratch and share some best practices.

The domain model

The first thing to think about is the domain or entity model that the service should expose. The model should be modeled in XML Schema. XML Schema is standard, portable and supported on all platforms. In fact it is a good idea to use XML Schema as the single source of model information.

The following listing defines the entity Product:
 



  
    
      
        
          Name of the product
        
      
      
        
          Quantity on stock

The entity Product has two attributes name and quantity. Please note that the xml elements (line 10 and 15) are abbreviated. This helps to reduce the message size on the wire, expecially important in WANs. The downside is obviously reduced readability. That is not really a problem as the contract should be documented anyway using the standard XML Schema annotation elements as shown in line 11 to 13.
This is the result:

Wouldn’t it be nice to have a common base class in order to create a more generic service contract?
Luckily XML Schema allows that. Let’s create another schema BaseTypes.xsd in which we define the base types.



  
    
      
      
    
  

  
    
      
      
      
      
    
  

  
    
      
        
          Unique key
        
      
      
        
          Status of the entity

This base type Entity (line 24) defines common attributes for all entities. In XML Schema one can define enumerations as well (line 8 and 15). The enumeration EntityStatus defines the state of the entity. This is important in scenarios in which the entities are detached from the database. By marking the entities, it is easy to implement change tracking on the consumer side and just return the changed or deleted parts to the server. This saves network and processing time. EntityType is used later in our service interface to request different types of entities.
We can now import the base types into Types.xsd and change the Product type:





  
    
      
        
          
            
              Name of the product
            
          
          
            
              Quantity on stock

You can see the import in line 9. In line 12 and 13 the base type is added to the Product type.

This is the result:
Entity with base class

Now we have our model completely defined in XML Schema. By using imports and type inheritance one can create very modular entity models. Those models can be reused without any change by interface definitions such as WSDL and WADL for SOAP or REST services. Moreover the code models (Java, C#, etc.) can be fully generated from the schema. Every good framework has a generator tool on board. For instance Apache Axis/wsdl2java, Apache CXF/wsd2java, Silverlight/slsvcutil, .NET/svcutil, etc. The schema can be stored in a service repository in order to increase reusability amongst the organisation.

The messages
Every service operation receives a message and returns a message to the sender. Those messages should also be defined using XML Schema. The following snippet shows the message definitions. The fault messages are ommitted for the sake of simplification.


    
      
    
  

  
    
      
    
  

    
        
            This represents a request to read business entities
        
        
            
                
                    
                        Type of the entities to read
                    
                
                
                    
                        Unique keys of the entities to read
                    
                
            
        
    

    
        
            This represents the read response
        
        
            
                
                    
                        List of entities

Those are our messages:

ReadEntitiesRequest and ReadEntitiesResponse are bulk operations that allow to read multiple entities with one call. As you can see ReadEntitiesResponse does not return Products but Entities. Do you remember? This is our base class type for all entities. By using the base class, the message can return arbitrary entity types in the same message. This is an example for a strongly typed but yet extensible interface.

The schemas can be reused in WSDL files:

The type information and message definitions are completely externalized to individual schemas. As a result the WSDL is small. The imported types (line 4) are used to define the WSDL messages (line 7 and 10). A WADL file would look like this:



  
    
    
  

  
    
      
        

            
              
                This is the product id

The types are imported (line 8 and 9) and used to define the return message (25).
By using the respective tool, such as wsdl2java or svcutil, we can completely generate the model and messages for the client and server on JEE, .NET and other platforms.

Summary
As you can see it is not too difficult to create schema based service contracts in a platform agnostic way. If you do, you will be rewarded with a great level of control, portability and reusability. Powerful features of XML Schema, such as inheritance, allow to create robust and extensible service contracts, which are the basis for interoperability. And if you want, you can even use an XML editor. 😉

Three challenges of BPMN 2.0

There are three major challenges in using BPMN 2.0 as a holistic (360°) approach to Business Process Management (BPM):

  1. Semantic Alignment
    BPMN like all other high-level process languages is context agnostic. This is good, as it allows a wide adoption throughout the industry. On the other hand it means that it does not explicitly express concepts found in business contexts, such as customer, accounts, discounts etc. As people from business and IT often have a very different view on certain business aspects, it is essential that they share the same semantic context in order to understand each other. If not, a process designed by a business person will not be understandable by an IT person and vice versa. In case BPMN is applied in combination with an outsourcing approach things get even worse.
  2. Level of Details
    BPMN2.0 is designed for automatic execution on a process engine. The goal is to have one process model from analysis to execution. But people from business and IT require different very levels of detail in their process descriptions. This is why a processes created by business people are usually not detailed enough for IT people. IT processes are usually full of technical information required for automation, rendering the process almost useless for business people.
  3. Portability
    The BPMN standard introduces several conformance levels (modeling, execution, BPEL, choreography). This is good as it fosters reusability of BPMN diagrams amongst different tools. At the same time the standard explicitly allows model extensions, “to satisfy a specific need, such as the unique requirements of a vertical domain” (quote from the spec). Thus, to avoid vendor lock-in, one at least has to be very careful in choosing the right model elements.

BPMN 2.0 should not be applied naively. Unfortunately this is often happens, especially when the standards are young. To be successful with BPMN 2.0 one has to find the right mixture of standards, design principles and methods.
For instance the semantic aligment problem can be mitigated by applying Scrum for analysis and process development. A multi-layer process design can help to address the level of detail problem. And choosing the the right product can increase portability.
This is nothing that comes for free by just using BPMN 2.0. It is rather something that needs to be actively managed by experienced engineers.

Definition of Done – Never without

One of the keys to success in agile projects is a proper Definition of Done (DoD). Only if everybody knows what has to be produced in order to complete a sprint, the goal can be achieved. 

The ideal outcome of a sprint is a product increment that is potentially shippable. To achieve that, all necessary actions to create a high quality product, such as writing documentation and thorough testing, have to be carried out within a sprint.  

In order to be really sure whether a an artifact is done, acceptance criteria are needed. Otherwise “done” would not be measurable. The criteria depends on the produced artifact. 

Software-Artifacts 

Software products usually comprise the following: 

  • Database structures (tables, stored procedures, triggers, …)
  • Application (user interfaces, Services, …)
  • Interfaces to external systems
  • Data (probably migrated from older data, …)
  • User documentation (online help, …)
  • Installer

    • Acceptance criteria 

    In order to prove that the above artifacts are really done, they need to be tested. Because the amount of test grows for each sprint, there is no way around automated regression testing. Therefore a continuous build,test and integration system, such as Team Foundation Server or Hudson, is essential for agile projects. The following tasks should be automated (in the brackets you can see an example of acceptance criteria for each task).

    • Unit testing  (error ratio maximum=10%, code coverage minimum=60%)
    • Load + Performance testing (concurrent users=20)
    • User acceptance testing, UI tests (error ratio maximum = 15%)
    • Integration (successful installation and availability)
    • Code quality checks (warnings maximum = 20)

    Some tests, such as UI tests, might be difficult to automate. But if you do it, you team will be rewarded with a highly accepted software product at the end of each sprint. As you can see the acceptance criteria is not 0% errors or 100% coverage, because this would not be realistic.

    Concept-Artifacts 

    Some projects develop their concepts using Scrum as well. Something that I would encourage to do. Although in Scrum the amount of written documents is greatly reduced, concepts are often helpful and required. For instance to refine coarse grained user stories from the product backlog or if the implementation of an idea can not be realised immediately. Concepts can be written in many ways as long as they clearly describe the idea down to a level that is sufficient for the implementation. For instance text documents, wiki pages, prototypes or design sketches. 

    Acceptance criteria 

    How can a concept be defined as done in terms of a DoD? As always! Conduct a review with people from within the team or other stakeholders from within the organization. When a concept is successfully reviewed, it is done.

    Summary

    Having a proper Definition of Done which clearly lists the required artifacts and acceptance critera is essential for successful Scrum projects. It creates a common understanding of what “done” actually means and is a key artifact to deliver high quality software in agile projects.

    Forms Over Data vs. Domain Driven Design

    A common challenge in Service-oriented architecture (SOA) is the design of service contracts. Beside many variations we see two basic types of contracts.

    Forms over data (FOD) is an approach in which the forms in the user interface directly reflects the data structures in a database. The service-layer, if present, primarily offers create, read, update and delete (CRUD) operations for the individual database entities.

    FOD is fostered by technologies such as Microsoft Access, Oracle Forms, .NET RIA Services. In JEE it is recognized as an anti pattern addressed by design (anti-)patterns such as Face Off or Session Facade.
    FOD is tempting because large parts of the service-layer can be generated from an existing data model. By using the right tools, developing an FOD-application can be quick and easy.

    Domain driven design (DDD) is an approach that utilizes a service-layer whose data objects are inspired by the business domain. The database structures are hidden behind a service-layer and not known by the service consumer, for instance the user interface. The domain model is usually modeled using XML Schema. DDD is typically used in SOA environments.

    DDD-Pros

    1. Portable domain model – If modeled in XML-Schema the domain object can be generated on JEE, .NET and other platforms.
    2. Business alignment – A business oriented domain model can be better understood by business users, thus increasing agility.
    3. Higher degree of reusability – Services representing business concepts are more likely to be reused, for instance in BPM environments.
    4. Decoupling of user interface and database – Data structures can be changed (up to a certain degree, of course) without affecting the clients. Changes can be handled in the service layer.
    5. Maintenance – Proper domain driven services are designed to last. Paired with a versioning approach service maintenance can be simplified.

    FOD-Pros

    1. Development effort – FOD is often supported out-of-the-box in many products, so that no using it would require more knowledge and development effort.

    Although there are always areas in which one or the other approach makes more sense, you can clearly see my preference. I think forms over data can be particularly useful for small applications and prototypes. Especially with existing database models. For large enterprise applications one should prefer a domain driven service design in combination with a sound SOA approach.