Building a mercurial external hook to update Target Process

Whilst learning the new Mercurial Distributed Version Control system ( DVCS ) I thought it would be useful exercise to learn about hooks by writing one. There are two types of hooks you can use, either an in process hook which executes with the hg process or an external hook which is executed as a separate process by hg, the core mercurial process. I wanted to develop a hook which integrates with Target Process to update a specific entity with the changeset comment if the comment referred to it e.g. a comment such as

TP#23000 Resolved issue with print manager crashing on XP

would update bug 23000 with this comment. The mercurial definitive guide recommends using an external hook if your not familiar with python and performance isn't critical. Given my python experience is non existent I created a NET console applicaiton to perform the work. The application interacts with the hg process to obtain changeset information and uses WSE 3.0 web service calls to the target process API to retrieve and update the entities. The important parts of the process are as follows

1. Environment variables passed to the hook process define useful information such as the path to the hg process. However also the parameters to the hook are passed by using a variable with the suffix HG_ followed by the variable name e.g. HG_NODE contains the changeset identifier.
2. The changeset identifier passed to the hook as the env variable HG_NODE is used to execute the hg process with the log command to extract the description of the changeset. This uses the -template option to control the specific output of the log command. When calling the hg process the console output is redirected to a string so the contents can be captured.
3. The captured changeset description is then parsed to determine whether it matches a TP entity using a regular expression. If the changeset references a TP entity the TP web service methods are called to obtain the entity and update it with the comment.

Although this doesn't provide a feature rich hook similar to the packaged bugzilla hook, it does provide you few useful features and enough information to build your own external hook. You may wish to change this implementation i.e.

1. Only bugs are currently supported however it shouldn't be hard to add support for task / story ids.
2. The bug status entity is currently obtained using a fixed process name "All Practises" however this could be obtained from the entities related projects related process.
3. Customising the text added to the TP bug through some kind of template.
4. Mapping TP users to mercurial users. Currently the user credentials are taken from the config file.

The hook must be installed by adding the following line to mercurial.ini

[hooks]

commit.TPMercurialHook = TPMercurialHook.exe

Target Process actually supports a plug in model for a number of different SC repositories e.g. one for TFS, which work by continually running and polling the repository for changes. Mercurial hooks work by intercepting events in the repository immediately whereas the plug in model polls for changes.

The code for the hook can be downloaded here

Enterprise Windows Workflow Foundation Development Part II

Over the last week I’ve changed from using the manual thread scheduling service to the default CLR thread pool based one. Some of the techniques I mentioned before have been updated in this post and also a few new items discussed

• Exception handling
• Workflow performance
• Workflow custom tracking service for state mapping
• Unit testing
• WF design

Exception handling

Using the default thread scheduling service and checking a LastExceptionRaised property from a local service to check exceptions simply will not work now. As the WF execution is performed asyncronously in all cases the property will simply be null when checked straight after calling a WF local service method. Only some time later when the WF thread is scheduled will this property be set.

The best approach I've found to handle this case is to firstly log all exceptions passed to your local service property setter when the WF fault handler kicks in. You can use the enterprise libraries HandleException method or something similar depending on what logging framework you use. This ensures the rich detailed exception information, including all inner exceptions and their stack traces, is captured and logged.

You should also manage a collection of exceptions within the local service to ensure you don’t miss any e.g. if two WF threads raise exceptions at a similar time. To handle the exceptions you will to create a timer which periodically checks whether any exceptions exist in the collection and simply throws the first exception in the list. This can then be caught and handled/ logged / rethrown in your application depending on the exception handling policies defined. You need to use a timer to check the exception and re throw otherwise the workflow would terminate if an exception re thrown on the WF thread.

In terms of "handling" unexpected exceptions, our application uses the AppDomain UnhandledException filter which processes the exception and uses the “Unhandled Exception” enterprise library policy to define the handling. By default we configure this policy to log the exception to the event log and presents the user with a friendly dialog that a serious issue has occurred ( well about as friendly as an unexpected dialog type window can appear )

Workflow performance

In some scenarios you may have many workflows which continually load / unload e.g. if they are triggered by a delay activity. If the delay activity is relatively short e.g. few minutes and you have a few hundred WF's you may find the WF runtime spending most of it's time loading / unloading WF's. There are a few approaches you could take

1. Develop a customised persistence service and override the UnloadOnIdle method e.g. you might customised SqlWorkflowPersistence. In there you could put some custom logic to only unload some workflows. E.g. you might not unload workflows which are in a state with logic driven by a DelayActivity.
2. Don’t set UnloadOnIdle to true in the constructor of the persistence service and in the WorkflowIdled event you can decide if a workflow should be unloaded using similar rules as in 1.

In relation to diagnosing general WF performance issues I highly recommend reading Performance Characteristics of Windows Workflow Foundation. There are a variety of useful WF performance counters you can use to inspect your running WF host to avoid second guessing where any bottlenecks might be. As always good detailed analysis is essential so you really are tuning the least performant parts of your app and not those you most like to tinker with :)

Workflow custom tracking service for state mapping

The original approach I mentioned works by mapping the current state to the workflow state in the workflow idle event, however this has the following implications.

1. The StateMachineWorkflowInstance class can be expensive to construct every time your workflow idles.
2. The change of status in the state machine may not be immediately reflected in your entities mapped state. i.e. some time passes between entering a specific state and the workflow idling e.g. if performing some work in a StateInitializationActivity within that state the workflow will not idle until the StateInitializationActiviy has completed.

Matt Milner wrote an excellent article on tracking services and presents one which tracks state changes in a state machine activity. As soon as a new state activity starts executing the tracking service is called. I’ve adapted this service so it fires an event on a state change which a local service responds to by mapping the current state to an enumerated status value. The status of the WF state machine and status in the entities record are now immediately in sync with each other

Unit testing

There are many excellent articles on unit test approaches to WF i.e. Matt wrote one and Cliffard has also produced an excellent lab on applying TDD with WF. Two important points when creating tests to add to these articles.

1. Always try and refactor logic within activities / workflow into separate components and test them in isolation first. Keep the minimal amount of code in your activity. It’s fine for an activity to simply provide a simple façade around an existing service. If you adopt a TDD approach you would actually find yourself more likely to write those services first, and thus make the activities easier to test using mock objects.
2. Use IOC mechanisms such as dependency injection so you can mock most of the objects your workflow / activities use. This means your tests can focus as much as possible on testing just the activity or workflow class and not many other dependent classes.

In testing a state machine I’d like to test everything about the model I’ve created i.e. the start state / completed states, possibles transitions from each state and assert any rules embedded within the workflow are evaluated, and ensure ALL activities end up being executed and covered. This is very important when you have many IfElse branch activities in your workflow based on rules. The last thing you want to do is change a condition within a branch only to find the branches beneath it can never be reached :(

WF Design

One thing I’ve found out over the last month working on WF is the vast number of approaches there are to implement any given design. WF is highly extensible, both in the new services you can plug in and existing services that can be customised using standard mechanisms such as inheritance / events / overriding methods.

What ever approach you choose it’s important you try and fit within the WF framework as much as possible. It’s up to you how little or much of your workflow logic you decide to implement in WF. Even if you don’t think your design is ideal and doesn’t feel right I would encourage you to simply start and be prepared to refactor as you learn more about WF.

Enterprise Windows Workflow Foundation Development

It's nearly a year ago since I blogged about some of the really cool features of WF and it's only recently I've had an opportunity to apply this technology in my day job. Although other projects I've worked on until now could have used WF equally well, they were simply too large and complex to refactor WF in easily. However they would have been a great candiate to use WF from the start as they use complex hierarchical state machines for which the logic is unfortunately littered throughout the application.

My previous WF projects were simply for my own learning ( aiming for an MCP qualification eventually ) and although I covered a lot of different areas of WF, it's only through developing a production application I've come across areas that needed to be reviewed further. e.g.

Workflow authoring mode - Out of the three options which is the most appropriate

Unhandled exceptions in the workflow . Exceptions need to be logged by the host and in some cases we want to avoid termination of the workflow

Querying workflow state - It is often not feasible to simply load every workflow to query it's state.

Dependency injection - How can types be resolved

Dynamic updates - Useful for simple customisation

Associating business objects with workflow instances

Runtime scheduling service - Is the default service always appropriate

Workflow runtime and instance wrapper classes

Workflow authoring mode

Although there are three authoring modes to choose from code, code with separation, and no code, visual studio only lets you choose you the first two. All the guidance on this mentions you should evaluate how much flexibility the end user requires to change activities and workflows. Based on this advice though I found it hard to choose an appropriate model. On the one hand we should keep our applications as simple as possible right now based on our requirments, however on the other hand we need to design to make future changes easier :).

I actually think the no code authoring mode is probably the preferred route as it encourages developing good practises i.e. building your activities upfront and then building your workflow around them being clear to separate the rules and the binding of properties between activities as separate tasks. However given the poor tool support in VS 2008 and also 2010 for no code workflows I find it hard to recommend. If you really can't perceive a need for this level of flexibility i.e. hosting the designer and allowing users to change the workflow, then it maybe overkill.

Note if you do choose a code only route and want to move to code separation it is possible with a bit of work. Simply copy all activities within the designer and then paste into a workflow with code separation. This will created the XOML and then you can add all code activities in as required.

For the project I worked on I applied the code separation model which seems to be no more complex to manage than the code only model, but at least provides some flexibility. The workflow and rules could be modified relatively easily by hosting the designers and new workflow types compiled as required.

Exception handling

It's inenvitable your application will have some bugs no matter how hard you try and in this case it's important that the application ( or more specifically a data store ) maintains it's integrity so users can continue using the system. It's also important to capture these exceptions and log them appropraite e.g. we use enterprise library 4.0 which provides rich support to log exceptions.

The default behaviour for unhandled exceptions is to terminate the workflow which also has the effect of removing the workflow from the persistence store. However if your using a state machine which is currently in one of states you may not wish to terminate the workflow abruptly. If the workflow ends you would have to create a new workflow and start the state machine in the same state perhaps and also ensure workflow properties were set correctly.

In our project we managed this by setting up fault handlers in nearly every state which could throw an exception. ( it was acceptable for the first state state to throw an exception and terminate the workflow ). In this fault handler we always handle the generic Exception type and when the exception is raised a local service method is called on the host application which passes the Exception object. The host application can then perform any necessary action e.g. transitioning to a failed state , logging the exception and in most cases presenting the user with an exception has occurred type window. Now we should have good information to diagnose this exception further and the state of the workflow is maintained.

Querying workflow state

There are many mechanisms to query the workflow state e.g. from StateMachineWorkflowInstance there are properties such as CurrentState / PossibleStateTransitions which provide you with a "picture" of the workflow state. However this does require the workflow to be loaded to obtain this state. For many applications it wouldn't be feasible to load all workflow instances just to find out which workflows are in a specific state. It also doesn't lend itself well to standard querying techniques used on a data store e.g. select all invoices from a table which are in state x

Most application entities typically contain a state column which usually maps to an enumerated value defined in the application e.g. InvoiceState with values New, Awaiting Approval / Approved. It would be nice if we could still use this state column to filter out the records were interested in, but at the same time use the workflow instance to drive the workflow, determining possible state transitions / events that can be raised e.t.c. My first thought was to use a custom tracking service to track the changes to the state machine and report them back to the host application which could then update the state column of the related entity to reflect the current state. This way both the workflow instance and state column would always be in sync.

However after creating a custom tracking service to do this I found the excellent article Matt has written on tracking services which also builds a similar service for tracking state machines, well at least I'd learnt tracking services well :) . I then found a much simpler solution which works well and doesn't require the use of tracking services. In the workflow idle event the current state can be checked from the StateMachineWorkflowInstance wrapper class and then the related entitiy updated. This works well because the workflow always idles between transitioning to new states.

Dependency Injection

Dependency injection / service locator which are implementations of the IOC pattern are well known good practises however it's not clear how you can inject instances into workflow types / activities. I did raise this question here and got some useful links. I haven't followed up the related links in detail however I did manage to find a relatively simple solution by using the services based infrastructure which WF provides.

The host application which creates workflow instances creates the types which need to be resolved by dependency injection. Those objects are then passed to AddService from the WorkflowRuntime instance. You then need to access the interfaces through the IServiceProvider interface or the ActivityExecutionContext instance calling GetService . If you only need to access the interfaces in the Activity.Execute method then you can simply use the ActivityExecutionContext instance directly within Execute. Otherwise override the method OnActivityExecutionContextLoad and store away the interface references. If you do this however be sure to mark the fields in the activity / workflow with the NonSerialized attributes to prevent them also being serialised when the workflow is persisted.

Scheduling services

The default scheduling service uses a thread pool to schedule workflows to be executed as required. However given the workflow types in our project AND the host application access the same business object underling the workflow, this does introduce potential concurrency issues. The manual scheduling service provides a model which is simpler to manage and would be more appropriate here.

You have to specifically schedule the ManualWorkflowSchedulerService to run a workflow using the RunWorkflow method. The host application simply does this after raising any events into the workflow through the local service. However some parts of the state machine activities require work to be continually performed in the background waiting for a specific rule evaluate to true and using a DelayActivity to sleep the workflow. To handle this a simple WPF DispatcherTimer can be used with DispatcherPriority.ApplicationIdle priority to execute any workflows which have expired timers. The timer can be setup using the following code

DispatcherTimer workflowPollingTimer = new DispatcherTimer(new TimeSpan(0, 0, 0, 5)),DispatcherPriority.ApplicationIdle,new EventHandler(workflowPollingTimer_Tick), Dispatcher.CurrentDispatcher);workflowPollingTimer.Tick += new EventHandler(workflowPollingTimer_Tick);workflowPollingTimer.Start();

Within the timer the workflows will be executed when their timers expire

ManualWorkflowSchedulerService scheduler = workflowManager.WorkflowRuntime.GetService(<>);

foreach(SqlCePersistenceWorkflowInstanceDescription workflowDesc in persistence.GetAllWorkflows()){

if (workflowDesc.NextTimerExpiration <>

{

scheduler.RunWorkflow(workflowDesc.WorkflowInstanceId);

}

Dynamic updates for customisation

Although dynamically updating a workflow is often used for versioning reasons e.g. migrating old to new workflow versions, or making ad hoc changes to specific workflows at runtime it can often be used to provide flexible customisation within a workflow. Although rules can be managed directly within the workflow, they can also be managed within the application e.g. preferences for document approval. You may find a need where a number of different activities ( based on runtime conditions ) need to be executed and using If / Else type branches within the workflow can quickly make your workflow very complex and hard to maintain.

It's very simple to use a place holder activity within your workflow and then dynamically add activities to add at runtime based on application defined rules. Perhaps you have a number of different approval strategies which need to be executed, these could be added at the point of approval in the workflow. This does provide another option you can use to get the customisation you require without having to write designer hosting support in your applcation :)

Workflow runtime and instance wrapper classes

As you develop the host application you will probably find yourself writing proxies over the WorkflowInstance and WorkflowRuntime CLR classes to provide additional services when executing workflows. They can also make it easier to execute those workflows e.g. defining a ManualResetEvent within the workflow wrapper and then setting this when the workflow completed event fires so you can execute workflows and wait until they've completed. Bruce Bukovics has developed a good set of wrapper classes you can use as a base and adopt as required for your own requirements.

Associating business objects with workflow instances

One of the requirements in a workflow applications is to load a specific business object and then raise a workflow event. To do this you need to maintain a link between the business object and it's associated WF workflow instance. You can add the business object link directly into the workflow or add the workflow instance link into the business object or even both.

If you do both so the business object contains a WorkflowId of type Guid in it's entity table, and the workflow itself contains a reference to the identifier of the business object. In this way the business object can be loaded by the client application, when a workflow event needs to be raised into the business object the correct workflow instance can be retrieved and called on. Having the business object reference directly in the workflow also allows the workflow to access the object to query any data within it or to make modifications and save to the data store.

You may find it preferable to you use an object identifier rather than the object reference otherwise for long running workflows you may find yourself using stale data e.g. if the host application modifies the business object and workflow is using a stale copy. Using an object identifier or even using an Identity pattern ( link to Martn fowlers identify pattern reference ) means you can load the object as and when required in the workflow instance.

Conclusion

I hope I've shown you a few techniques you can use to make your life a little simpler when developing WF applications. Both Maurice de Beijer and Bruce Bukovics have provided me with useful feedback when tackling WF design issues. Bruce has written an excellent book on WF which is even worth getting just for the additional chapters covering NET 3.5 changes. Maurice also teaches a highly respected course on WF.

User interface automated testing

If you’ve read any of my previous posts you’ll know I’m very keen on unit testing and continuous integration. Up until now we’ve managed to automate the testing of pretty much everything i.e. XSL / components covering many different technologies / web applications both ASP and ASP.NET / and also test and integrate database scripts. One thing I’ve always struggled on though is effective test automation of thick client GUI applications and always managed to push this task aside.

Recently whilst fixing a defect within a server component, I introduced another defect in a thick client user interface application which was using this component. At the time I added a few new unit tests for the component changes and checked all 3544 of our existing unit tests passed. When I got the green light on cruise control I was very happy with my changes and completely oblivious to the fact I’d just broken the GUI application :( When I discovered this a few weeks later I was determined to create a user interface unit test for this application– this was the kick start I needed.

John Robbins is usually more renowned for his excellent debugging articles however here he writes about the new features in NET framework 3.0 which support UI automation. This sounded like an ideal technology to drive the UI and test features. I’d strongly encourage you to read this article and download the sample source code.

In the article Johns written a wrapper over the UIAutomation object model which makes programmatic control of UI elements much simpler than using the UIAutomation API directly. However the wrapper assembly is still incomplete and only covers a few of the common controls. If you use this you will quickly find yourself needing to add new control types however this is relatively simple to do. To select an item in a combo box I created a class called UIComboBox which supports simple methods such as SelectItem. As you can see calling SelectItem sure beats writing this every time you need to select a combo box item in your client code.

public void SelectItem(Int32 comboBoxItemIndex)
{
AutomationElementCollection comboBoxItems = this.RootElement.FindAll(TreeScope.Descendants,
new PropertyCondition(AutomationElement.IsSelectionItemPatternAvailableProperty, true));
AutomationElement itemToSelect = comboBoxItems[comboBoxItemIndex];
object pattern = null;
if (itemToSelect.TryGetCurrentPattern(SelectionItemPattern.Pattern, out pattern))
{
((SelectionItemPattern)pattern).Select();
}
}

One wrapper control I certainly missed was one associated to the NET DataGrid. I was actually automating a NET 1.1 application and although I found this solution , it only works on a NET 2.0 DataGridView. Still if you are using this you can build a very nice API with methods such as SetAt and GetAt to retrieve / set cell content within the grid. The only solution I could find to fill the NET 1.1 grid was to use the SendKeys method to send all the required keystrokes to navigate through the grid.

All of the UIAutomation and the corresponding wrapper methods in the article download use methods which take a UIAutomation identifier. To create tests quickly it’s vital to use a tool such as UISpy to obtain these identifiers. All you need to do is put UISpy in “hovering mode” and then simply hold down the Ctrl key whilst hovering over a window in the application being automated. The window is highlighted in red and UISpy then reports all the details in the right hand pane, most importantly including the AutomationId.

Once you have the automation identifiers it’s very straightforward to write code to access any GUI elements e.g.

UIEditControl invoiceNoTextBox = appWindow.FindDescendant("invoiceNumberTextBox", ControlType.Edit) as UIEditControl;
invoiceNoTextBox.Value = "SLOG_Invoice_1";

It is usually far more effective to use FindDescendant to find controls rather than FindChild given this searches all children recursively and will eventually find the control if it exists :)

When building unit tests you’ll usually find you automate the GUI to a point where you need to test the expected output. This could either be

1. Verifying GUI control values have been set correctly by asserting the contents of the controls. E.g. you could take the user through a task such as create a new invoice and then assert the contents of a screen which shows the invoice detail ( perhaps a wizard summary screen ).
2. Verifying a specific action has occurred by invoking business objects and then asserting the values from the business object. E.g. you could automate the GUI task to create a new invoice and then when this is complete verify the invoice exists in the data store and the values e.g. invoice no, match the values you entered in the GUI.

User interface testing is just as important as any other component testing that I do hope more people embrace this and start to automate the testing of a good portion of their user interfaces. As developers we cannot and should not rely on QA to find the defects, at least certainly not the obvious ones. With a few simple unit tests we can make great strides to catch any serious defects before QA and then the customer see the product.

Design - Modelling user interfaces in UML

In my previous post I focused on how to model a web application and it's component parts e.g. server and client side pages into UML elements. In this post I'll describe an EA add in I built to help reverse engineer UML models from user interface elements such as dialogs and there component parts e.g. check boxes, text boxes, button. Although the add in works only on thick client applications, the approach could also be used for web applications.

Most of the screens in your application, there contents and flow to other screen are typically driven by the use cases you write. It helps to model the screens using UML notation so you can describe each element and model how each element links or depends on other elements. Once the elements are in a UML model you could also show interaction e.g. an actor clicking a button which instigates a business object function which then builds a list box and populates a text box, and then enables another button.

Enterprise Architect is an excellent low priced modelling package, however whilst it provides many useful features and powerful forward and reverse engineering options ( it even includes a debugger which can auto generate sequence diagram based on a run through code ), it also provides great extensibility through the add in interface. In the previous post I discussed an addin I wrote to reverse engineer the UML model from various web page files, however this addin works differently by using a spy++ type capture pointer. The user only needs to hover the pointer over a window they need to capture, the addin then creates a UML model of the dialog using all the component parts of the window, with the exact same size and position as they appear on screen.

Thanks to Mark Belles for writing the windows capture code which provides the hover utility to obtain a window handle. From this it was a simple matter of extracting all the windows information using the Win32 API. The add in also should also provide a good template for anyone else writing one and contains supporting classes that help wrap the unmanaged EA interfaces.

The screen below shows the EA options dialog





When you start the add in through EA you are presented with the following dialog



The UML diagram below then gets generated.

Although the output isn't perfect it's a lot more productive than creating all yours screens manually. Not a job Id want to do often :)

The full source code to the add in written in C# is available in the EA WIKI, I hope you find this useful.

My first open source project - Web modelling UML add-in for Enterprise Architect

I recently created my first open source project called EA WebModeller Add In which is an add in for the Enterprise Architect ( EA) UML modelling tool. The add in provides a much filled gap to the product to enable users to reverse engineer web projects into UML artifacts. Specifically using the web extensions for UML which Jim Conallen produced. The web extensions provide a standardised way of modelling the components which make up a web application.

A seemingly simple concept, which most would just think of as one web page associated with another quickly becomes very complex and is proof of why Jim could write a substantial book on this subject and still leave room for more. Classes are used with a variety of different UML stereotypes to define the various components which combine to produce a web page e.g. server side page which builds a “client side page” which aggregates a number of “forms”. Various stereotypes of association are also defined to support different associations between the components e.g. one server side page redirects to another server side page or a client side pages links to another client side page ( through hyperlinks perhaps ). Components contained within pages both on the server and client are also modelled appropriately.

The example web project included with the download available on source forge contains a number of scenarios e.g. file 1.asp includes files include1.asp and include2.asp. Both included files contain server side functions. 1.asp also contains a redirect call to 2.asp, an embedded class, some client side functions ( in java script ) and an embedded form. The 1.asp server side page and it’s associated elements are shown in the diagram below

This is produced by the following settings in the add in

The add in is built in an extensible way to allow new script parsing languages to be easily integrated within the framework e.g. a base ScriptParserBase class contains JavaScriptParser and VBScriptParser derived classes. Similarly the ServerPage class which models the UML server side page contains derived classes ActiveServerPage (ASP ) JavaServerPage (JSP) and DotNetActiveServerPage ( ASP.NET ) for each of the web application technologies. Note only the ActiveServerPage class is fully supported for ASP pages, I’m hoping the open source community will develop the JSP and ASP.NET technologies :)

The add in uses mostly regular expressions to parse the script source into it’s appropriate components. They range from very simple expressions such as #include\s+(?virtualfile)=""(?.*) which parses include directives within ASP pages. The PathType and IncludedFile are named captured groups which here capture the type of file reference e.g. virtual or absolute file location. To relatively compex expressions such as “^\s*function\s*(?\w+)\((?.*)\)\s*(?(?:(?\{)(?\})(?(LeftBrace)[^\{\}]*.*))*)” which use “balance grouping” to support recurision within the expression. This power is required here to match all function definitions because we can’t simply match an opening and closing brace as we may find the first closing brace is not closing the function, it’s in fact closing the first for loop defined within the function.

To parse the client side page the internet explorer active x control is used to load the htm file produced when navigating to the web page. By using the IE control, the parts of the web page such as embedded forms and controls within the form can be found and added to the UML components. I actually wrote a web application testing framework around this control however recently I’ve started to use the WatIN framework which is more feature rich.

Given the main web application our team develops is written in ASP with many hundreds of pages, the productivity boost from using the add in is huge. Although we couldn’t possibly try and reverse engineer the whole web application in one go, being able to selectively single out and reverse engineer a few web pages when adding new features or doing some refactoring is of great help. It clearly shows the effected pages and where the new logic is best placed and perhaps most importantly helps to show the impact of any changes on existing functionality.

Window Workflow Foundation - A new way to develop workflow

I’ve recently reviewed the new workflow framework called Windows Workflow Foundation ( WWF ) which is part of the .NET framework 3.0 runtime. Using the framework successfully within our applications will require a completely different thought process on traditional methods e.g. where we typically put most of the workflow behaviour directly within the code.

I started to develop a generic workflow framework implementation of core business object services a few years back, however even with considerable effort, the implementation lacks many of the features provided by WWF. Given that most business applications go some way to make them flexible by developing a generic workflow framework, use of WWF should allow most teams to concentrate more on the back end business logic rather than trying to do what WWF does very well.

I would highly recommend reading Bruce Bukovics excellent WWF book http://www.apress.com/book/bookDisplay.html?bID=10213 which provides coverage of all the important features in WWF very clearly.

I have summarised the main points of WWF below

General Points

1. Multiple types of workflow – It handles both sequential and state machine workflows, although typically P2P / Banking applications which Bottomline develops use state machine workflows because the processes don't define a fixed flow of control within the workflow.
2. Scalability - The ability of a workflow to persist itself could allow multiple servers to be workflow runtime hosts and therefore a workflow could be loaded and executed on a different server than the one that started it.
3. Workflow tracking - This is one of the most powerful features of WWF. However whilst it provides rich support for tracking the workflow through the different stages in the lifecycle, it also tracks rules. Rules evaluation records each rule or rule set as it is evaluated. This could also help to easily explain to users why the workflow has progressed in a particular way. You can even go further and use custom tracking profiles to track the values of the entities at particular points in time when for example a rule was evaluated. This provides a complete tracking picture for rules evaluation
4. Hosting of workflow designers - The tools to host the designer tools within your own applications are exposed and available as part of the WWF framework. This allows you to create even simple designer interfaces very easily e.g. allowing a user to create a number of different states within a state machine and chain them together to describe how each state transitions to the next and the event which cause the transition.
5. Parallel execution of activities within a workflow – Whilst activities cannot execute truly concurrently within a workflow the framework does support execution of two parallel branches of activities in that the first activity of the first branch will execute, when this finishes, the first activity in the second branch executes and so on
6. Exception handling – Provides support for handling exceptions in the workflow e.g. if activities which are part of the workflow do not complete as expected, different activities can be called and activities cleaned up correctly. Related to this is compensation which allows an activity to be undone rather than rolled back, when typical long running transactional flows could not possibly maintain a database transaction for a long period of time.

Rules

1. The rules and workflow data can be specified in an XML based format which is compiled by the WWF into code making it very quick to execute at runtime. The rules files can either be created manually or modified through using a rules designer.
2. Very feature rich rules can be created e.g. if you have access to the invoice entity you could say if Invoice.LineItems.Count > 1 then HasMultipleLineItems, and then write control of flow in your workflow using this rule.
3. Support for forward chaining of rules also means that you don't need to be so concerned with the order or priority you specify rules in. If a rule further down in a list of rules causes fields to change for which previous rules relied upon, this will cause those previous rules to be re-evaluated. Although you can override this for specific rules and specify a priority and indicate you don’t want any forward chaining to occur

Persistence of workflow

1. Objects associated with the workflow are serialized and de-serialised seamlessly. E.g. an invoice workflow object may require references to the invoice itself, and perhaps related purchase orders.
2. Handles the persistence of the workflows in the database or any other data store as required. Given the complex workflows you can create this support coming out of the box is a big plus. Not only is the workflow itself persisted i.e. the current state machine workflow , but all the associated rules and rule sets that the workflow contains
3. Workflow versioning / Dynamic workflow updates - Versioning is also handled well i.e. as you evolve your workflows and add new activities, create new or modify existing rules. Multiple entities can therefore exist which are associated with older and newer workflows. Although conversely with very long running workflows e.g. when a process will take a considerable time to complete you may want to change an existing workflow, this is also supported through dynamic updates.

Integration

1. Publishing of workflows – Workflows can be published as web services, which provide very powerful integration features to third party systems. This enables applications to call into the workflow and take appropriate actions e.g. approve an invoice.
2. Workflow chaining - Allows you to chain workflows together i.e. call one entirely new workflow from within an activity of another workflow.
3. Activities within workflows can call into web services, which provide easy integration points into many different ERP systems.

State machine workflows

1. Support complex state machines - Support for recursive composition of states which effectively means states can contains other states i.e. any events defined by the outer state are available while the workflow is currently in any of the inner states. This allows more elegant state machines to be designed which don’t contain redundant activities e.g. for an invoice a Cancel activity could be defined in a single state rather than all possible states from which the invoice could be cancelled.
2. Visibility of permissible events - The framework provides support within state machine workflows to easily see which events can be called when the workflow is in a specific state. We can easily see how this would be useful e.g. for invoices to determine which action buttons to display such as "Approve" could be driven on whether the "Approve" event is available. This would also facilitate external components knowing the events which they could call through a web service interface.

Using WWF should allow you more time to concentrate on the development of the business logic and also reduce the risks that are inherent in building a generic workflow framework.