Matt Adamson

Building a mercurial external hook to update Target Process

2009-11-09T13:52:00.000-08:00

Whilst learning the new Mercurial Distributed Version Control system ( DVCS ) I thought it would be useful exercise to learn about hooks by writing one. There are two types of hooks you can use, either an in process hook which executes with the hg process or an external hook which is executed as a separate process by hg, the core mercurial process. I wanted to develop a hook which integrates with Target Process to update a specific entity with the changeset comment if the comment referred to it e.g. a comment such as

TP#23000 Resolved issue with print manager crashing on XP

would update bug 23000 with this comment. The mercurial definitive guide recommends using an external hook if your not familiar with python and performance isn't critical. Given my python experience is non existent I created a NET console applicaiton to perform the work. The application interacts with the hg process to obtain changeset information and uses WSE 3.0 web service calls to the target process API to retrieve and update the entities. The important parts of the process are as follows

Environment variables passed to the hook process define useful information such as the path to the hg process. However also the parameters to the hook are passed by using a variable with the suffix HG_ followed by the variable name e.g. HG_NODE contains the changeset identifier.
The changeset identifier passed to the hook as the env variable HG_NODE is used to execute the hg process with the log command to extract the description of the changeset. This uses the -template option to control the specific output of the log command. When calling the hg process the console output is redirected to a string so the contents can be captured.
The captured changeset description is then parsed to determine whether it matches a TP entity using a regular expression. If the changeset references a TP entity the TP web service methods are called to obtain the entity and update it with the comment.

Although this doesn't provide a feature rich hook similar to the packaged bugzilla hook, it does provide you few useful features and enough information to build your own external hook. You may wish to change this implementation i.e.

Only bugs are currently supported however it shouldn't be hard to add support for task / story ids.
The bug status entity is currently obtained using a fixed process name "All Practises" however this could be obtained from the entities related projects related process.
Customising the text added to the TP bug through some kind of template.
Mapping TP users to mercurial users. Currently the user credentials are taken from the config file.

The hook must be installed by adding the following line to mercurial.ini

[hooks]

commit.TPMercurialHook = TPMercurialHook.exe

Target Process actually supports a plug in model for a number of different SC repositories e.g. one for TFS, which work by continually running and polling the repository for changes. Mercurial hooks work by intercepting events in the repository immediately whereas the plug in model polls for changes.

The code for the hook can be downloaded here

Enterprise Windows Workflow Foundation Development Part II

2008-12-20T08:02:00.000-08:00

Over the last week I’ve changed from using the manual thread scheduling service to the default CLR thread pool based one. Some of the techniques I mentioned before have been updated in this post and also a few new items discussed

Exception handling
Workflow performance
Workflow custom tracking service for state mapping
Unit testing
WF design

Exception handling

Using the default thread scheduling service and checking a LastExceptionRaised property from a local service to check exceptions simply will not work now. As the WF execution is performed asyncronously in all cases the property will simply be null when checked straight after calling a WF local service method. Only some time later when the WF thread is scheduled will this property be set.

The best approach I've found to handle this case is to firstly log all exceptions passed to your local service property setter when the WF fault handler kicks in. You can use the enterprise libraries HandleException method or something similar depending on what logging framework you use. This ensures the rich detailed exception information, including all inner exceptions and their stack traces, is captured and logged.

You should also manage a collection of exceptions within the local service to ensure you don’t miss any e.g. if two WF threads raise exceptions at a similar time. To handle the exceptions you will to create a timer which periodically checks whether any exceptions exist in the collection and simply throws the first exception in the list. This can then be caught and handled/ logged / rethrown in your application depending on the exception handling policies defined. You need to use a timer to check the exception and re throw otherwise the workflow would terminate if an exception re thrown on the WF thread.

In terms of "handling" unexpected exceptions, our application uses the AppDomain UnhandledException filter which processes the exception and uses the “Unhandled Exception” enterprise library policy to define the handling. By default we configure this policy to log the exception to the event log and presents the user with a friendly dialog that a serious issue has occurred ( well about as friendly as an unexpected dialog type window can appear )

Workflow performance

In some scenarios you may have many workflows which continually load / unload e.g. if they are triggered by a delay activity. If the delay activity is relatively short e.g. few minutes and you have a few hundred WF's you may find the WF runtime spending most of it's time loading / unloading WF's. There are a few approaches you could take

Develop a customised persistence service and override the UnloadOnIdle method e.g. you might customised SqlWorkflowPersistence. In there you could put some custom logic to only unload some workflows. E.g. you might not unload workflows which are in a state with logic driven by a DelayActivity.
Don’t set UnloadOnIdle to true in the constructor of the persistence service and in the WorkflowIdled event you can decide if a workflow should be unloaded using similar rules as in 1.

In relation to diagnosing general WF performance issues I highly recommend reading Performance Characteristics of Windows Workflow Foundation. There are a variety of useful WF performance counters you can use to inspect your running WF host to avoid second guessing where any bottlenecks might be. As always good detailed analysis is essential so you really are tuning the least performant parts of your app and not those you most like to tinker with :)

Workflow custom tracking service for state mapping

The original approach I mentioned works by mapping the current state to the workflow state in the workflow idle event, however this has the following implications.

The StateMachineWorkflowInstance class can be expensive to construct every time your workflow idles.
The change of status in the state machine may not be immediately reflected in your entities mapped state. i.e. some time passes between entering a specific state and the workflow idling e.g. if performing some work in a StateInitializationActivity within that state the workflow will not idle until the StateInitializationActiviy has completed.

Matt Milner wrote an excellent article on tracking services and presents one which tracks state changes in a state machine activity. As soon as a new state activity starts executing the tracking service is called. I’ve adapted this service so it fires an event on a state change which a local service responds to by mapping the current state to an enumerated status value. The status of the WF state machine and status in the entities record are now immediately in sync with each other

Unit testing

There are many excellent articles on unit test approaches to WF i.e. Matt wrote one and Cliffard has also produced an excellent lab on applying TDD with WF. Two important points when creating tests to add to these articles.

Always try and refactor logic within activities / workflow into separate components and test them in isolation first. Keep the minimal amount of code in your activity. It’s fine for an activity to simply provide a simple façade around an existing service. If you adopt a TDD approach you would actually find yourself more likely to write those services first, and thus make the activities easier to test using mock objects.
Use IOC mechanisms such as dependency injection so you can mock most of the objects your workflow / activities use. This means your tests can focus as much as possible on testing just the activity or workflow class and not many other dependent classes.

In testing a state machine I’d like to test everything about the model I’ve created i.e. the start state / completed states, possibles transitions from each state and assert any rules embedded within the workflow are evaluated, and ensure ALL activities end up being executed and covered. This is very important when you have many IfElse branch activities in your workflow based on rules. The last thing you want to do is change a condition within a branch only to find the branches beneath it can never be reached :(

WF Design

One thing I’ve found out over the last month working on WF is the vast number of approaches there are to implement any given design. WF is highly extensible, both in the new services you can plug in and existing services that can be customised using standard mechanisms such as inheritance / events / overriding methods.

What ever approach you choose it’s important you try and fit within the WF framework as much as possible. It’s up to you how little or much of your workflow logic you decide to implement in WF. Even if you don’t think your design is ideal and doesn’t feel right I would encourage you to simply start and be prepared to refactor as you learn more about WF.

Enterprise Windows Workflow Foundation Development

2008-12-06T20:30:00.000-08:00

It's nearly a year ago since I blogged about some of the really cool features of WF and it's only recently I've had an opportunity to apply this technology in my day job. Although other projects I've worked on until now could have used WF equally well, they were simply too large and complex to refactor WF in easily. However they would have been a great candiate to use WF from the start as they use complex hierarchical state machines for which the logic is unfortunately littered throughout the application.

My previous WF projects were simply for my own learning ( aiming for an MCP qualification eventually ) and although I covered a lot of different areas of WF, it's only through developing a production application I've come across areas that needed to be reviewed further. e.g.

Workflow authoring mode - Out of the three options which is the most appropriate

Unhandled exceptions in the workflow . Exceptions need to be logged by the host and in some cases we want to avoid termination of the workflow

Querying workflow state - It is often not feasible to simply load every workflow to query it's state.

Dependency injection - How can types be resolved

Dynamic updates - Useful for simple customisation

Associating business objects with workflow instances

Runtime scheduling service - Is the default service always appropriate

Workflow runtime and instance wrapper classes

Workflow authoring mode

Although there are three authoring modes to choose from code, code with separation, and no code, visual studio only lets you choose you the first two. All the guidance on this mentions you should evaluate how much flexibility the end user requires to change activities and workflows. Based on this advice though I found it hard to choose an appropriate model. On the one hand we should keep our applications as simple as possible right now based on our requirments, however on the other hand we need to design to make future changes easier :).

I actually think the no code authoring mode is probably the preferred route as it encourages developing good practises i.e. building your activities upfront and then building your workflow around them being clear to separate the rules and the binding of properties between activities as separate tasks. However given the poor tool support in VS 2008 and also 2010 for no code workflows I find it hard to recommend. If you really can't perceive a need for this level of flexibility i.e. hosting the designer and allowing users to change the workflow, then it maybe overkill.

Note if you do choose a code only route and want to move to code separation it is possible with a bit of work. Simply copy all activities within the designer and then paste into a workflow with code separation. This will created the XOML and then you can add all code activities in as required.

For the project I worked on I applied the code separation model which seems to be no more complex to manage than the code only model, but at least provides some flexibility. The workflow and rules could be modified relatively easily by hosting the designers and new workflow types compiled as required.

Exception handling

It's inenvitable your application will have some bugs no matter how hard you try and in this case it's important that the application ( or more specifically a data store ) maintains it's integrity so users can continue using the system. It's also important to capture these exceptions and log them appropraite e.g. we use enterprise library 4.0 which provides rich support to log exceptions.

The default behaviour for unhandled exceptions is to terminate the workflow which also has the effect of removing the workflow from the persistence store. However if your using a state machine which is currently in one of states you may not wish to terminate the workflow abruptly. If the workflow ends you would have to create a new workflow and start the state machine in the same state perhaps and also ensure workflow properties were set correctly.

In our project we managed this by setting up fault handlers in nearly every state which could throw an exception. ( it was acceptable for the first state state to throw an exception and terminate the workflow ). In this fault handler we always handle the generic Exception type and when the exception is raised a local service method is called on the host application which passes the Exception object. The host application can then perform any necessary action e.g. transitioning to a failed state , logging the exception and in most cases presenting the user with an exception has occurred type window. Now we should have good information to diagnose this exception further and the state of the workflow is maintained.

Querying workflow state

There are many mechanisms to query the workflow state e.g. from StateMachineWorkflowInstance there are properties such as CurrentState / PossibleStateTransitions which provide you with a "picture" of the workflow state. However this does require the workflow to be loaded to obtain this state. For many applications it wouldn't be feasible to load all workflow instances just to find out which workflows are in a specific state. It also doesn't lend itself well to standard querying techniques used on a data store e.g. select all invoices from a table which are in state x

Most application entities typically contain a state column which usually maps to an enumerated value defined in the application e.g. InvoiceState with values New, Awaiting Approval / Approved. It would be nice if we could still use this state column to filter out the records were interested in, but at the same time use the workflow instance to drive the workflow, determining possible state transitions / events that can be raised e.t.c. My first thought was to use a custom tracking service to track the changes to the state machine and report them back to the host application which could then update the state column of the related entity to reflect the current state. This way both the workflow instance and state column would always be in sync.

However after creating a custom tracking service to do this I found the excellent article Matt has written on tracking services which also builds a similar service for tracking state machines, well at least I'd learnt tracking services well :) . I then found a much simpler solution which works well and doesn't require the use of tracking services. In the workflow idle event the current state can be checked from the StateMachineWorkflowInstance wrapper class and then the related entitiy updated. This works well because the workflow always idles between transitioning to new states.

Dependency Injection

Dependency injection / service locator which are implementations of the IOC pattern are well known good practises however it's not clear how you can inject instances into workflow types / activities. I did raise this question here and got some useful links. I haven't followed up the related links in detail however I did manage to find a relatively simple solution by using the services based infrastructure which WF provides.

The host application which creates workflow instances creates the types which need to be resolved by dependency injection. Those objects are then passed to AddService from the WorkflowRuntime instance. You then need to access the interfaces through the IServiceProvider interface or the ActivityExecutionContext instance calling GetService . If you only need to access the interfaces in the Activity.Execute method then you can simply use the ActivityExecutionContext instance directly within Execute. Otherwise override the method OnActivityExecutionContextLoad and store away the interface references. If you do this however be sure to mark the fields in the activity / workflow with the NonSerialized attributes to prevent them also being serialised when the workflow is persisted.

Scheduling services

The default scheduling service uses a thread pool to schedule workflows to be executed as required. However given the workflow types in our project AND the host application access the same business object underling the workflow, this does introduce potential concurrency issues. The manual scheduling service provides a model which is simpler to manage and would be more appropriate here.

You have to specifically schedule the ManualWorkflowSchedulerService to run a workflow using the RunWorkflow method. The host application simply does this after raising any events into the workflow through the local service. However some parts of the state machine activities require work to be continually performed in the background waiting for a specific rule evaluate to true and using a DelayActivity to sleep the workflow. To handle this a simple WPF DispatcherTimer can be used with DispatcherPriority.ApplicationIdle priority to execute any workflows which have expired timers. The timer can be setup using the following code

DispatcherTimer workflowPollingTimer = new DispatcherTimer(new TimeSpan(0, 0, 0, 5)),DispatcherPriority.ApplicationIdle,new EventHandler(workflowPollingTimer_Tick), Dispatcher.CurrentDispatcher);workflowPollingTimer.Tick += new EventHandler(workflowPollingTimer_Tick);workflowPollingTimer.Start();

Within the timer the workflows will be executed when their timers expire

ManualWorkflowSchedulerService scheduler = workflowManager.WorkflowRuntime.GetService(<>);

foreach(SqlCePersistenceWorkflowInstanceDescription workflowDesc in persistence.GetAllWorkflows()){

if (workflowDesc.NextTimerExpiration <>

{

scheduler.RunWorkflow(workflowDesc.WorkflowInstanceId);

}

Dynamic updates for customisation

Although dynamically updating a workflow is often used for versioning reasons e.g. migrating old to new workflow versions, or making ad hoc changes to specific workflows at runtime it can often be used to provide flexible customisation within a workflow. Although rules can be managed directly within the workflow, they can also be managed within the application e.g. preferences for document approval. You may find a need where a number of different activities ( based on runtime conditions ) need to be executed and using If / Else type branches within the workflow can quickly make your workflow very complex and hard to maintain.

It's very simple to use a place holder activity within your workflow and then dynamically add activities to add at runtime based on application defined rules. Perhaps you have a number of different approval strategies which need to be executed, these could be added at the point of approval in the workflow. This does provide another option you can use to get the customisation you require without having to write designer hosting support in your applcation :)

Workflow runtime and instance wrapper classes

As you develop the host application you will probably find yourself writing proxies over the WorkflowInstance and WorkflowRuntime CLR classes to provide additional services when executing workflows. They can also make it easier to execute those workflows e.g. defining a ManualResetEvent within the workflow wrapper and then setting this when the workflow completed event fires so you can execute workflows and wait until they've completed. Bruce Bukovics has developed a good set of wrapper classes you can use as a base and adopt as required for your own requirements.

Associating business objects with workflow instances

One of the requirements in a workflow applications is to load a specific business object and then raise a workflow event. To do this you need to maintain a link between the business object and it's associated WF workflow instance. You can add the business object link directly into the workflow or add the workflow instance link into the business object or even both.

If you do both so the business object contains a WorkflowId of type Guid in it's entity table, and the workflow itself contains a reference to the identifier of the business object. In this way the business object can be loaded by the client application, when a workflow event needs to be raised into the business object the correct workflow instance can be retrieved and called on. Having the business object reference directly in the workflow also allows the workflow to access the object to query any data within it or to make modifications and save to the data store.

You may find it preferable to you use an object identifier rather than the object reference otherwise for long running workflows you may find yourself using stale data e.g. if the host application modifies the business object and workflow is using a stale copy. Using an object identifier or even using an Identity pattern ( link to Martn fowlers identify pattern reference ) means you can load the object as and when required in the workflow instance.

Conclusion

I hope I've shown you a few techniques you can use to make your life a little simpler when developing WF applications. Both Maurice de Beijer and Bruce Bukovics have provided me with useful feedback when tackling WF design issues. Bruce has written an excellent book on WF which is even worth getting just for the additional chapters covering NET 3.5 changes. Maurice also teaches a highly respected course on WF.

User interface automated testing

2007-12-04T13:58:00.000-08:00

If you’ve read any of my previous posts you’ll know I’m very keen on unit testing and continuous integration. Up until now we’ve managed to automate the testing of pretty much everything i.e. XSL / components covering many different technologies / web applications both ASP and ASP.NET / and also test and integrate database scripts. One thing I’ve always struggled on though is effective test automation of thick client GUI applications and always managed to push this task aside.

Recently whilst fixing a defect within a server component, I introduced another defect in a thick client user interface application which was using this component. At the time I added a few new unit tests for the component changes and checked all 3544 of our existing unit tests passed. When I got the green light on cruise control I was very happy with my changes and completely oblivious to the fact I’d just broken the GUI application :( When I discovered this a few weeks later I was determined to create a user interface unit test for this application– this was the kick start I needed.

John Robbins is usually more renowned for his excellent debugging articles however here he writes about the new features in NET framework 3.0 which support UI automation. This sounded like an ideal technology to drive the UI and test features. I’d strongly encourage you to read this article and download the sample source code.

In the article Johns written a wrapper over the UIAutomation object model which makes programmatic control of UI elements much simpler than using the UIAutomation API directly. However the wrapper assembly is still incomplete and only covers a few of the common controls. If you use this you will quickly find yourself needing to add new control types however this is relatively simple to do. To select an item in a combo box I created a class called UIComboBox which supports simple methods such as SelectItem. As you can see calling SelectItem sure beats writing this every time you need to select a combo box item in your client code.

public void SelectItem(Int32 comboBoxItemIndex)
{
AutomationElementCollection comboBoxItems = this.RootElement.FindAll(TreeScope.Descendants,
new PropertyCondition(AutomationElement.IsSelectionItemPatternAvailableProperty, true));
AutomationElement itemToSelect = comboBoxItems[comboBoxItemIndex];
object pattern = null;
if (itemToSelect.TryGetCurrentPattern(SelectionItemPattern.Pattern, out pattern))
{
((SelectionItemPattern)pattern).Select();
}
}

One wrapper control I certainly missed was one associated to the NET DataGrid. I was actually automating a NET 1.1 application and although I found this solution , it only works on a NET 2.0 DataGridView. Still if you are using this you can build a very nice API with methods such as SetAt and GetAt to retrieve / set cell content within the grid. The only solution I could find to fill the NET 1.1 grid was to use the SendKeys method to send all the required keystrokes to navigate through the grid.

All of the UIAutomation and the corresponding wrapper methods in the article download use methods which take a UIAutomation identifier. To create tests quickly it’s vital to use a tool such as UISpy to obtain these identifiers. All you need to do is put UISpy in “hovering mode” and then simply hold down the Ctrl key whilst hovering over a window in the application being automated. The window is highlighted in red and UISpy then reports all the details in the right hand pane, most importantly including the AutomationId.

Once you have the automation identifiers it’s very straightforward to write code to access any GUI elements e.g.

UIEditControl invoiceNoTextBox = appWindow.FindDescendant("invoiceNumberTextBox", ControlType.Edit) as UIEditControl;
invoiceNoTextBox.Value = "SLOG_Invoice_1";

It is usually far more effective to use FindDescendant to find controls rather than FindChild given this searches all children recursively and will eventually find the control if it exists :)

When building unit tests you’ll usually find you automate the GUI to a point where you need to test the expected output. This could either be

Verifying GUI control values have been set correctly by asserting the contents of the controls. E.g. you could take the user through a task such as create a new invoice and then assert the contents of a screen which shows the invoice detail ( perhaps a wizard summary screen ).
Verifying a specific action has occurred by invoking business objects and then asserting the values from the business object. E.g. you could automate the GUI task to create a new invoice and then when this is complete verify the invoice exists in the data store and the values e.g. invoice no, match the values you entered in the GUI.

User interface testing is just as important as any other component testing that I do hope more people embrace this and start to automate the testing of a good portion of their user interfaces. As developers we cannot and should not rely on QA to find the defects, at least certainly not the obvious ones. With a few simple unit tests we can make great strides to catch any serious defects before QA and then the customer see the product.

Design - Modelling user interfaces in UML

2007-10-18T14:32:00.000-07:00

In my previous post I focused on how to model a web application and it's component parts e.g. server and client side pages into UML elements. In this post I'll describe an EA add in I built to help reverse engineer UML models from user interface elements such as dialogs and there component parts e.g. check boxes, text boxes, button. Although the add in works only on thick client applications, the approach could also be used for web applications.

Most of the screens in your application, there contents and flow to other screen are typically driven by the use cases you write. It helps to model the screens using UML notation so you can describe each element and model how each element links or depends on other elements. Once the elements are in a UML model you could also show interaction e.g. an actor clicking a button which instigates a business object function which then builds a list box and populates a text box, and then enables another button.

Enterprise Architect is an excellent low priced modelling package, however whilst it provides many useful features and powerful forward and reverse engineering options ( it even includes a debugger which can auto generate sequence diagram based on a run through code ), it also provides great extensibility through the add in interface. In the previous post I discussed an addin I wrote to reverse engineer the UML model from various web page files, however this addin works differently by using a spy++ type capture pointer. The user only needs to hover the pointer over a window they need to capture, the addin then creates a UML model of the dialog using all the component parts of the window, with the exact same size and position as they appear on screen.

Thanks to Mark Belles for writing the windows capture code which provides the hover utility to obtain a window handle. From this it was a simple matter of extracting all the windows information using the Win32 API. The add in also should also provide a good template for anyone else writing one and contains supporting classes that help wrap the unmanaged EA interfaces.

The screen below shows the EA options dialog

When you start the add in through EA you are presented with the following dialog

The UML diagram below then gets generated.

Although the output isn't perfect it's a lot more productive than creating all yours screens manually. Not a job Id want to do often :)

The full source code to the add in written in C# is available in the EA WIKI, I hope you find this useful.

My first open source project - Web modelling UML add-in for Enterprise Architect

2007-09-06T13:33:00.000-07:00

I recently created my first open source project called EA WebModeller Add In which is an add in for the Enterprise Architect ( EA) UML modelling tool. The add in provides a much filled gap to the product to enable users to reverse engineer web projects into UML artifacts. Specifically using the web extensions for UML which Jim Conallen produced. The web extensions provide a standardised way of modelling the components which make up a web application.

A seemingly simple concept, which most would just think of as one web page associated with another quickly becomes very complex and is proof of why Jim could write a substantial book on this subject and still leave room for more. Classes are used with a variety of different UML stereotypes to define the various components which combine to produce a web page e.g. server side page which builds a “client side page” which aggregates a number of “forms”. Various stereotypes of association are also defined to support different associations between the components e.g. one server side page redirects to another server side page or a client side pages links to another client side page ( through hyperlinks perhaps ). Components contained within pages both on the server and client are also modelled appropriately.

The example web project included with the download available on source forge contains a number of scenarios e.g. file 1.asp includes files include1.asp and include2.asp. Both included files contain server side functions. 1.asp also contains a redirect call to 2.asp, an embedded class, some client side functions ( in java script ) and an embedded form. The 1.asp server side page and it’s associated elements are shown in the diagram below

This is produced by the following settings in the add in

The add in is built in an extensible way to allow new script parsing languages to be easily integrated within the framework e.g. a base ScriptParserBase class contains JavaScriptParser and VBScriptParser derived classes. Similarly the ServerPage class which models the UML server side page contains derived classes ActiveServerPage (ASP ) JavaServerPage (JSP) and DotNetActiveServerPage ( ASP.NET ) for each of the web application technologies. Note only the ActiveServerPage class is fully supported for ASP pages, I’m hoping the open source community will develop the JSP and ASP.NET technologies :)

The add in uses mostly regular expressions to parse the script source into it’s appropriate components. They range from very simple expressions such as #include\s+(?virtualfile)=""(?.*) which parses include directives within ASP pages. The PathType and IncludedFile are named captured groups which here capture the type of file reference e.g. virtual or absolute file location. To relatively compex expressions such as “^\s*function\s*(?\w+)$(?.*)$\s*(?(?:(?\{)(?\})(?(LeftBrace)[^\{\}]*.*))*)” which use “balance grouping” to support recurision within the expression. This power is required here to match all function definitions because we can’t simply match an opening and closing brace as we may find the first closing brace is not closing the function, it’s in fact closing the first for loop defined within the function.

To parse the client side page the internet explorer active x control is used to load the htm file produced when navigating to the web page. By using the IE control, the parts of the web page such as embedded forms and controls within the form can be found and added to the UML components. I actually wrote a web application testing framework around this control however recently I’ve started to use the WatIN framework which is more feature rich.

Given the main web application our team develops is written in ASP with many hundreds of pages, the productivity boost from using the add in is huge. Although we couldn’t possibly try and reverse engineer the whole web application in one go, being able to selectively single out and reverse engineer a few web pages when adding new features or doing some refactoring is of great help. It clearly shows the effected pages and where the new logic is best placed and perhaps most importantly helps to show the impact of any changes on existing functionality.

Window Workflow Foundation - A new way to develop workflow

2007-08-27T13:27:00.000-07:00

I’ve recently reviewed the new workflow framework called Windows Workflow Foundation ( WWF ) which is part of the .NET framework 3.0 runtime. Using the framework successfully within our applications will require a completely different thought process on traditional methods e.g. where we typically put most of the workflow behaviour directly within the code.

I started to develop a generic workflow framework implementation of core business object services a few years back, however even with considerable effort, the implementation lacks many of the features provided by WWF. Given that most business applications go some way to make them flexible by developing a generic workflow framework, use of WWF should allow most teams to concentrate more on the back end business logic rather than trying to do what WWF does very well.

I would highly recommend reading Bruce Bukovics excellent WWF book http://www.apress.com/book/bookDisplay.html?bID=10213 which provides coverage of all the important features in WWF very clearly.

I have summarised the main points of WWF below

General Points

Multiple types of workflow – It handles both sequential and state machine workflows, although typically P2P / Banking applications which Bottomline develops use state machine workflows because the processes don't define a fixed flow of control within the workflow.
Scalability - The ability of a workflow to persist itself could allow multiple servers to be workflow runtime hosts and therefore a workflow could be loaded and executed on a different server than the one that started it.
Workflow tracking - This is one of the most powerful features of WWF. However whilst it provides rich support for tracking the workflow through the different stages in the lifecycle, it also tracks rules. Rules evaluation records each rule or rule set as it is evaluated. This could also help to easily explain to users why the workflow has progressed in a particular way. You can even go further and use custom tracking profiles to track the values of the entities at particular points in time when for example a rule was evaluated. This provides a complete tracking picture for rules evaluation
Hosting of workflow designers - The tools to host the designer tools within your own applications are exposed and available as part of the WWF framework. This allows you to create even simple designer interfaces very easily e.g. allowing a user to create a number of different states within a state machine and chain them together to describe how each state transitions to the next and the event which cause the transition.
Parallel execution of activities within a workflow – Whilst activities cannot execute truly concurrently within a workflow the framework does support execution of two parallel branches of activities in that the first activity of the first branch will execute, when this finishes, the first activity in the second branch executes and so on
Exception handling – Provides support for handling exceptions in the workflow e.g. if activities which are part of the workflow do not complete as expected, different activities can be called and activities cleaned up correctly. Related to this is compensation which allows an activity to be undone rather than rolled back, when typical long running transactional flows could not possibly maintain a database transaction for a long period of time.

Rules

The rules and workflow data can be specified in an XML based format which is compiled by the WWF into code making it very quick to execute at runtime. The rules files can either be created manually or modified through using a rules designer.
Very feature rich rules can be created e.g. if you have access to the invoice entity you could say if Invoice.LineItems.Count > 1 then HasMultipleLineItems, and then write control of flow in your workflow using this rule.
Support for forward chaining of rules also means that you don't need to be so concerned with the order or priority you specify rules in. If a rule further down in a list of rules causes fields to change for which previous rules relied upon, this will cause those previous rules to be re-evaluated. Although you can override this for specific rules and specify a priority and indicate you don’t want any forward chaining to occur

Persistence of workflow

Objects associated with the workflow are serialized and de-serialised seamlessly. E.g. an invoice workflow object may require references to the invoice itself, and perhaps related purchase orders.
Handles the persistence of the workflows in the database or any other data store as required. Given the complex workflows you can create this support coming out of the box is a big plus. Not only is the workflow itself persisted i.e. the current state machine workflow , but all the associated rules and rule sets that the workflow contains
Workflow versioning / Dynamic workflow updates - Versioning is also handled well i.e. as you evolve your workflows and add new activities, create new or modify existing rules. Multiple entities can therefore exist which are associated with older and newer workflows. Although conversely with very long running workflows e.g. when a process will take a considerable time to complete you may want to change an existing workflow, this is also supported through dynamic updates.

Integration

Publishing of workflows – Workflows can be published as web services, which provide very powerful integration features to third party systems. This enables applications to call into the workflow and take appropriate actions e.g. approve an invoice.
Workflow chaining - Allows you to chain workflows together i.e. call one entirely new workflow from within an activity of another workflow.
Activities within workflows can call into web services, which provide easy integration points into many different ERP systems.

State machine workflows

Support complex state machines - Support for recursive composition of states which effectively means states can contains other states i.e. any events defined by the outer state are available while the workflow is currently in any of the inner states. This allows more elegant state machines to be designed which don’t contain redundant activities e.g. for an invoice a Cancel activity could be defined in a single state rather than all possible states from which the invoice could be cancelled.
Visibility of permissible events - The framework provides support within state machine workflows to easily see which events can be called when the workflow is in a specific state. We can easily see how this would be useful e.g. for invoices to determine which action buttons to display such as "Approve" could be driven on whether the "Approve" event is available. This would also facilitate external components knowing the events which they could call through a web service interface.

Using WWF should allow you more time to concentrate on the development of the business logic and also reduce the risks that are inherent in building a generic workflow framework.

Continuous Integration – Using integration queues

2007-08-16T14:02:00.000-07:00

One of the long standing issues we had with our continuous integration project was the lack of any concurrency control when multiple projects were executing. Although there was some support for this through custom plug ins to ccnet it wasn’t integrated within the core code well i.e. you couldn’t easily see through cctray which projects were executing and which projects were pending, waiting on other projects to complete.

I managed to successfully integrate this new release with our project today which means for example when our unit test project is building, the core NET build project can no longer execute, which previously could cause all components to be removed whilst tests were running L. We have quite a few ccnet projects with a few dependencies so this is very useful. I’ve included the project dependencies in the diagram below

The dependencies shown in the diagram are not true UML dependencies, they simply show the project dependencies which trigger builds to occur i.e. if the Sprinter3.6.4 database project performs a new successful build, it will trigger the web app project to build.

If the diagram showed true UML dependencies then there would be a dependency between the NET assemblies project and the Sprinter3.6.4 database because the NET components depend on the schema of the database. However the NET code doesn’t need to be re built if the database changes, only component or web application unit tests should be re-executed.

Debugging - Exception stack traces are not always detailed enough

2007-05-18T03:23:00.000-07:00

We often use the Microsoft Exception Management Application Block ( MEMAB )to publish unhandled exceptions in our applications. This provides a detailed log including all inner exceptions contained within the outer most exception. For each exception the message and stack trace are output. For the inner most exception the stack trace usually contains the actual method which caused the exception.

Although most well written applications should contain small reusable methods of perhaps no more than 20 lines, this is often not the case. If methods only contain a few lines, even with a stack trace it is sometimes very difficult or impossible to pin point the exact line which is causing the exception to occur. To find out the exact line causing the error the best course of action would be to take a dump. You can find out how to do this for a .NET application in my article http://msdn.microsoft.com/msdnmag/issues/05/07/Debugging under “Managed First-Chance Exception”. Once you have a dump you can usually find the line of code on the production server without even using debugging symbols using the techniques highlighted below.

Assuming you have a System.NullReferenceException thrown from a .NET component you should trap access violation exceptions ( av code ) and take a full dump on this type of exception. E.g. assuming the MEMAB logs the following exception info

Exception Type: System.NullReferenceException
Message: Object reference not set to an instance of an object.
TargetSite: Void Tranmit.Common.Interfaces.IXmlSerialisable.LoadFromNode(System.Xml.XmlNode, System.Collections.Hashtable)
We can find out the exact location in LoadFromNode by loading the full dump file in windbg then executing

We can find out the exact location in LoadFromNode by loading the full dump file in windbg then executing

0:000> !u @eip
Will print '>>> ' at address: 04a34312
Normal JIT generated code
[DEFAULT] [hasThis] Void Tranmit.Sprinter.AccountCodeCollection.Tranmit.Common.Interfaces.IXmlSerialisable.LoadFromNode(Class System.Xml.XmlNode,Class System.Collections.Hashtable)
Begin 04a34168, size 25b
04a34168 55 push ebp
04a34169 8bec mov ebp,esp
04a342de ff15381ea204 call dword ptr [04a21e38] (Tranmit.Sprinter.AccountCode.ExtractIdentityFromNodeAsString)
04a342e4 8945d8 mov [ebp-0x28],eax
04a342e7 b9241fa204 mov ecx,0x4a21f24 (MT: Tranmit.Sprinter.HistoryItem)
04a342ec e827ddf8fb call 009c2018mscorwks.pdb not exist
Use alternate method which may not work.
04a342f1 8bf0 mov esi,eax
04a342f3 8b45e8 mov eax,[ebp-0x18]
04a342f6 8945c8 mov [ebp-0x38],eax
04a342f9 8b157cc31202 mov edx,[0212c37c] ("The level {0} account code {1} could not be found")
04a342ff 8955cc mov [ebp-0x34],edx
04a34302 b978afba79 mov ecx,0x79baaf78 (MT: System.Int32)
04a34307 e80cddf8fb call 009c2018mscorwks.pdb not exist
Use alternate method which may not work.

04a3430c 8945d0 mov [ebp-0x30],eax
04a3430f 8b4ddc mov ecx,[ebp-0x24]
>>> 04a34312 8b01 mov eax,[ecx]
04a34314 8b400c mov eax,[eax+0xc]
04a34317 8b803c040000 mov eax,[eax+0x43c]

Reviewing the source code we can see

String accountCodeIdentity = Tranmit.Sprinter.AccountCode.ExtractIdentityFromNodeAsString(accountCodeNode);
historyItems.Add(new HistoryItem(this,
String.Format("The level {0} account code {1} could not be found", accountCode.Level, accountCodeIdentity),
DocumentType.SupplierInvoiceLineSplit));

Although we have no symbols loaded on the production server we can be pretty sure that the line

04a34302 b978afba79 mov ecx,0x79baaf78 (MT: System.Int32)

Refers to the references in the String.Format method call to the Int32 property Level referenced on the accountCode property. Reviewing the code surrounding this line we can see why the account code object would be equal to a null reference and proceed to diagnose the issue further.

Code Reviewing - Improving our process

2007-01-26T06:23:00.000-08:00

We have implemented a code review process over the past 3 years which seems to work reasonably well. This helps considerably to reduce the number of defects QA and ultimately the customer will see. However it’s only recently that I’ve started to analyse our current process with a view to improve it.

One of my biggest concerns is that we find a lot of non functional defects and not many functional defects in code review. Whilst the non functional defects are important in terms of helping with code maintainability, functional defects should be the top priority on any code review as these defects usually manifest themselves as issues which would be visible to a customer.

I have recently read a copy of “Best Kept secrets of peer code review” which is freely available from smart bear software at http://smartbearsoftware.com. It provides useful techniques which can be used to improve your code review process and are based on real life case studies. Although of course the book does lean towards using smart bears code collaborator product to help with the code review process it does have a lot of useful content to warrant reading it even if you’re not likely to purchase these tools.

Some points I picked up from the book which I’m going to look at in improving our process are

Initial quick scan – The first initial scan can be very important in helping find defects throughout the code. As there is a negative correlation in the time the reviewers takes on the first scan versus the detected speed. The more time the reviewer takes on their first scan, the faster the reviewer will find defects. If you don’t perform a preliminary scan your first issues identified may focus on the wrong areas. With a quick scan you can identify areas with the highest possible issues and then address each one with a good change you’ve selected the right code to study.
Review for at most one hour at a time – The importance of reviewing in short time periods shouldn’t be underestimated. We should try and avoid large reviews and try and split them up into smaller reviews where possible. E.g. you could certainly split up reviews of unit test codes versus core code. Although of course this isn’t an excuse to submit code with known defects for review.
Omissions are hardest defects to find – It’s very easy to find issues for code you’re reviewing, it’s much harder to find defects in code which isn’t there. Having a review check list illeviates this issue some what to remind the reviewer to look for code which may not be present. Defects that aren’t associated with any checklist item should be scanned periodically. Usually there are categorical trends in your defects; turn each type of defect into a checklist item that would cause the reviewer to find it.
Defect density and author preparation – Providing some preparatory comments before code is submitted for review does seem to have an impact on defect density. The key point here is that by annotating certain areas of code, developers are forced to do some kind of self review of their code which obviously increases the quality. Although there is a danger that the developer by preparing could disable the reviewer’s ability for criticism i.e. he’s not thinking afresh. However usually by developers preparing these kind of comments makes them think through their logic and hence write better code.
Leverage the feedback you’re getting - Improve your ability and make yourself a better developer by reviewing the points you commonly receive. Code review points should be learned so you reduce the number of defects over time. A useful exercise could be to put your comments in your own personal check list which you visit when ever performing a code review. Any points which aren’t covered on the global check list should be added
Developer and reviewer joint responsibility – Defects found in code review are good. A defect found in review is a defect which never gets to QA or the customer. Developer and reviewer are a team and together plan to produce excellent code in all respects. The back and forth of coding and finding defects is not one developer punishing another, it’s simply a process in which two people can develop software of far greater quality than either could do single-handedly. reviewer finds in the developers code the better, this should be thought excellent summary
Checklist – Use check lists as an aid to identify common defects. The most effective way to build and maintain the checklist is to match defects found during review to the associated checklist item. Items that turn up many defects should be kept. Defects that aren’t associated with any checklist item should be scanned periodically. Usually there are categorical trends in your defects; turn each type of defect into a checklist item that would cause the developer to find them next time.

Design - Transitioning from conceptual domain to physical data model

2007-01-09T02:52:00.000-08:00

When creating the design for new requirements we often design a domain model of key entities and then when were happy with it we migrate this domain model into a logical and then physical data model. Although we should go to the logical model and then transition to the physical data model we often go straight to a physical data model.

We currently perform this process manually however Enterprise Architect ( UML modelling tool ) has got great support for transforming a domain model into a data model using model driven architecture ( MDA ) templates. It simply reads the entities and the multiplicity relationships between them and generates an appropriate data model. Handling many to many relationships isn't handled with the built in templates however you can easily customise the transformation template using an editor e.g. you can specify

%if attTag:"columnLength" != ""%
length=%qt%%attTag:"columnLength"%%qt%
%elseIf classTag:"columnLength" != ""%
length=%qt%%classTag:"columnLength"%%qt%
%endIf%

to use tagged values to denote the length of a character string in the domain model. Although you shouldn't be too concerned with lengths of strings in the domain model, if you want to specify this information in a tagged value you can at this point, and then ensure it's carried over to the data model e.g. to create a varchar(100) field for a UserName attribute of a User entity. Sparx support are currently developing these templates further and the latest version of the template handles many to many relationships by creating a link table in the format JoinTo however you could customise this to fit your own database standards.

It would be great if through automation we could eventually create a data access layer from the conceptual model using the following steps

Create the conceptual domain model. A business analyst would ideally create the first draft
Add attributes as required to turn this model into a logical data model
Transform the logical data model into a physical data model

We actually use LLBLGen ( object relational mapping tool ) to generate a data access / business object layer from the database so in this way we could continue this process and generate the LLBLGen classes by following some additional steps

Create the database from the physical data model.
Create the LLBLGen classes from the database directly.

Both of these steps can actually be automated through the automation ability within Enterprise Architect and LLBLGen can be automated too to forward engineer the generated code. In this way we could go from a logical model to a generated database and then to an auto generated business object / data access layer automatically.

Unit tests - Web applications

2006-12-01T03:46:00.000-08:00

We have had our unit test .NET assemblies up and running through Cruise Control for nearly a year now. This has worked very well in catching bugs early introduced by developers hastily checking in code without ensuring that ALL unit tests execute successfully. We use a main nant target which executes about a dozen testing assembly and any failures send an email to the development group so the whole team can see the user/s responsible for the checkins which broke the build. We name and shame them and this usually gets them, me quite a few times :(, to fix the build promptly.

However over the last year I noticed 90% of the defects we introduced related to the web application e.g. common script errors or simple page errors which should have been caught however because

The quality of testing performed by each developer was minimal
No developer ever regression tested many other potentially effected areas

We usually had a large number of issues which QA found come each major release. It was then a battle to fix them AND fix any other defects found from the new changes in the release. We barely had enough bandwidth to fix defects from the new changes let alone issues introduced in other areas. Since then I have had a strong motivation to enable a cruise control project which checks for changes in the web application and runs a unit test assembly with a selection of tests in the web app.

The unit testing of the web application requires a new class I’ve created called InternetExplorerWrapper which provides a number of interesting features

Managed access to the unmanaged internet explorer object to manipulate the object model of page contents when pages are navigated to. E.g. forms and their input fields, frames, anchors, images among many other elements.
Support to create unit test assertion with methods such as AssertFormFieldClassName which assert the class name of a specific form field e.g. used in test which specify a different class for fields which fail validation.
Internal auto reset event objects to ensure methods such as ClickAnchor only return back to the unit test code when the anchor has been clicked and the new page navigated too. The document complete event is used in this case, however this is made tricky because in some cases 3 or 4 document complete events maybe fired together because IE is loading multiple documents for a given request.
Methods to assist with viewing and manipulating the IE object model e.g. outputting the contents HTMLDocument object to the trace window. In this way you can see the elements you can access through your unit tests at specific points in time. Also other methods such as TraceDocsLoaded show you how many events are fired for a given request to help determine how many document complete events to wait for in requests such as ClickAnchor ( c)

There were quite a few details which were problematic e.g. the creation of pop up windows e.g. these need to be wrapped by catching the NewWindow event, then create a new IE wrapper object around this and storing in a hash table within the original requested page. Client code can then access the new windows using methods off the main IE wrapper object such as GetNewIEWindow.

Has anyone else had experience creating unit tests for web applications, I’d be interested to hear your thoughts. The framework should work regardless of the platform used to build the application e.g. ASP / ASP.NET and JSP.

Unit tests - Pattern for testing localisable code

2006-09-20T05:38:00.000-07:00

Currently we have a few unit tests, although this is sure to increase, which test code should work in any locale ( culture in .NET terms ). To test this we currently have an implementation such as

public AppVersionTest()
{
// Set the current locale to French Thread.CurrentThread.CurrentCulture = new CultureInfo("fr-FR");

Assert.AreEqual(3.6, Tranmit.Sprinter.Configuration.Database.TBConfig.ApplicationVersion, "Failed to read the configuration ApplicationVersion");
}

We did this as we actually had a bug when the code was running on a french machine. I thought a better pattern would actually tests the code in all defined cultures e.g. changing the implementation to

// Obtain the current culture
CultureInfo oldCurrentCulture = Thread.CurrentThread.CurrentCulture;
try
{
foreach(CultureInfo cInfo in CultureInfo.GetCultures(CultureTypes.AllCultures)
{
Thread.CurrentThread.CurrentCulture = cInfo
// Perform test code
}
}
finally
{
// Restore the previous culture
Thread.CurrentThread.CurrentCulture = oldCurrentCulture;
}

Any code which is locale independant should certainly work in any culture e.g. if monetary code was being tested the culture would allow the code to find thousand / decimal separators values, so this seems a sensible implementation. However this certainly strikes me as a pattern which NUnit could implement for test methods decorated with an attribute such as TestAllCultures which would reduce this to

Test, TestAllCultures]
public AppVersionTest()
{
// Test code
}

This would certainly reduce a lot of boilerplate code required for each test method and I'm sure useful to many other developers who have to write code which works across all cultures.

Unit tests - Pattern for time changing test methods

2006-09-18T06:03:00.000-07:00

Some of our recent unit tests change the system time within the execution of the test method. e.g. testing components which escalate purchase orders to new users if the approvers don't respond within a configured time period. We test this by setting up configuration, such as the period of time which must elapse before escalation occurs, and then changing the system time to add this time period and call the component again. The assertion is usually a simple one e.g. a new user such as the approvers manager should now in the approval route for the order.

We currently use a simple method of restoring the system time when the test method is executed by resetting it to the value captured when the test method is started. If the test method takes 2 seconds to elapsed and we have perhaps 10 test methods we can easily see that we lose 20 seconds of time every time the test fixture is called.

I've thought of a better way to do this using a class such as RestoreTimeChangingCode. The class should be created as the first object which holds the following values

a) StartTickCount
b) StartTime

The StartTickCount value is obtained using Environment.TickCount and StartTime from DateTime.Now. The Restore method should be called when the test method has finished executing, preferrably in a finally block so it's called even if an exception is raised. The Restore method simply adds the number of ticks which have elapsed since the object was created, which represents the elapsed time the test method took to execute, and adds this to the start time, and sets this as the current time.

Whilst this class is useful it would be great if the NUnit framework provided perhaps an attribute called TestChangesSystemTime which could be added in addition to the standard Test attribute. The NUnit framework could then time the method and restore the time appropriately. It could also use a more accurate timer such as the performance counter timer.

The TestChangesSystemTime attribute, in addition to reducing the amount of repeatable code you have to write in each test method, would also provide the ability to reflect on a testing assembly to see whether any test methods change the system time. This would be useful if changing the system time was a concern on any server e.g. we would not want to execute this test assembly on our source control server to avoid clients inadvertently checking in / checking out files with the incorrect date / time.

Unit tests - Testing performance

2006-09-13T22:14:00.000-07:00

We currently use a single performance unit test assembly which has all performance related times e.g. testing X invoices can be created in under Y seconds. If the tests fail it's usually because either

Database changes have been applied such as indexes / triggers or modfications to stored procedures, to slow the time to perform the database access code
Business logic has been changed e.g. when saving the invoice additional validation has been added which slows the time to create an invoice.

There are two important things to note about our implementation

The expected time periods for the test methods are obtained from the configuration file. This is required because of the differences in expected time from machine to machine. If the expected time is hard coded, the code would need to change when the test is executed on different machines. The key value is read from the configuration file using reflection to obtain the method name the test is executing in, e.g. a configuration key of SP_INVOICE_PERFM_1-MaxTime maybe used for a test method called SP_INVOICE_PERFM_1. The configuration file with all the expected timings can be managed in your source control repository and checked in when the unit test assembly is. We usually check in the configuration file with the timings working on our cruise control build machine. If anyone checks in some bad database code, or some ridiculous index which slows the whole process down, we soon know about it when the next build breaks because of the failing tests.
The actual time period tested is not the duration of the test method, it really starts and stops at specific points in the method e.g. an invoice creation performance test is fine to test the whole method because all the method does is created a number of invoices which should be complete in X seconds. However a second test which verifies X invoices can be updated in Y seconds firstly needs to create the X invoices, and only then start the timer when just before the first update occurs, and stops when the last invoice has been updated. And of course all the invoices have to be deleted when the test finishes which shouldn't be included in the expected time. The SetUp and TearDown methods can't be used here because the unit test class such as InvoicePerformanceTest has many different methods with different set up and tear down implementations, and SetUp and TearDown are common methods called by all test methods in a text fixture. On a side note perhaps it would be useful to extend NUnit one day to support multiple set up and tear down attributed methods which apply to specific test methods.

In terms of implementing this generically in NUnit an attribute such as TimedTest with an optional attribute to specify whether the time period is obtained from a configuration file using a similar scheme we've devised. Also to cope with b), the framework could have a Start and Stop timing event which the test method could raise to indicate when the timing should start and when it should stop. This behaviour could also be specified using another parameter in the TimedTest constructor, e.g. if set to false then the start and stop would be implicit as soon as the test method started and finished.

Unit tests - Executing over multiple application versions

2006-08-03T02:05:00.000-07:00

The main application I design and develop usually contains additional database schema with each new version e.g. a recent 3.6.1 release will no doubt have a few new tables / new stored procedures e.t.c. Currently all our automated unit tests execute against a single database using the latest schema. When we complete and release 3.6.1 the cruise control build will only execute tests against this schema. However I would like to be able to execute our unit tests against old versions at least one back if not more e.g. so we can execute tests against 3.6.0 and 3.6.1. We do put effort into the assemblies to ensure backwards compatibility so it would be very handy to test this within our continuous integration process. Even in cases where we haven’t added any specific backwards compatibility code it would be nice to know existing tests still worked on old databases as we will still apply database patches to them.

The obvious issue with executing the most up to date unit test assemblies is some new features in 3.6.1 won't work on a 3.6.0 database. e.g. assuming a new calendar table was included in a feature in 3.6.1, any tests which required this table would fail if run on databases less than 3.6.1. A calendar class may typically exist in a common assembly which contains many other types which work in previous versions. There are essentially three types of modiciations we could be making to unit test assemblies in each new release

Addition of schema which wouldn't work in previous versions e.g. using new table / sps / UDFs
Updates to existing unit tests. As we adapt existing code we may have to change tests in some cases slightly, although this should be quite rare
Addition of new tests for old schemas e.g. we may find some additional scenarios which haven't previously been tested.

2 and 3 obviously don't pose any issues however when executing unit tests against old databases we clearly need to conditionally execute the tests in 1 only if the schema is valid. A solution to this could be to apply an optional attribute against any test method called something like ApplicationVersion which would take the version number of the schema the unit test can be executed on. The attribute could be applied on a class or method. If applied on a class all tests within the class would require this version number unless any test methods overrode this by specifying a different number. e.g. a CalendarTest class could be defined like this

[TestFixture]
[ApplicationVersion(3,6,1,0)]
public class CalendarTest

However I thought whilst the attribute and use itself seemed quite generic, the actual implementation to retrieve the version number would not be. There are two ways in this case NUnit could be modified to incorporate this

Use a plug in framework within NUnit. The plug in could have one method which allows the method to determine whether an actual method is executed or simply ignored. A Bottom line plug in could simply have knowledge about the custom attribute, read it using reflection at runtime, and then verify the version number against that stored in the database and decide whether or not to ignore the method.
Have specific knowledge of the ApplicationVersion attribute and perhaps allow a simple piece of configured embedded C# script which could evaluate what the application number would be at runtime e.g. for us we could read it from the data store from the a table which contains the schema version.

If tests were ignored, the ignore reason could be added automatically e.g. "Test method ignored as version executing against is 3.6.0 and this method requires 3.6.1".

This feature would certainly be useful for us as we have a large number of clients using old versions and only a few at a time initially using the most up to date versions. I'm sure the solution would be useful to others too, thoughts?

Continuous Integration of database scripts

2006-08-02T06:37:00.000-07:00

I've recently got Cruise Control working on our development server to get latest from source safe, build our .NET assemblies and execute all unit tests. I'm extremely pleased with this and has really proved it's worth recently to show complete visibility to the team of failed builds. Any broken builds which do occur get turned around very quickly; I would strongly encourage anyone who hasn't already integrated cruise control into their build process to do so now.

I have also recently taken this further by integrating database script changes into the process. In source safe we keep all the creation scripts for tables / stored procedures / UDFs e.t.c. When any changes are detected on the VSS database the scripts are applied using a few nant tasks and then the unit tests are re executed. This has quickly caught a few issues recently when stored procedures weren't in sync with the .NET code e.g. wrong data types or too many parameters in the DB compared to the client code.

Like most development teams we have to maintain a few versions of our application and upgrade databases from old version to new versions. The latest database script are rerunnable against old versions of the schema i.e. it will add columns / stored procedures if their not already there, Ideally I would like to execute the database scripts against a number of different versions of the database and then execute the unit tests against the newly updated database to ensure the tests still passed. This would find issues for clients upgrading very quickly. I'd be interested in hearing from anyone else who has tried to automatically integrate database changes within their continous integration process.

Unit tests - Changing app.config values at runtime

2006-07-21T00:48:00.000-07:00

It would be useful for a number of unit tests to change the configuration values used in the app.config file of a unit test DLL. Actually changing the value in the file is straightforward enough i.e.

Get access to the app.config file full path from the process using AppDomain.CurrentDomain.SetupInformation.ConfigurationFile
Load the file into an XmlDocument
Extract the relevant node using code such as XmlNode node = xmlDocument.SelectSingleNode(@"/configuration/appSettings/add[@key='" + configurationName + "']");
Set the value of the “value” attribute using node.Attributes.GetNamedItem("value").Value = configurationValue;

However by default changes to the configuration file won’t be reflected at runtime i.e. code such as

String fromEmailAddress = System.Configuration.ConfigurationSettings.AppSettings["EmailFromAddress"];

will not reflect the new value if EmailFromAddress is changed in the app.config file. The AppSettings class caches the values retrieved within the AppDomain and never refreshes them. ASP.NET as a CLR host actually does refresh the settings when the web.config file is changed however by default the standard CLR host won't.

Since there is no way to force the settings to be re loaded the simplest approach would be to ensure that all clients use the existing façade defined in Tranmit.Common.ConfigurationSupport. The GetValue method looks like this

public static Object GetValue( String key,
Type typeOfObject,
Object defaultValue)
{
// Assign default value to object if no key found
Object value = defaultValue;

System.Configuration.AppSettingsReader appSettingsReader = new System.Configuration.AppSettingsReader();

// Check value is present first to avoid exception
// being thrown from AppSettingsReader.GetValue
if(ConfigurationSettings.AppSettings.Get(key) != null)
{
value = appSettingsReader.GetValue(key, typeOfObject);
}

return value;
} // GetValue

This is fine for production code however when executing unit tests we need to make sure the value is loaded directly from the app.config file rather than through the AppSettings or AppSettingsReader class. We could do this by

Set a value within the AppDomain to indicate a unit test is being executed e.g. AppDomain.CurrentDomain.SetData(“UnitTestExecuting”,true);
Modifying the existing GetValue method to check the app domain value UnitTestExecuting. If this is enabled then extract the configuration value using the steps 1 to 3 above directly from the file. Otherwise use the existing code which uses the AppSettings class

There should only be a slight performance impact for unit test code which is fine, but production code should execute the same. This will obviously require that client code uses the GetValue method rather than the classes in the ConfigurationSettings namespace directly.

VBScript plug ins for ASP applications

2006-02-24T08:31:00.000-08:00

Recently one of the developers in my team was developing some custom code in a COM table rendering object we've built for an ASP web application. The object essentially takes source SQL with added meta data and creates a HTML table which is returned into the calling ASP application, and then written out using Response.Write.

The design required some very custom presentation within one of the table cells. The table rendering object is quite generic ( knowing little about specific tables in the database schema ) and the custom presentation task required work on very specific tables. Custom work is essentially associated with each table cell using meta data tags such as "SubordinateList(%UserID%)". Assume the table object is rendering a list of users and if a user is a manager displaying a comma separate list of subordinate user names.

Although this example is contrived (as our real example would require a lot more explanation) the design issue is the same. The obvious easy approach would be to simply add handling within the table rendering object in a switch statement to match on the start of the tag "SubordinateList" and then perform some custom processing in SQL to find the list of sub ordinates. The list would then be converted into a comma separate string which would be added to the table cell. This business logic ideally should not be contained within a generic table rendering engine.

You may have seen plug in design patterns which support the extensiblilty we are trying to achieve here. Typically this would require the creation of a well defined interface e.g. IValueResolver which is defined and called in the table rendering object. The implementation of different IValueResolver objects could be created in one or many plug in components. The IValueResolver interface could only require one method such as GetValue which takes a row, column and table data.

Although ASP, using VB script, doesn't support strongly typed interfaces we can still use the late bound IDispatch support in VBScript to implement this design. We could create a VB script class on the ASP page such as

Class SubordinateListValueResolver
Dim UserColumn

Public Function GetValue(row, col, tableData)
' Work to retrieve the list of subordinates from the user specified
' in the column UserColumn in the row from the tableData data source.
End Function
End Class

The plug in class could then be instantied on the ASP page and passed to the table rendering object using code such as

Dim oValueResolver
Set oValueResolver = New SubordinateListValueResolver
oValueResolver.UserColumn = 4 ' Means the 4th column in the result
oTableRenderer.AddValueResolverPlugIn(oValueResolver, "SubordinateList")

The table rendering object can then determine whether any plug ins exist using the meta data tag passed when the data source SQL is specified. e.g. if a matching plug in object is found for the tag SubordinateList the VBScript object of type SubordinateListValueResolver will be returned. The GetValue method can then be called directly passing the row, column and table data from within the table rendering object. i.e.

oCellResolvedValue = plugin.GetValue(row, col, tableData)

Although designs which use well defined interfaces and contain the plug in code in compiled languages are preferrable to this approach, it does demonstrate how easy a simple plug in mechanism can be developed for a script based application.

Remote debugging - Release build optimisations

2006-02-14T03:20:00.000-08:00

I recently had to diagnose an issue which required remote debugging the application as no exception was raised and therefore no dump could be obtained and analysed. The issue was caused by an error code deep down in the call stack being returned and ignored a few levels up the call stack :(

Whilst this was trivial to debug using the windbg client and dbgsvr on the server ( both part of the debugging tools package ) I found in some cases, particularly where the error code was being returned, that local variables couldn't be seen. The message <Memory access error> was displayed in the locals window for each variable.

Our standard process is to generate debugging symbols with each release build by simply selecting the generate debugging symbols option (http://msdn.microsoft.com/library/en-us/dnvc60/html/gendepdebug.asp). To accurately see variables in both the watch and locals windows you need to make sure you disable optimisations i.e. through Project Settings / C++ / Optimisations - Set this to Disable ( Debug ).

Obviously this should only be disabled for ad hoc builds as you will most likely want to enable all optimisations in your standard builds you deploy. It's hard enough trying to get our applications to be as performant as possible without disabling optimisiations which the compiler gives us automatically :)

Database filtering of documents - performance issues

2006-02-10T03:18:00.000-08:00

Our application has a requirement to filter the documents a user can see based on various criteria e.g. security / approval rights. A user may only wish to see documents which they can approve or have approved. This would seem to be a common pattern for a lot of business applications.

To implement this our business objects create a list of document identifiers which are then appended to a SQL statement which selects the data and filters the documents returned. We used to use the IN clause to filter the documents returned using a pattern such as

SELECT * FROM PurchaseOrders WHERE PurchaseOrderID IN ( CommaSeperatedListOfIDs )

where the list of order ids was generated from the business object. After usage of the application quickly increased we encountered a stack overflow exception issue http://support.microsoft.com/kb/q288095. To resolve this we changed our approach to use temporary tables i.e. insert all the IDs from the business object into a temporary table and then use this to filter the main query e.g.

SELECT * FROM PurchaseOrders WHERE PurchaseOrderID IN ( SELECT ID FROM UniqueTempTableName )

The IDs are inserted into the temporary table using batch INSERTS e.g. in batches of 20 statements at a time. However when upwards of 10000 document identifiers of inserted into the table, it can take over 5 seconds to do this. E.g. an ASP screen which presents a list of invoices takes 15 seconds of which 11 seconds is attributed to the time to insert into the temporary table. Over the last day I've been exploring ways to minimise this.

Extended stored procedure
My first thoughts were creating an extended stored procedure which would create an in memory result set using a comma separated list of identifiers. The extended stored procedure would be called from a UDF which would return a table variable which could be used in a select statement e.g.

CREATE FUNCTION dbo.Tfn_IDs(@listIdentifiers text)
RETURNS @idTable TABLE (id int )
AS
BEGIN
INSERT INTO @idTable
EXEC master..xp_IDContainer(@listIdentifiers)
RETURN
END

However unfortunately EXEC cannot be used as the source for an INSERT INTO statement.

OPENXML
I then realised I maybe able to generate an XML representation of the identifiers using the extended stored procedure e.g. in the format

<values>
<value id="84">
<value id="85">
</values>

and then use the XML in a SQL OPENXML statement using

INSERT INTO @IDTable
SELECT ID
FROM OPENXML (@hDoc,'/Values/Value')
WITH (ID int)

UDF Only
I then went back to an approach without using an extended stored procedure and only using a UDF. The UDF could parse the comma separated list of values and insert into a table variable which would be returned. I won't outline the solution in detail but it basically encompasses using CHARINDEX to find the start and end positions of each identifiers using the comma as the delimmiter. It would be interesting to compare the timings of this and the OPENXML approach. The XML approach could be slower because of the more verbose structure of the data and the cost of creating the XML internally in SQL Server and performing XQuery expressions however it maybe offset by the improvement in speed of using a single INSERT INTO statement rather than multiple INSERT INTO statements for each identifier found and parsed in the UDF only approach.

However after increasing the size of the input to this solution it quickly became apparent that CHARINDEX truncates text data passed to varchar(8000) and so won't work with long text types which contains 10000 identifiers with an average of 3 characters each.

Back to a UDF + extended stored procedure
To overcome the issues faced in the UDF only solution an extended stored procedure would probably be the logical place to put all the parsing logic of the comma separated string into values used for an insert statement. However instead of using the resultset of an EXEC call as source to an INSERT INTO statement, which weve shown isn't possible, we could use an output parameter to get each identifier. Psuedo TSQL could look like

@sessionToken = xp_IDContainer_ParseValue('1,2,3')
WHILE xp_IDContainer_EndOfResults(@sessionToken) <> 0
BEGIN
@ID = xp_IDContainer_GetNextID(@sessionToken)
Insert id into table variable
END

Has anyone solved this problem using the approaches I've highlighted or other approaches which are successful with large volumes of data.

Debugging - Data grid exceptions

2006-01-12T08:08:00.000-08:00

I recently noticed an issue in one of our .NET controls which contains a Data grid which was causing values editted to be reverted back to their original values without any events being raised or any exceptions being raised to the client.

However if you enable all CLR exceptions to break into the debugger you will see an exception e.g. in this case "1,444.66 is not a valid value for Decimal". You can then get the call stack at this point and determine where the code is failing.

The data grid seems to handle quite a few exceptions silently which can cause confusion sometimes. If you are experiencing any odd behaviour with the Data Grid I strongly recommend you enable all exceptions to break into the debugger. To do this from Visual Studio .NET perform the following

Select the Debug menu
Select Exceptions
Select Common Language Runtime Exception.
Change "When the exception is thrown" to "Break into the debugger". This will apply the setting to all exception types unless you've overridden this.

Visual Basic Production Debugging

2005-08-07T07:00:00.000-07:00

A Visual Basic (VB) error is usually raised by either an explicit call to Err.Raise or by calling another COM object which raises an error. In both cases VB provides similar error handling to VB Script through the On Error Resume Next and On Error Goto statements. In VBScript production debugging I discussed how to generate dumps and obtain error information when VB Script errors occurred. Obtaining the same information when a VB error is raised is raised takes a bit more work.

You may be using VB components from within an ASP web application to perform a lot of business logic and may find some errors are being raised but ignored by either the components or the ASP script. Trapping these errors in a production environment is therefore very useful.

The main difference between VBScript and VB errors are VB errors are raised as true native exceptions (exception code c000008f ). To trap VB script errors break points had to be placed within the scripting runtime module. In this post I will walk you through the commands you need to execute on a live or full dump.After we’ve gone through the commands we’ll put together an adplus script to capture this information which you can use on production servers.

When the dump is open switch to the VB thread - you can use ~*kb to see which threads have MSVBVM60. e.g.

0:000> ~20s

Determine the index of the EBTHREAD object in thread local storage (TLS).

0:020> dd msvbvm60!g_itlsebthread l1
6aae0ec0 0000001e

Find the location of this thread's TLS based on the thread environment block (TEB). If you want to dump the Err object for other VB threads just get the TEB for those threads.

0:020> ~. 20
Id: 758.5bc Suspend: 1 Teb: 7ffa8000 Unfrozen
0:020> ?7ffa8000 +0xe10 Evaluate expression: 2147126800 = 7ffa8e10

Get the location of our EBTHREAD object based on the index into TLS

0:020> ? 7ffa8e10 + 0000001e * 0x4
Evaluate expression: 2147126920 = 7ffa8e88
0:020> dd 7ffa8e88 l1
7ffa8e88 0977ef58

Get the address of the EXCEPINFOA member variable based on its offset into EBTHREAD

0:020> ?0977ef58+0x78
Evaluate expression: 158855120 = 0977efd0

Now dump the EXCEPINFO data

0:020> dd 0977efd0 l8
0977efd0 00000000 250613ec 250314da 00000000
0977efe0 00000000 00000000 6aaa8bcf 80004005

At this point you have all of the information in the VB Error object. In this particular instance we have the bstrSource, bstrDescription, the pfnDeferredFillIn, and the scode. The pfnDeferredFillIn is simply a function in VB that is called as needed to get more information about the error object (ie. the description). This is called when the VB component tries to get the data (ie. access the Err.Description property) which is why the information may not be filled in sometimes.

For this dump we can see the error source

0:020> du 250613ec
250613ec "Microsoft OLE DB Provider for SQ"
250613ec "L Server"

and the description

0:020> du 250314da
250314da "[DBNETLIB][ConnectionOpen (Conne"
250314da "ct()).]SQL Server does not exist"
250314da " or access denied."

and the error number

0:020> ? 80004005
Evaluate expression: -2147467259 = 80004005

As a full dump is required to access this information you can typically run into performance issues on the server if many exceptions are being generated due to the size of the files. It would be far more effective if we could use an adplus script to extract the error source, description, and number from a process at runtime. This avoids the need to create dump files and adds all the relevant information in the adplus log file. The script is shown below

<ADPlus>
<Settings>
<RunMode> CRASH </RunMode>
</Settings>
<Exceptions>
<NewException>
<Code> c000008f </Code>
<Name> VBError </Name>
</NewException>
<Config>
<Code>c000008f</Code>
<CustomActions1>
.echo VB error occurred.;
.echo ;
r $t1 = poi($teb+0xe10+poi(MSVBVM60!g_itlsebthread)*4)+0x78;
.echo Source ;
.if poi($t1 + 0x4)=0 { .echo Unknown } .else { du poi($t1 + 0x4) };
.echo Description ;
.if poi($t1 + 0x8)=0 { .echo Unknown } .else { du poi($t1 + 0x8) };
.echo Error number ;
?poi($t1 + 0x1c);
</CustomActions1>
</Config>
</Exceptions>
</ADPlus>

The script uses some useful commands such as $teb to extract the TEB memory address and .if / .else to allow commands to be conditionally executed. It also sets a pseudo register $t1 to improve the performance of the error extraction, otherwise this data would have to be evaluated three times.

For some VB errors where the description is not filled in you can open VB6 and in the immediate window just type 'error ’. If you don’t have access to debugging symbols on the server you can use the same techniques I discussed in Setting breakpoints without symbols. In this case you should find the memory address of the symbol MSVBVM60!g_itlsebthread and replace it with the memory address e.g. MSVBVM60 + 0x12345678.

Setting breakpoints when production servers have no access to symbols

2005-07-07T07:13:00.000-07:00

In some cases a production server may not

Have access to the internet to use the Microsoft symbol server.
Access to symbol files using a UNC path.

In both cases it is impossible to set break points using symbolic names in either a live debug session or when using adplus. This is clearly a problem for the adplus configuration script I discussed in VB Script production debugging.

If symbols aren’t available there is another solution by adding break points set using a memory address location. In the previous blog the adplus config file was setting a breakpoint on the function vbscript!CScriptRuntime::RecordErrorContext. To find out the memory address for the breakpoint we need to determine what the offset to vbscript!CScriptRuntime::RecordErrorContext is in vbscript.dll.

First we need to take a dump of the process on the production server and load it into windbg on a machine which does have access to a symbol store. Open the dump and enter the x command to find the address of the function

0:031> x vbscript!CScriptRuntime::RecordErrorContext
734753cf vbscript!CScriptRuntime::RecordErrorContext =

Then we need to find the module address for vbscript.

0:031> lmm vbscript
start end module name
73460000 734c5000 vbscript

Subtract the module start address from the symbol address

0:031> ?734753cf - 73460000
Evaluate expression: 86991 = 000153cf

Now we have our offset we can use it to set a break point either during a live debug or in the adplus config file. The dump I used in the example above was using 5.6.0.8515 of vbscript.dll. The Address element of the adplus configuration file should be modified from

vbscript!CScriptRuntime::RecordErrorContext

vbscript + 0x000153cf

Effective Production Debugging MSDN Article

2005-07-03T02:43:00.000-07:00

Following my recent production debugging MSDN article (http://msdn.microsoft.com/msdnmag/issues/05/07/Debugging/default.aspx.
I've written a number of posts on techniques I wanted to cover in the article but couldn't as I ran out of space.

I've already written on topics such as creating dumps through terminal services and VB Script debugging however If there's anything you'd especially like to see let me know and I'll try and cover it.

VBScript Production Debugging

2005-06-29T14:33:00.000-07:00

An error typically occurs in an ASP application running VB Script when either

An On Error Resume Next statement is called prior to executing some script. A number of method calls are then made which can potentially raise errors. After each statement is completed the Err.Number property is usually checked and action taken such as logging the error and providing user friendly messages.
No On Error Resume Next statement is called prior to executing some script . In this case any statements that follow will halt the processing of the script and return a runtime application error to the user.

Whilst the first case is the preferred approach to handle errors gracefully most developers typically do not check most statements for errors. If Err.Number is not checked errors can be silently ignored by the server side script, and then consequently cause other errors which are logged. It is therefore essential to be able to log information on the original error, namely the source, description and number.

Unlike languages such as C++ / VB or .NET exceptions aren’t raised to the debugger from scripting engines such as VBScript. Consequently we can’t use adplus to capture a dump file when a specific type of exception occurs. However when an error is raised a number of functions are called within the scripting runtime.

vbscript!COleScript::OnScriptError. This is called when a script occurs in the 2nd case above
vbscript!CScriptRuntime::RecordErrorContext. This is called to record an error in the scripting runtime when On Error Resume Next has been called.

Although we can’t use adplus to create a dump file on an exception (because none is raised) we can use it to create a dump when a specific break point is reached. Opening the dump file shows the following call stack

0:023> kvn
# ChildEBP RetAddr Args to Child
00 0148f4a8 733183bf 0003aa70 000b5f48 00000000 vbscript!CScriptRuntime::RecordErrorContext
01 0148f4c4 73318320 0003aa70 0148f5cc 00000010 vbscript!CSession::RecordExcepInfoAndClear
02 0148f5b8 73304b01 0003a9c0 000b5f48 60030001 vbscript!InvokeDispatch

If we dump the first argument plus 8 we can see the EXCEPINFO structure which is passed to this method. EXCEPINFO is the automation structure which defines an error i.e.

struct EXCEPINFOA, 9 elements, 0x20 bytes
+0x000 wCode : Uint2B
+0x002 EXC_unused1 : UChar
+0x004 bstrSource : Ptr32 to Void
+0x008 bstrDescription : Ptr32 to Void
+0x00c bstrHelpFile : Ptr32 to Void
+0x010 dwHelpContext : Uint4B
+0x014 EXC_unused2 : UChar
+0x018 pfnDeferredFillIn : Ptr32 to Void
+0x01c scode : Uint4B

Reviewing the dump we see

0:023> dd 0003aa70 + 8
0003aa78 00000000 000b85bc 00121254 00000000
0003aa88 000f43e8 00000000 00000000 800a0063
0003aa98 00000000 00000000 00000000 00000000
0003aaa8 00000000 00000000 000a0004 0008012a

To find the source and description we can use the du command i.e.

0:023> du 000b85bc
000b85bc "MyVBClass"
0:023> du 00121254
00121254 "Failed to get supplier invoice, "
00121294 "Reason=Failed to open connection"
001212d4 ", reason=Object required"

However to get this information a full dump needs to be created. If many script errors occur this maybe unworkable in a production environment. A more elegant approach involves using a custom action in an adplus config file to extract the source, description and number and add to the log file adplus produces. i.e.

<ADPlus>
<Settings>
<RunMode>CRASH</RunMode>
<ProcessID><Put process id in here></ProcessID>
</Settings>
<Breakpoints>
<NewBP>
<Address>vbscript!CScriptRuntime::RecordErrorContext</Address>
<CustomActions>.echo -----;.echo VB Script On Error Resume Next error occurred.;.echo;.echo Source:;du poi(poi(esp+4)+c);.echo Description:;du
poi(poi(esp+4)+10);.echo Code:;?poi(poi(esp+4)+24);g</CustomActions>
<Actions>Log;Stack;</Actions>
<ReturnAction>G</ReturnAction>
</NewBP>
<NewBP>
<Address>vbscript!COleScript::OnScriptError </Address>
<CustomActions>.echo -----;.echo VB Script On Error Resume Next error occurred.;.echo;.echo Source:;du poi(poi(esp+4)+c);.echo Description:;du
poi(poi(esp+4)+10);.echo Code:;?poi(poi(esp+4)+24);g</CustomActions>
<Actions>Log;Stack;</Actions>
<ReturnAction>G</ReturnAction>
</NewBP>
</Breakpoints>
</ADPlus>

The most interesting line in the config file is

.echo Source:;du poi(poi(esp+4)+c);.echo Description:;du poi(poi(esp+4)+10);.echo Code:;?poi(poi(esp+4)+24);

The source and description are output as strings using the du command. From the parameter poi(poi(esp+4)+c), esp+4 is the stack pointer + 1 DWORD which equals the first parameter to the function were breaking on ( RecordErrorContext or OnScriptError ). This is dereferenced using the poi operator from which we then add 0xC to get to the offset of the Source member in the EXCEPINFO structure. The Source member is just a pointer to a string so poi is used again to dereference it. Description is handled the same and the error code is simply output using the ? evaluate expression command.

When a VB Script exception occurs you get all the relevant information provided in the log file along with any other exceptions which are raised i.e.

VB Script On Error Resume Next error occurred.

Source:
0017edf4 "MyVBClass"
Description:
0017eeac "Failed to get supplier invoice, "
0017eeec "Reason=Failed to open connection"
0017ef2c ", reason=Object required"
Code:
Evaluate expression: -2146828189 = 800a0063

Getting dumps for apps failing on start up using terminal services

2005-06-16T03:12:00.000-07:00

In my previous post http://mattadamson.blogspot.com/2005/05/getting-dumps-for-apps-failing-on.html I discussed the use of gflags to invoke a debugger to start when a process starts and then sending debugger commands to create dumps when exceptions occur. However if your trying to debug against a process run in session 0, e.g. a web application or service, this technique will not work. As most support teams will probably be using terminal services to access production servers clearly we need a technique to get the dumps.

Remote debugging is commonly used to debug across two machines i.e. setting up a debugger on the server and using a debugging client to connect to it using named pipes or tcp. However there is nothing stopping us using the same machine to host the debugging server and the client. We need to set up a debugger server which runs in session 0 and a debugger client which runs in the terminal services session i.e. 3 in our example.

As before open gflags and specify the image module name you need to debug however instead of simply entering the path to the debugger you should specify some command line options to tell windbg to start as a debugging server. Enter the following in the debugger field

C:\Debuggers\windbg.exe -server tcp:port=123 –Q

The server option creates a debugging server and the tcp:port option specifies the port number. The –Q option suppresses the “Save workspace” dialog which is important if the debugger is running in session 0 and can’t interact with the user. Note the command line options are case sensitive so be sure to copy this exactly as it appears. As soon as the process your debugging starts the windbg debugee should be loaded in session 0 and broken in on the LdrpInitializeProcess method.

To control the debugger server start a new instance of windbg as the debugging client. Press Ctrl+R to connect to a remote debugger session and enter the following

tcp:server=MACHINENAME,port=123

When you connect the windbg instance will connect to the remote debugger and allow you full control to enter commands e.g. you can enter

Sxe *
g

to break on all exceptions and then continue. You can also send the same commands to create dump files as mentioned in my previous post. Or you could simply use .dump to create a dump when a specific exception is raised e.g.

.dump /ma c:\debugger\ExceptionDump.dmp

If you look at task manager you should also see two instances of windbg open. If you select the “Session ID” column you can see the different terminal services sessions used for each instance.

Choosing between an ascending or descending index

2005-05-28T07:39:00.000-07:00

I was recently asked whether making an index ascending or descending would improve query performance in SQL Server. In almost all cases the direction would make absolutely no difference to the speed of retrieval. Performance of queries is usually best measured by examining the number of logical page reads required to find the data.

Searching a table for a specific row
Any search across a single non-clustered / clustered index to locate a specific row would always result in a similar number of pages read. In the non-clustered case the time to retrieve the row would be navigating the index pages through the binary trees root index page and then through the intermediate level index pages to find the leaf level index page which matches. Then if no clustered index existed on the table the index page would point to the data page which would then be read to obtain the data.

If a clustered index existed on the table the leaf level non clustered index page would point to the clustered index key. The clustered index key would then be examined in a similar way to the non clustered index to find the leaf level index page. However for a clustered index the data exists on the leaf level index page so no further page reads would occur.

Searching for a range of rows
Even cases when searches are performed for a range of rows e.g. ID BETWEEN 100 AND 105 direction of the index would make no difference. A single column index can be scanned well in both directions because the index pages at a given level are linked by next and prior pointers pointing to the next and previous index pages at that level. The query optimiser may decide to search for the data either by starting at ID 105 and moving backwards through the index pages, or from 100 and working forwards.

When does direction make a difference?
The direction you specify when creating an index is used to determine the order of the leaf level index pages. E.g. names Adamson, Bamber, Doherty, Roche would appear exactly in that order in an ascending index, and in a descending index they would appear as Roche, Doherty, Bamber and Adamson.

The only case I can think of when the direction would have the most impact is a query where a composite index was used and a query was submitted which
Covers all columns used in the index. This means all fields specified in the SELECT statement can be found in the index page.

Covers all columns used in the index. This means all fields specified in the SELECT statement can be found in the index page.
Specifies a different direction in the ORDER BY clause in the SELECT statement of the query to the direction the index was created in.

Consider a query

SELECT SupplierNo, CreatedDate
FROM Invoices
ORDER BY SupplierNo, CreatedDate DESC

WHERE CreatedDate BETWEEN ‘1/5/05’ AND ‘10/5/05’

If either two single non clustered indexes or a single composite index on SupplierNo ASC, CreatedDate ASC was created a Sort operator would still be used in the execution plan of the query. If we can remove the sort operator the query time should be reduced.

If you created a single composite non clustered index on SupplierNo ASC, CreatedDate DESC the index pages could be navigated without the sort operator being used and only an index scan performed. As the index is covered because all the columns requested existed on the index pages then no data pages would have to be navigated from the leaf level index pages.

If you decided to change the query and add a column which didn’t exist on the index such as a reference number, the data page would have to be located for each matching row on the index page. Depending on the total number of index pages and data page scans required to get all the data, the query optimiser may even decide not to use an index and simply perform a table scan.

This example cites the usefulness of the query execution plan to show you how slow a specific query is. This is the only effective method to diagnose potentially slow running queries and improve the performance in a methodical manner. You can either show an execution plan for a query directly in query analyser or capture the event from a SQL profiler trace in a production environment. Simply executing the query in the query analyser window and trying to time it manually will not yield good results unless the speed differences before and after are that noticeable.

Getting dumps for apps failing on start up

2005-05-18T00:55:00.000-07:00

Applications such as services or out of process servers may generate errors on start up. Adplus provides a spawn option to launch processes and attach the debugger on startup. Unfortunately the spawn option cannot be used because the process is either started by the Service Control Manager (SCM) or the COM runtime loads the module. For these type of applications it is still possible to create dump files but the process is a little convoluted.

What we need is some mechanism to attach to the process as soon as it launches but before the error is raised. The debugging tools package provides a tool called gflags which allows us to specify a number of debugging options both globally and for specific applications. One of the options allows us to configure a debugger which is attached as soon as the process starts.

Enter the full image name in the image file page then press tab to enable all the other controls. Select the full path to the windbg debugger in the Debugger field which can either be cdb.exe or windbg.exe. Now start the failing application either by forcing the client to create the COM object or start the service. The debugger should then be launched ready to recieve commands.

Now we've broken in the debugger we need to send all the commands adplus usually sends to configure specific exceptions to generate dump files. The debugger command "sxe" controls the action the debugger takes when an exception occurs in the application being debugged. When you use an Adplus XML config file to configure what exception codes generate dumps for 1st or 2nd chance exceptions Adplus sends a number of these commands to the console debugger (CDB) e.g. one is shown below with some detail omitted.

sxe -c @"dump -u /mdi /c 1stchanceAVSCANmini.dmp; GN"
-c2 @"dump -u /ma /c 2ndchanceAVSCANfull.dmp; GN" av

The last parameter specifies the exception code ( access violation in this case ) and the -c and -c2 options specify the actions to take on first and second chance exceptions respectively. The /ma option effectively takes a mini dump with all additional options which is more commonly known as a full dump. However if you have a lot of different exception codes to configure this process can be very tedious and error prone. Fortunately when you run adplus it generates a file with a CFG extension in the Crash_.. folder. This file contains a list of all commands needed to be sent to CDB to configure all exception codes.

The most effective approach is to run adplus against any other process e.g. notepad, to generate a CFG file using the options you specify in an adplus XML config file. When the process is started and windbg breaks in you can use the "$<" debugger command with the full path of the cfg file i.e.

D:\Development\Debugging\Debugging Tools for Windows\DebuggerCommands.cfg

This will send all the necessary commands in the CFG file to the debugger to create the dump files for first and second chance exceptions. It is worth editing the CFG file first to modify the application from notepad in this example to the name of your app. Then simply press F5 or enter the g command to allow the debugger to continue and the dump files should be created.

Should the GAC always be avoided?

2005-05-15T00:49:00.000-07:00

Currently we use the Global Assembly Cache (GAC) to install shared assemblies which are called by a number of different applications; however articles such as the infamous “Avoid the GAC” http://www.sellsbrothers.com/spout/#Avoid_the_GAC article have made me question the usefulness of this approach.

To summarise our deployment we have perhaps 40 or so assemblies which are referenced by a number of different applications namely.

A windows service.
An ASP application.
A thick client windows application.
A .NET console application.

All applications use the same set of core assemblies to perform most of the functions e.g. core invoicing and ordering components as well as components which are applicable across many different functional areas.

Every time we rebuild and deploy a new component e.g. assembly A which has upgraded from say 1.0.8.0 to 1.0.9.0 we ship a new publisher policy assembly which redirects 1.0.0.0 to 1.0.9.0 references to 1.0.9.0 and obviously also re installs the new updated assembly into the GAC. E.g. 1.0.9.0 may fix an issue in the database where a stored procedure is being used with an incorrect parameter order causing data integrity issues.

This deployment works well in that components which are shared amongst all applications only need to be installed in one place. In the future we may provide breaking components with major functionality changes e.g. we introduce a new assembly A 2.0.0.0 version which may have breaking changes. In this case we envisage we may update the publisher policy with multiple redirections setup such as

1.0.0.0 - 1.0.9.0 ---> 1.0.9.0
2.0.0.0 - 2.0.2.0 ---> 2.0.2.0

In this way we can support side by side versioning of both sets of assemblies. We make a statement with the publisher policy that version increments containing changes across 1.0.0.0 – 1.0.9.0 are compatible and are needed for all applications referenced assemblies in this range however applications which reference new assemblies in the range 2.0.0.0 + are breaking and should reference a different assembly.

Currently we use the GAC to deploy all our assemblies and it seems to work well. The GAC was chosen because of the following benefits in order of importance:

Ease of deployment. When shared assemblies are updated they only have to be updated in one location as opposed to four directories namely system32, windows\system32\inetsrv, c:\program files\Tranmit\Transaction Engine and C:\program files\Tranmit\Products\SprintLog. Deploying the assemblies to more than one location is obviously more error prone and an additional burden on an administrator installing patches.
Performance. When strongly named assembly files are loaded from a location other than the GAC the CLR compares hash values when the assembly is loaded. A hash of the file is performed every time an application executes. In addition any referenced assemblies in that assemblies manifest are also hashed and compared. This performance hit is required to ensure that the assembly file’s content hasn’t been tampered with.
Security. The GAC is only accessible to those users with sufficient privilege.
Traced references. Assemblies installed in the GAC can be added with a reference to the application. This prevents assemblies being removed accidentally and breaking client applications dependent on them.

Chris Sells author of the “Avoid the GAC” article actually agreed the GAC was the best choice in our case however I am keen to hear other peoples’ opinions on this interesting topic. I would certainly consider changing to XCOPY deployment if there is good reason.