Software Expressions

Convert Class Libary to Test Project

Troy — Wed, 04 Feb 2009 15:53:27 +0000

Converting a class library in Visual Studio 2008 to a Test Project is as easy as opening up: the .vbproj or .csproj in NotePad and adding a ProjectTypeGuids element under the PropertyGroup element. This should be around the AssemblyName element.

{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{F184B08F-C81C-45F6-A57F-5ABD9991F28F}

There is one more catch. If you convert multiple Class Libraries, the second guid must be unique. Otherwise you will receive the “A project with the same name is already opened in the solution” error when adding the second Test Project or when you open the solution.

Mounting virtual hard disks at the command line

Troy — Wed, 03 Dec 2008 14:26:23 +0000

Backing up the data on my virtual PC has always been a little painful. I use a NAS Server (NASLite) with SecondCopy to perform the backups in the early hours of the morning. However, it’s very likely that my VirtualPC is shutdown, and so SecondCopy running in the VirtualPC only gets to perform the backup every few weeks - and that’s not guaranteed. With this command line utility it’s possible to include a batch file in SecondCopy which mounts the VHD and then performs the backup. It’s perfect. http://technet.microsoft.com/en-us/library/cc708295.aspx
…and then there’s also this article which describes compacting your VHD outside of VirtualPC using the VHD mount utility: http://vscommunity.com/blogs/virtualzone/archive/2007/01/17/three-steps-to-vhd-compaction-with-virtual-server-2005-r2-sp1.aspx

Troy — Tue, 18 Nov 2008 17:33:03 +0000

I am always dazzled with your consistency and your practical approach to issues. You surely must be brilliant for being able to create the architecture we have - QA Lead

Button theory

Troy — Fri, 14 Nov 2008 22:04:30 +0000

Buttons on Windows applications have been around since the first version of Windows. One would think that the patterns for using buttons would have been firmly established by now. In particular rules for disabling and enabling buttons.
Concerning visibility:

Buttons should never be made invisible based on some transient state. i.e. Don’t hide the button when the user is disconnected from the internet, or hide the button when the order has already been placed.

Users should expect to see buttons in the same order and same location for a given context.

Hiding buttons is good for things like user permissions. If you don’t have the permission to delete the customer information, there’s no need to see the button ever.

Concerning enable and disable:

If a button is no longer available due to a transient state, or state of the underlying object, disable it. This keeps the screens and button layout consistent.

Don’t disable the button without giving the user a reason. This may seem strange, but you can display a tooltip on a disabled button. Let the user know why they’re not able to perform the action. Don’t leave them guessing.

I suppose there could be exceptions to this. In some cases it may really be obvious.

If it’s not really obvious and you can’t display a tooltip on a disabled button, leave the button enabled and display a pop-up message indicating why.

The more rules there are that control whether or not the button is available, the more reasons there are to clearly communicate this to the user.

Serializing images

Troy — Fri, 14 Nov 2008 21:50:26 +0000

The System.Drawing.Image class is not serializable by default. This class can be used to serialize images as well as load the image from the IDataRecord.

   1:  Imports System.Drawing

   2:  Imports System.Runtime.Serialization

   3:  Imports System.Drawing.Imaging

   4:  Imports System.IO

5:

   6:   _

   7:  Public Class SerializableImage

   8:      Implements ISerializable

9:

  10:      Private _master As MasterEnum = MasterEnum.Image

  11:      Private _imageBuffer As Byte()

  12:      Private _image As Image

  13:      Private _serializationFormat As ImageFormat = ImageFormat.Png

14:

  15:      Public Property Image() As Image

  16:          Get

  17:              If _master <> MasterEnum.Image Then

  18:                  CreateImageFromBuffer()

  19:              End If

  20:              Return _image

  21:          End Get

  22:          Set(ByVal value As Image)

  23:              If (value Is _image) Then Return

  24:              _image = value

  25:              ‘The image becomes the master once the setter is used

  26:              _master = MasterEnum.Image

  27:              _imageBuffer = Nothing

  28:          End Set

  29:      End Property

30:

  31:      Private Sub CreateImageFromBuffer()

  32:          ‘Once this method is called, the image becomes the master

  33:          _master = MasterEnum.Image

  34:          ‘Check if there is a buffer

  35:          If _imageBuffer Is Nothing OrElse _imageBuffer.Length = 0 Then

  36:              Image = Nothing

  37:          Else

  38:              ‘http://support.microsoft.com/?id=814675

  39:              Using s As New MemoryStream(_imageBuffer)

  40:                  If Not s Is System.IO.Stream.Null Then

  41:                      Dim tempImage As Image = _

  42:                          DirectCast(Bitmap.FromStream(s), Bitmap)

  43:                      Image = New Bitmap(tempImage)

  44:                      Dim g As Graphics = Graphics.FromImage(tempImage)

  45:                      g.DrawImage(Image, New PointF(0, 0))

  46:                      g.Dispose()

  47:                      tempImage.Dispose()

  48:                  End If

  49:                  s.Close()

  50:              End Using

  51:          End If

  52:      End Sub

53:

  54:      Private Property ImageBuffer() As Byte()

  55:          Get

  56:              Select Case _master

  57:                  Case MasterEnum.Image

  58:                      ‘Used for serialization

  59:                      Return CreateBufferFromImage()

  60:                  Case MasterEnum.Buffer

  61:                      Return _imageBuffer

  62:              End Select

  63:              Return _imageBuffer

  64:          End Get

  65:          Set(ByVal value As Byte())

  66:              If (value Is _imageBuffer) Then Return

  67:              _imageBuffer = value

  68:              ‘The Buffer becomes the master

  69:              _master = MasterEnum.Buffer

  70:              _image = Nothing

  71:          End Set

  72:      End Property

73:

  74:      Private Function CreateBufferFromImage() As Byte()

  75:          If _image Is Nothing Then Return Nothing

  76:          ‘Convert the image to a byte array using the serialization format

  77:          Using stream As New MemoryStream

  78:              _image.Save(stream, SerializationFormat)

  79:              Dim result As Byte() = stream.ToArray()

  80:              stream.Close()

  81:              Return result

  82:          End Using

  83:      End Function

84:

  85:      Public Property SerializationFormat() As ImageFormat

  86:          Get

  87:              Return _serializationFormat

  88:          End Get

  89:          Set(ByVal value As ImageFormat)

  90:              _serializationFormat = value

  91:          End Set

  92:      End Property

93:

  94:      Public Sub New()

95:

  96:      End Sub

97:

  98:      Public Sub New(ByVal image As Image)

  99:          Me.Image = image

 100:      End Sub

 101:

 102:      Public Sub New(ByVal record As IDataRecord, ByVal fieldIndex As Integer)

 103:          Load(record, fieldIndex)

 104:      End Sub

 105:

 106:      Public Sub New(ByVal record As IDataRecord, ByVal fieldName As String)

 107:          Load(record, fieldName)

 108:      End Sub

 109:

 110:      Private Sub Load(ByVal record As IDataRecord, ByVal fieldName As String)

 111:          If record Is Nothing Then Throw New ArgumentNullException(“record”)

 112:          Load(record, record.GetOrdinal(fieldName))

 113:      End Sub

 114:

 115:      Private Sub Load(ByVal record As IDataRecord, ByVal fieldIndex As Integer)

 116:          If record Is Nothing Then Throw New ArgumentNullException(“record”)

 117:          If record.IsDBNull(fieldIndex) Then Return

 118:          ImageBuffer = DirectCast(record.GetValue(fieldIndex), Byte())

 119:      End Sub

 120:

 121:      Public Sub New(ByVal info As SerializationInfo, _

 122:          ByVal context As StreamingContext)

 123:          ‘Image

 124:          Dim data As Byte() = _

 125:              DirectCast(info.GetValue(“image”, GetType(Byte())), Byte())

 126:          If data IsNot Nothing AndAlso data.Length > 0 Then

 127:              ‘http://support.microsoft.com/?id=814675

 128:              Using s As New MemoryStream(data)

 129:                  If Not s Is System.IO.Stream.Null Then

 130:                      Dim tempImage As Image = _

 131:                          DirectCast(Bitmap.FromStream(s), Bitmap)

 132:                      Image = New Bitmap(tempImage)

 133:                      Dim g As Graphics = Graphics.FromImage(tempImage)

 134:                      g.DrawImage(Image, New PointF(0, 0))

 135:                      g.Dispose()

 136:                      tempImage.Dispose()

 137:                  End If

 138:                  s.Close()

 139:              End Using

 140:          End If

 141:      End Sub

 142:

 143:      Public Sub GetObjectData(ByVal info As SerializationInfo, +

 144:          ByVal context As StreamingContext) Implements ISerializable.GetObjectData

 145:          info.AddValue(“image”, ImageBuffer, GetType(Byte()))

 146:      End Sub

 147:

 148:      Private Enum MasterEnum

 149:          Buffer

 150:          Image

 151:      End Enum

 152:

 153:  End Class

How to make a bootable thumb drive virus scanner for NTFS

Troy — Fri, 14 Nov 2008 14:11:48 +0000

Some friends were experiencing problems with a virus (run32dll - possibly WhisperB-worm), while searching for the best approach to resolve this issue I came across this really simple and impressive article. I guess I’m rather old-school when it comes to viruses and haven’t dealt much with them for many years. Back in the DOS days you would create a boot disk with your virus software. You would boot up on the clean boot disk, and run the scan on C:. There was nothing else to it. It seems to be rather more complicated these days - and I’m not sure it needs to be. For one thing, how can you download virus scanning software and run it on a machine with a virus? How can you run a Windows virus scanning software on a machine where when Windows starts up it already has the virus loaded. This was evidently the problem in this case because the virus scanners that were downloaded picked up no viruses. Indeed the virus software had cleverly intercepted the scanner. It was obvious because during the scan the machine would reboot. The scanning software couldn’t even download all it’s updated virus signatures…
http://askthegeek.kennyhart.com/2005/12/how-to-make-bootable-thumb-drive-virus.html

Sync Services Headaches

Troy — Sat, 11 Oct 2008 02:50:47 +0000

On my current project we decided to leverage Microsoft Sync Services (since we were already using SQL CE for our local database in a smart-client solution with an Oracle back end, it seemed to be the logical choice). After a few months of tweaking and jumping through a few hoops to get SQL CE and Oracle to play together, we had a solution that appeared to work just fine… on the surface anyway. We decided to assess the solution we had and see how far down the road it would take us. This article describes our findings which are some inherent problems, some problems integrating with Oracle, some performance related, and some just because one size does not fit all.

Note: I will not assume that any or all of these hurdles cannot be overcome with some level of effort. On our project the effort to implement our own synchronization service far outweighed the cost of trying to work around these issues and in the end we still weren’t sure it would work.

System overview
Database Server: Oracle 10g
App Server: Sync Web Services hosted in IIS
Local Database: SQL CE

Synchronization characteristics
The database contains around 40-50 tables, some tables are download-only and others are unidirectional. The business rules ensure that only one user can edit a record at one time and the user must explicitly give up ownership before another user can edit the record. The data is what I would call document based, in that the user owns and works with a parent record and many detailed child records. Each parent record and it’s child records are self-contained. Because users cannot edit a record at the same time there is no need for merges (at least in theory). Data is only logically deleted which simplifies the sync process in that there is no need to the tombstone table.

How Sync Services works
Client-side change tracking: SQL CE has a hidden feature to track changes on rows; this feature is leveraged by Sync Services as long as the tables are created via Sync Services. Each time a row is modified a flag is set on a hidden field for that row.

Server-side change tracking: If you’re using SQL Server 2008, change tracking is integrated. SQL Server 2008 change tracking resolves some of the issues that be identified in this article. It’s change tracking is built into the SQL Server core engine and outperforms similar implementations using triggers. Change tracking also occurs per transaction which is harder to accomplish with triggers. If you’re unfortunate to be working without SQL Server 2008 (or Oracle in our case) you will need to create a couple columns for each table and implement a trigger for each table to be change tracked. The trigger simply updates a client id and timestamp when the record is modified.

Synchronization occurs via the Client and Server Providers. Each of these provide a set of methods which are used to perform sync operations. No transport mechanism is provided out of the box, however it’s a trivial task to publish the server provider methods in a web service.

A sync operation consists of looping through each table on the client, checking for changes, packing up the changes and sending them to the ServerProvider which applies the changes. This occurs for each table.

Tables can be synchronized at different frequencies by creating a separate service. For example lookup tables may only require synchronization once per day.

Tables can be grouped. All grouping does is ensure that a dependency tables are updated after their dependency. Customer would need to be updated before Address (for example).

The anchor: Each time a client requests data from the server, the server provides a timestamp which the client stores. On subsequent download requests the client sends its anchor to the server. The server queries the table in question to determine if there are any records with a timestamp greater than the supplied timestamp.

Transaction isolation: On the server, serializable or snapshot isolation is required for both upload and download operations to ensure that data does not change during the sync.

The first problem: no-change-sync takes 30 seconds
Sync Services must perform a separate check on each client and server table to check for changes. Each table check is a separate call to the web service. In our case it took 30 seconds to perform a no-change-sync. Ideally a no-change-sync should be sub-second. The client should “know” if it has any changes (just maintaining a single flag would suffice). For server changes, the client should ask “Are there any changes for me?” with a simple yes/no response from the server. Further on I will discuss why background sync is not feasible, but basically a 30 second no-change-sync means that the user is locked out of the system for 30 seconds when there aren’t even any changes.
The sub-second goal is particularly important on our project because of background sync. In most cases there will be no data changes and the number of hits on the server will be pretty high. 6000 users pinging the server every 30 seconds equates to 720,000 requests per hour (200 requests per second) on the server. It has to be fast.

No granularity on transactional units
A sync operation is an all-or-nothing deal. Because Sync services has no knowledge of the relationships on your data (other than order which is determined by sync groups), the longer the client waits to sync, the more data there is to sync in a single operation. The transactional unit in Sync Services is the entire mass of data that needs to be uploaded or downloaded. There is no way to break the data up into smaller chunks without knowing how the data is related. Of course you can sync your lookup tables separately and batch those into smaller chunks. But when the data is all related there are no other options. This inherent flaw is the cause of the next three problems.

Long running transactions
Each time an upload operation starts a transaction is started on the server and the transaction is closed when the last table is checked and uploaded. The longer you wait to sync, the more data you have to sync and the longer the transaction. Long running transactions can cause serious problems. The length of these transactions is also dependent on the user’s bandwidth and other conditions which leads to the next problem…

Client controlled transactions
Just off the bat this seems like a bad idea. What I mean by this is that it is the client that initiates the start of the Oracle (server) transaction and the client that determines the end of the transaction. The length of the transaction can be determined by: their bandwidth, how much data they have that’s not synchronized, or if they just decide to suspend their laptop in the middle of a sync operation. If user’s behave nicely and sync often, make sure they have adequate bandwidth and be sure to not terminate the application or suspend the laptop while a sync is in progress things may work just fine… but we all know this isn’t going to happen.

Sync operations are non-recoverable
By non-recoverable, I mean that if an upload or download fails part-way through, the sync operation must completely restart (again for tables without relationships it’s easy to continue where you left off). For data that has relationships to other data, ALL the data must be synchronized in a single operation and if there are any failures the sync operation must start from the beginning. If the user has an unreliable connection (low QoS), this problem could compound. Poor QoS means that the connection could be lost resulting in a hanging long running Oracle transaction. The user will possibly continue to add more data, and by the time they try to sync again, they have even more data to synchronize which means even less chance of getting all their data through, possibly leaving a few more hanging transactions before a successful attempt is made. These scenarios increase the network traffic and load on the servers and database.

Data integrity issues under load conditions
The recommended solution for Database Servers (outside of SQL Server 2008) is using triggers with timestamps. This solution is flawed right off the bat. The best way to describe this is with the help of the help of the following example. (Be sure to read the definition of anchor above before continuing). Assuming a parent table, “Project” and child table “Line Item.” Client A is performing an upload which will last for 15 seconds. Prior to upload a transaction is started. After 7 seconds of uploading all the project records are uploaded and just then Client B performs a download sync. Because the projects have not yet been committed the server returns no results (however, Client B’s anchor is updated to the 7 second mark). At 15 seconds client A has completed its upload and Client B performs another download – the only data that’s downloaded are the line items. Because Client B’s anchor was at 7 seconds only changes after the 7 second mark are retrieved. Of course this will result in FK Violations on the client because the Projects that the Line Item references don’t exist.

SQL Server 2008 solves this problem by (a) not using timestamps; and (b) by setting the version of the rows all to the same version which is based on the transaction.
Impossible to support background sync
Because the sync operations must scan ALL data and SQL CE does not support snapshot or serializable transaction isolation, it’s a lost cause even trying to attempt background sync. Once a sync operations starts the user cannot modify the data. If snapshot isolation were available, the sync operation could start a transaction and the data within the transaction wouldn’t change.

Troy — Fri, 25 Jul 2008 17:40:23 +0000

Please allow me to tell you again what a tremendous asset you are to this project. I told Jay again this morning that you are fabulous. This document is great. — Project Manager, Magenic

Troy — Thu, 24 Apr 2008 10:23:30 +0000

Troy is the solution architect and mentor for the Marvin Windows and Doors OMS product. OMS allows customers to configure, quote and purchase virtually any Marvin product. His major accomplishment has been the implementation of a business rule engine that has allowed Marvin to encode a decade of product knowledge. He also implemented a presentation framework, extended CSLA, and provided extensive mentoring to a development team scattered across three locations.

However, technical skills are only a single part of a successful engagement. The soft skills presented by Troy are extraordinary. This is best demonstrated in an email from Lisa Carlston, e-business manager of the Marvin marketing department.

In the course of my career I have worked for and on-behalf of many organizations, and have had the opportunity to partner with several IT firms and internal departments. Too often, my experience has been a comedy of disconnected conversations with the business (aka me and people like me) struggling to communicate and ask good questions, and the developers delivering short, binary-like answers that rarely answer the “real” question or completely inform me of the choices or impact of my decisions.

And, then there is Troy.

Troy is a good listener and an active questioner who engages completely in the conversation. He really works to understand the strategy and the detailed business processes that drive his development work. He always seeks to understand the “why!” Troy kindly bridges knowledge gaps, politely educates, and sheds light on what lays in the shadows of the decisions I make on the OMS project. I rely heavily on his insights and humble, guru-like guidance. So, not only is Troy’s contribution to the OMS project very great, but having Troy on our team empowers me (and others) to contribute at a higher level.

I appreciate the planning, technical and development work and long hours Troy has contributed to the progress of the OMS, as well as his can-do attitude and ability to persevere when the task looms so very large. Whatever recognition or reward may come as a result of this nomination, I hope Magenic will reward Troy with two scoops, because Troy is awesome.
- Consultant of the Quarter, Q1 2008

Troy — Thu, 24 Apr 2008 10:22:07 +0000

And, then there is Troy.