Rethinking the Evil of XML Configuration Files

Do you remember when your system's configuration grew from a simple file containing a few name/value pairs modified by a small set of people, into a large collection of files in multiple locations with their own business processes surrounding them?

When the config files were small and informal, you could easily get by with a YAML file or a Windows .INI file format. But now that there are hundreds or thousands of settings, maintaining them all has become challenging.

The main problems with YAML and .INI configuration files are:

• Lack of Type Safety - If the configuration reading code expects 'true' or 'false' and the configuration file contains '1', how will the code handle that input? Will it log an error? Will it silently misinterpret the '1' as 'false'? Do you want to have to write type-checking code throughout your application? You could write a comment in the configuration file that specified the type of the setting but not everyone understands all the various types and formats (e.g. dates and times).
• Lack of Range Checking - If the configuration code expects a value between 1 and 4 inclusive, and someone has configured the setting as 5, how will your system react? What if your configuration reading code expects 'high' or 'low' and someone enters 'medium'? That's another form of range violation. You could write comments that specify the range but the comments had better agree with the code that does the actual range checking.
• Lack of Validation Support - If one of your configuration settings is mandatory for the system to operate correctly (e.g. a web service endpoint) and it's missing, you don't discover the error until run-time. You could add a comment to the configuration file that stated that the setting was mandatory, but will people read it?
• Lack of Appropriate Defaults - If some of your configuration values have appropriate defaults that you want to communicate to the user, you are stuck writing comments in the YAML or .INI file that list the defaults. Unfortunately, you have now just introduced duplication between the code that must take the default when the configuration value is missing and the comment in the configuration file.

XML files and, more specifically, XSD files, provide for all of the above.

• XSD allows you to specify the type of a given configuration value. If your configuration value has the wrong type, XSD validation of the configuration file will alert you to your mistake.
• XSD has support for range checking of several types.
• XSD by it's very nature handles validation. If your application does XSD validation upon startup, you can quickly catch configuration errors.
• XSD allows you to specify default values for configuration settings.
• With an XSLT transform you could generate an HTML document that would list all your settings along with their defaults.

In short XSD codifies and enforces all of the constraints that we would otherwise add to our configuration files as comments.

Maybe XML configuration files aren't completely evil. The main complaint I had about XML configuration files was that they were so hard to get right. Isn't that ironic! YAML and .INI files are hard to prove that they're right.

How many of my YAML and .INI files are wrong and I just don't know about it?

NUnit and CruiseControl

One of my goals is to become more skilled with CruiseControl.NET because Continuous Integration plays a big part in Agile Development. Today I took the first step. It wasn't one of those easy, comfortable steps. That's why I'm documenting my results.

Here are some of the steps I took today to get a CruiseControl server running on my computer with integrated NUnit Reports. Some of these steps could definitely be improved upon. I don't like using absolute paths if I don't have to but I broke that rule left and right. This is just the first time I've gotten all this working (and trust me, it took hours).

Install Cruise Control

I installed version 1.5.6804.1 of CruiseControl.NET and after realizing that by default Vista does not enable ASP.NET, it ran quite nicely.

Incorporte NUnit into the Build Process

I wanted to make NUnit run as part of the Visual Studio 2008 build process. I tried to use the Microsft SDC NUnit Task but after hours of tracking down unhandled exceptions in the NUnit task, I decided to switch to Tigris' MSBuild NUnit Task which worked great the first time.

I had to import the tasks in my Unit Test assembly's csproj file:

<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>

And then I added the following to the same csproj file:

<IemGroup>

<TestAssemblies Include="$(TargetPath)"/>

</ItemGroup>

<Target Name="Test">

<NUnit

Assemblies="@(TestAssemblies)"

ToolPath="C:\Program Files (x86)\NUnit 2.5.3\bin\net-2.0" />

</Target>

The last step was to add "Test" to the Project element's DefaultTargets attribute:

<Project ToolsVersion="3.5" DefaultTargets="Build;Test" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

Now, whenever I built the project, the tests would run. If the tests failed, the build would fail too. I had to change the MSBuild verbosity to normal to see the actual results of the unit tests. This has the side-effect of showing too much information. If I figure out how to get the NUnit test results showing without showing too much information, I'll write another blog entry.

Build the Project with CruiseControl

I added the following to my ccnet.config file:

<project name="MyProduct">

<category>Production Code</category>

<tasks>

<msbuild>

<executable>C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\MSBuild.exe</executable>

<workingDirectory>C:\Users\Tim\Projects\CruiseControl\CruiseControlTests</workingDirectory>

<projectFile>CruiseControlTests.csproj</projectFile>

<buildArgs>/noconsolelogger /p:Configuration=Debug /p:Platform=AnyCPU /v:diag</buildArgs>

<targets>Build;Test</targets>

<timeout>900</timeout>

<logger>C:\Program Files (x86)\CruiseControl.NET\server\ThoughtWorks.CruiseControl.MsBuild.dll</logger>

</msbuild>

</tasks>

<publishers>

<merge>

<files>

<file>C:\Users\Tim\Projects\CruiseControl\CruiseControlTests\TestResult.xml</file>

</files>

</merge>

<xmllogger/>

</publishers>

</project>

The merge elements are required so that the output of the unit test run could be reported on.

Getting Reporting To Work

In the dashboard.config file I had to make a few changes too. To make a change take effect, I'd recycle the CruiseControl app pool.

I added the following elements (shown in context):

<buildPlugins>

<buildReportBuildPlugin>

<xslFileNames>

<xslFile>xsl\unittests.xsl</xslFile>

and then I added this right after </buildReportBuildPlugin>:

<xslReportBuildPlugin description="NUnit Details" actionName="NUnitDetailsBuildReport" xslFileName="xsl\tests.xsl" />

<xslReportBuildPlugin description="NUnit Timings" actionName="NUnitTimingsBuildReport" xslFileName="xsl\timing.xsl" />

After recycling the app pool, everything worked great.

Can we drop the S in SOLID? (Or NTP!)

Maybe I just don't get the Single Responsibility Principle (SRP). Maybe it's just named poorly. I like the O, L, I, and D principles plenty, but my stomach feels anything but SOLID when I hear about the SRP.

According to Wikipedia, "[Robert C.] Martin defines a responsibility as a reason to change". I wish Martin had picked another, less useful word to redefine. So now, to even think the acronym SRP, I have to pause mentally before the R, and then mentally replace it with RtC (as in Reason to Change).

Even with this mental substitution, I still have problems with SRP. Is the SRP telling me that every one of my classes should have one and only one reason to change? How can this be, especially if a class has multiple responsibilities (as defined by Webster and Wirfs-Brock, not Martin)?

Here's my problem. In the Business Layer I like classes that resemble their real-world counterparts. For example, the Account object resembles a real account. The SRP principle would tell me that if my Account object had more than one reason to change, I should split it up. So if an Account object can calculate its balance, and perform a withdrawal, those are two places where the Account could change. Both of those responsibilities (think Webster) could change for any number of reasons.

I refuse to break up a domain-specific class just for some arbitrary limit on the number of reasons to change. We expect that change will happen. That's one of the reasons why we refactor our code and write unit tests.

Yay OLID. Boo S.

By the way, NTP from the title stands for No Teal Principles (where I've redefined Teal to mean Useless and Overly Restrictive).

Helping Others for Fun and Profit

Perhaps, like me, you've found yourself saying:

• I want to write a program but I can't think of any real-world problem to solve*.
• I want to be able to show a prospective employer some of my code but my previous employers own all of the code I've written and they would not let me show it to others.
• I want to learn how to gather requirements and more effectively convert them into code but I can't talk with our customers.

I think I've stumbled upon a way of meeting these desires.

1. Find a friend, relative, or charity who has a problem that can be solved by software.
2. Offer to write some software to solve their problem. If they refuse, jump to step 1 with someone else. I would recommend that you offer to do the work pro bono publico.
3. Sign any non-disclosure agreements that might apply but stress that you will own the code you produce. After all, you want to be able to show the code to prospective employers.
4. Sit down with them and gather requirements.
5. Write the software as professionally as you know how in an iterative fashion seeking their input whenever it's needed (while respecting their time and schedules). Keep in mind that one of the goals of writing the software is to maintain the friendship. Buggy code may strain the friendship or even destroy it if the bug is serious enough.
6. Make sure that you protect yourself by releasing yourself from all liability just in case you have a litigious friend.
7. Deliver the software, the source code, and unit tests, and let them know that you'll be available to modify it if their needs change.

If you find that your friend is taking advantage of you, you may have to set some boundaries to protect the friendship.

By the way, the profit I was referring to in the title of this article is not necessarily monetary profit, but it definitely is personal and professional profit.

NOTICE: I am not responsible for any friends you may lose, family members who disown you, or charitable organizations who publicly defame you or sue after you follow these steps.

* Perhaps that's why there are so many ray tracers and sudoku solvers.

Resisting the Urge To Refactor

I'm working on the main Controller (as in the C in MVC) of a GUI application and trying to design the Controller along the lines of "The Humble Dialog" [PDF] by Michael Feathers. Some of the Controller's methods make repeated calls to the View object. For example:

view.clearAuctionEvents();

view.enableUpdateDatabaseButton(false);

view.displayErrorMessage("Could not download email messages.");

My first instinct was to follow the Refactoring advice of Fowler and Beck and move these three calls into a new method on the View. Then it struck me that it's actually desirable for the Controller to be so pedantic with the View. After all, I'm trying to "humiliate" the view as "The Humble Dialog" article puts it by stripping it of any logic or behavior.

So sometimes it's okay not to refactor.

Rapid Web Service Construction with Groovy

I'm reading Thomas Erl's book, "Principles of Service Design" and, when he discusses contracts, they're usually codified in WSDL and XSD. So I've been trying to find a way to quickly throw together Web Services that use WSDL and XSD so that I can play with some of the book's concepts.

Failed Attempts

I thought that Ruby would be the answer. I first tried SOAP4R but found that it could not generate WSDL from service methods and types. Since I don't want to generate WSDL files yet, this was a deal-breaker.

I then tried Ruby's ActionWebService but the generator script was not installed correctly so it wouldn't work.

Success

I then stumbled upon Groovy's WSServer library (installation instructions). Groovy is a dynamic language that runs on the JVM. I installed the Groovy support for Eclipse and had created a fully working web service in minutes. I fired up the Web Service from within Eclipse and then switched over to Microsoft Visual Studio and imported the Service Reference and called my service code from C#. It was so easy.

I then thought I'd try a web service call that would return a complex type (e.g. a Person object). This made the Web Service crash with a stack overflow.

It took me hours to find the solution. The following links helped a lot:

Basic Service - This page showed how to create a simple Web Service.

Using the Aegis Mapping - This showed me that I needed to create a mapping XML element for the Person class that ignored the 'metaClass' property of my Person object. My guess is that the stack overflow was caused by navigating a very deep or even cyclic object graph during the serialization process.

Aegis Binding - This link told me where to put the mapping XML element.

Now I can create Web Services very quickly just to try things out. And they run on any platform Groovy runs.

Have you found a better way to quickly develop Web Services? I'd love to hear about what you've found!

Exploring XML Using Haskell

I am trying to learn how to process XML using Haskell and I haven't found any "Hello World" tutorials on how to just access various nodes and attributes of an XML document using HaXml.

The following is a transcript of a ghci session I had tonight (minus a handful of bloopers).

I'm not a Haskell expert so expect less than stellar Haskell code. My main purpose for writing this is to give others a jumping off point into HaXml.

First I started ghci:

tim@laptop:~$ ghci
GHCi, version 6.10.1: http://www.haskell.org/ghc/ :? for help

Then I declared a String that held my XML document text (formatted to fit your screen):

Prelude> let xmlText = "<?xml version=\"1.0\"?><order>
<part number="\">Hammer</part>
<part number="\">Nail</part>
</order>"

Before I can parse the XML, I'll need to import some of HaXml's libraries. The :m +
command will do that for us:

Prelude> :m +Text.XML.HaXml
Prelude Text.XML.HaXml> :m +Text.XML.HaXml.Parse

Now I can parse the XML and extract just the root of the document:

Prelude Text.XML.HaXml Text.XML.HaXml.Parse> let (Document _ _ root _) = xmlParse "(No Document)" xmlText

Actually, no parsing has taken place yet. xmlParse is lazy and will only parse the XML text when necessary.

The first argument to xmlParse is "(No Document)". That's just a dummy value that's used by HaXml for error reporting purposes. If I had parsed XML from a file, I would have substituted the file name for the dummy value.

Let's see what type the root has:

Prelude Text.XML.HaXml Text.XML.HaXml.Parse> :t root
root :: Element Text.XML.HaXml.Posn.Posn

The combinators I'll be using expect a Content value not an Element. So let's create a Content value from this Element. But first we need to load another module:

Prelude Text.XML.HaXml Text.XML.HaXml.Parse> :m +Text.XML.HaXml.Posn

Having all these modules in the prompt is getting annoying. Let's remove them:

Prelude Text.XML.HaXml Text.XML.HaXml.Parse Text.XML.HaXml.Posn> :set prompt "> "

Now our prompt will just be the > character followed by a space.

And now we can wrap the Element value in a Content value:

> let rootElem = CElem root noPos
> :t rootElem
rootElem :: Content Posn

To select nodes in the XML tree we can use the tag function:

> :t tag
tag :: String -> Content i -> [Content i]

It takes a String and a Content value (our document root) and returns potentially multiple Content values (each node whose tag name matched the supplied String).

Let's see the type of value we get when we supply tag with a String:

> :t tag "order"
tag "order" :: Content i -> [Content i]

No surprise there if you're familiar with currying.

And let's see the type of the value returned from tag when both a String and a Content value are supplied:

> :t tag "order" rootElem
tag "order" rootElem :: [Content Posn]

How many element names matched "order"?

> length $ tag "order" rootElem
1

You can search for tags within tags using the /> function. Notice that the chained functions below have the same type as the tag function:

> :t tag "order" /> tag "part"
tag "order" /> tag "part" :: Content i -> [Content i]

Let's search for "part" tags within the "order" tag and see how many nodes we get (it should be 2 because we have 2 orders in our XML text):

> length $ tag "order" /> tag "part" $ rootElem
2

Let's grab just the first "part" element and examine its type:

> let firstPart = (tag "order" /> tag "part" $ rootElem) !! 0
> :t firstPart
firstPart :: Content Posn

Great. We now have a single XML element in firstPart.

Let's poke around the internals of firstPart. To do that we'll pattern match on firstPart:

> let (CElem (Elem name attributes _) _) = firstPart

Now we can look at the part elements tag name:

> name
"part"

And see how the attributes are stored:

> :t attributes
attributes :: [Attribute]

That makes sense since an element can have multiple attributes.

We know that this node just has one attribute so let's grab it:

> let (attrName, _) = attributes !! 0
> attrName
"number"

The second part of an Attribute is an AttValue which is a list of "Either String Reference" values. I'm not sure why this is. I thought that a single attribute could only have one value. Perhaps not?

Let's grab the first attribute's AttValue:

> let (_, attrValue) = attributes !! 0
> :t attrValue
attrValue :: AttValue

And now we'll grab the list of Either values and store the first one in "firstAttrValue":

> let (AttValue (firstAttrValue:_)) = attrValue
> :t firstAttrValue
firstAttrValue :: Either String Reference

Now we'll try to access the attribute String:

> let (Left value) = firstAttrValue
> value
"101"

Sure enough, the first "part" has a part number of "101".