Microsoft SharePoint Development and Tractor Pulls

If you don't know what a tractor pull is, you might want to check out Wikipedia. If you're a SharePoint developer, you are probably more familiar with tractor pulls than you might expect.

The basic concept of a tractor pull is that you attach a big weight on wheels to a vehicle and the vehicle starts moving forward. As time progresses the weight being towed shifts in such a way that it gets harder for the vehicle to tow it forward. After a while, the vehicle cannot even move.

So how is that like SharePoint development? If you do all your development in the production environment, then you probably aren't experiencing the "Tractor Pull Effect". There are other effects that you're more likely to encounter (e.g. the "Who deleted all of the Customers Data? Effect", or the "Users keep asking why the system is down Effect").

But if you've carefully set up a Development, Test, and Production environment, you probably already know the "Tractor Pull Effect".

When you need to take your changes that you made in Development and promote them to the Test environment, SharePoint tools only take you so far. Where I work, we've got some "extra special" tools written by a SharePoint genius that automate much of the promotion process. These tools are not perfect or automated (yet).

Even after these tools run, there is much work to be done to the Test environment to make it mirror the Development environment. Web Parts need to be reconnected, connection strings in charts need to be set to point to the Test database servers. Then the site needs to be manually checked to make sure that the promotion process didn't hiccup (which it does on occasion).

Then when you want to deploy from Test to Production a different set of rules apply. The promotion process has a tendency to delete data (which is fine in a Test environment, right?). These different rules add complexity and additional effort because now you're redoing a lot of work in production that you did in Development.

A Short Digression. Maybe every single list, site definition, and page you deploy to SharePoint is done via a SharePoint "feature" so you're not experiencing any of the problems I'm talking about. I've also done that style of development and it appears to be much more clean and reproducible. My frustration there is that if I want to add a link to a navigation area, I have to create a feature folder with two XML files in it and then install and activate that feature. That's a simple task. Creating a custom list took me days the first time I tried it. That is nowhere near agile. I would imagine that most SharePoint shops are somewhere between the do everything in the user interface approach and the do everything via a feature approach... Digression over.

So the more features you add to your product, the more setup, configuration, and rework you end up doing as you promote your work from Development onward.

In a tractor pull, the weight eventually ceases all forward motion. I'm not looking forward to the point where we have so many features, that all of our forward motion ceases. Hopefully before that point, the "extra special" tools will work 100% of the time and they will be automated and I'll have nothing to complain about.

Database Constraints and Usability

I've been playing with Grails and have not been satisfied with the user experience of creating an object that is on the one side of a one-to-many relationship.  You must first create and save the object on the one side of the relationship, then edit it to add objects on the many side of the relationship.  Almost certainly, that is because a relational database cannot save the objects on the many side until the object on the one side has a row ID.

I don't like making the user suffer just to appease the relational model.  So what if I create and save the object on the one side at the moment the user requests to create one.  That way it has its row ID and the user can add objects on the many side.  Unfortunately, database constraints may prevent me from saving my object on the one side of the relationship.

I've read Alan Coopers' book, "About Face" and it has changed the way I view a number of things in the world of software.  It's been a while since I've read the book so maybe this post's topic was covered in "About Face".

The facts:

1. Databases can have constraints to make sure that the data is "valid".  For example, in a customer billing application, a constraint on the customers table might say that all customers must have a mailing address.  This constraint would prevent you from entering a customer in the system that did not have an address.
   
2. Sometimes users don't have all the data and it would be inconvenient to force them to enter all the data or none of it.  Imagine filling out 80 fields in a form and then trying to save the data to the database only to be told that one field cannot be empty.  At that moment, you can either discard all of your work (potentially inconveniencing your customer who is providing the data to you) or entering some made up value into the form (possibly confusing the system or other people who interface with the system as in the case of a made up phone number or street address).

At the moment, I'm considering having a boolean/bit column in some of my tables whose name is 'valid'.  Then whenever a business object tries to persist to a row in a table, it performs validation and sets the 'valid' field to the result of the validation.  All other fields are also saved at that time. 

The user can be notified that the data they entered was invalid and will be ignored by the system until one or more errors are fixed.  If you put a 'saved_by' field in the row that contains the user name of the last person that saved the row, you could periodically ask the user to correct their data.

Most of the queries in the system would only seek rows where the 'valid' field was true.

Database constraints are useful because they actually prevent the data from being invalid (as much as a constraint is able to anyway) and that is their strength.  Removing all constraints and using the 'valid' column does leave the database open to containing invalid rows whose 'valid' field is true, but only if you go around the business objects.

Perhaps database vendors will see this problem and introduce constraints that can allow invalid data to be stored as long as it's marked as invalid.  Maybe then my mail will stop going to 101 Nowhere Lane.

Random Thoughts and Questions

Some random thoughts and questions....

If two objects of the same class have different rules, how did they get those different rules? Was there some object factory that gave them different rules? If other objects modify an object's rules, is that a violation of encapsulation? If an object is the only entity allowed to modify its rules, that means that it must know about all possible rules it can have within the domain. For some domain types (e.g. account types) where there is a closed set of kinds, this might not be a bad idea. For open ended domain types (e.g. widgets) this could not possibly work.

In the case of the Customer class, what would be involved when giving a Customer a discount? We certainly need to specify and store the amount of the discount. But do we also add a rule to the class? Or was the rule there all along waiting for a discount attribute to be set? Since rules may operate on attributes, it's important to keep them in sync. If you have a rule that expects a discount attribute but no discount attribute is in your object's Description Object, your rule will fail or misbehave.

Also it seems like a Description Object might encounter name collision issues. Perhaps there should be namespaces within a Description Object?

Rules vs. Overriding

In "Object Thinking" by David West, it appears that there are two main ways of making two different objects that respond to the same message react differently to those messages.

The first is using polymorphism but that necessitates two different classes. The other is making sure that the two objects will execute different rules when responding to the message.

There is a third way that I'm not going to go over in this post; it's when two different objects have different state and react differently because of their differing state.

The author discusses good and bad specialization on page 79. I can't say that I am sure I know what the author is talking about. If I take his words in their most literal sense, I get the following statements.

1. Any time one class derives from another, it is assumed that the subclass is a specialization of the superclass.

2. Specializing by extension means only adding methods but never overriding methods. This is the good kind of specialization because it preserves the substitutability of subclasses for superclasses.

3. Specializing by constraint (overriding a superclass' method) almost always results in a "bad" object because now you have to be concerned about how an object does something not just what it does.

4. The exception in the footnote is that there will be some methods introduced high in the inheritance hierarchy that are designed to be overridden by subclasses and how the method does what it does is unimportant.

Given that there are two ways of making two different objects react differently to a message, how do you pick between the two ways? Is one way preferred in general?

Let's say I'm doing some Object Discovery and definition of a Customer class. From the domain experts I've found that there are two kinds of customers customers with discounts and customers without discounts. Having a discount makes the payment calculation different but no additional behavior is added (for argument's sake). If a Customer object can be given a discount or their discount can be revoked, then modifying the Customer's rules would be better than recreating the Customer as a CustomerWithDiscount or vice versa. Perhaps the GoF's State Pattern would also be a good solution here.

Let's say that I'm doing some Object Discovery and definition on some classes that represent various shapes in a drawing software package. In this case it seems most fitting to use polymorphism for differing the way that the shapes draw themselves. The reason it fits better sounds a lot like the exception listed in the author's foot note.

Perhaps the reason that rules sound more natural for the Customer object is because it avoids us creating a subclass for every different kind of payment calculation (e.g. CustomerWithDiscountAndGoldMembershipAndOutstandingBalance). In this case the rules approach is superior to even the State Pattern because you'd have to have a separate state class for each different method of payment.

It looks like I need to read up on the author's thoughts on inheritance...

Object Thinking and Description Objects

I'm reading about Description Objects in "Object Thinking" by David West, and I'm having a hard time wrapping my mind around them. Their primary mentions are on pages 125 and 217. The index in the book is a bit thin. I've had to use O'Reilly's Safari book search feature to find things in the book.

At their simplest, an Object Description is a map from labels (strings? atoms? symbols?) to primitives (e.g. string, int, etc.).

Rather than fill up a CRC card with lots of attribute based responsibilities (e.g. know age, know first name, etc.), your object would have a responsibility called describe self. This responsibility is met by returning a Description Object that contains attributes (e.g. age, first name, etc.).

Description Objects create at least one benefit. Let's say that you have a customer class that has the following attributes: age, first name, and last name. Your domain analysis shows that some customers also have discount. In a non-Object Thinking world, you would have two classes: Customer and CustomerWithDiscount. The only need for the CustomerWithDiscount is the fact that it adds a single attribute. In an Object Thinking world however, you would have a single Customer class but some Customers would return a Description Object that had a Discount label in it and others would not.

This raises some questions:

How does one Customer object get a Discount label in its Description Object?

I see that we have two options:

The first is that if it's the Customer object that adds the Discount label that, taken to its somewhat logical conclusion, implies that a Customer object would have to know about all labels that a Customer could possibly have, when they should be added, and what values they should have.

The other option is that some other object adds the Discount label, and that sounds like a breach of encapsulation to me.

Of the two options, I dislike the first one the most (but the second option comes in close behind). The first option implies that potentially a lot of intelligence would be placed in one class (what labels to add in different circumstances). The second option might spread that intelligence all around the place.

Perhaps we can use factory methods to encapsulate the population of the Description object. For example:

Customer createCustomer(age, firstName, ...)
Customer createCustomerWithDiscount(age, firstName, ..., discount, ...)

In this case the intelligence is all in one place and all of the encapsulation violations are done in one place. Perhaps that's not too evil...

If the presence of a label in a Description Object or its value, influences the behavior of an object, how is difference of behavior implemented?

If the Customer has a responsibility to calculate cost and a Customer with a Discount label in their Description Object calculates a different amount that a Customer without the Discount label, where does that decision logic go?

You might say that this is a job for polymorphism but if that is the case, then we need two Customer objects: one with a discount and one without. Hmmm, we introduced the Description Object to reduce the number of classes in the system but now we're back to two classes.

So does this mean that if the presence of a label in a Description Object or its value can influence behavior of the object, that entry doesn't need to be in the Description Object (and probably should not be there)?

I'll have to read up on Rule Objects but perhaps the cost calculation is a job for one or more rule objects (if inheritance and polymorphism are not appropriate).

What happens when you need to change a value in the Object Description?

If the Object Description is implemented as a mutable map and another object just goes to the map and makes a change, this is a definite violation of encapsulation.

The author states that Description Objects might have their own behaviors. If the author considers setting values of entries in the Description Object as one of the Description Object's behaviors, then we have created a new problem. Now we need different kinds of Description Objects for different kinds of customers. One Customer Description Object knows how to set the discount, while others know how to set other attributes.

Perhaps rule objects would come to the rescue again. When you create a Customer with a discount, you attach different rules to the Description Object than when you create other kinds of Customers.

It seems that in Object Thinking rule objects might be used in many places where polymorphic methods would be used in non-Object Thinking.

All this to say, I think I'll reexamine my CRC cards...

Object Discovery on the BIll Payment System

I'm developing a bill payment system based on the principles of "Object Thinking" by David West.

I drew up a semantic net based on the domain and covered almost an entire page. I then started writing out CRC cards for the classes in the system and found that many of the classes I had drawn on the semantic net weren't classes at all. Some became part of an object's description (payment schedule, payment rule, website, user name, password). Several were outside of the system (e.g. website to pay bills on, check to mail the creditor).

Since I'm practicing anthropomorphization, I listed each class' motivation on the CRC card. For example a bill wants to pay itself. A creditor wants to be paid the full amount on time. A fund (that keeps track of how much money I have allocated for bill paying) wants to stay in the black.

I was surprised by how my complex semantic net shrunk down to four objects.

Object Thinking and Confusion over Collaboration

David West, the author of "Object Thinking", has a different definition of collaboration than I had ever seen.

The main difference (found on page 130) is noted in a scenario where object A is collaborating with object B. He states that the object being collaborated with (B) is "not an object occupying one of object A's instance variables, a temporary variable declared in the method that object A is executing in order to satisfy the original request, or an object supplied to object A as an argument".

Then on page 254, the author states, "Collaborating objects are very tightly coupled. For this reason, collaborations should occur inside the encapsulation barrier, with objects occupying the instance variables, objects being received along with messages, and objects occupying temporary variables".

At this point, I was confused, thinking I had found a contradiction. Let's see if other references to collaboration clear things up...

On page 226, the author talks about the airplane collaborating with an instrumentCluster which on page 227 is shown as being held in an instance variable (Figure 8-2). Okay, maybe that first definition just had a typo...

On the previous page, page 225, the author restates the definition of collaboration as found on page 130, the definition where collaborations don't take place between arguments, instance and temporary variables.

If you're keeping score, that's two references to collaborations taking place between arguments, temporary variables, and instance variables and two references for collaborations not being between arguments, and the rest.

My thinking on the matter is that any time one object uses the services of another object, there is collaboration. If collaboration is not occurring between instance variables, temporary variables, and arguments, then what is it called when you use the services of these other objects?

After some more thought, I think we have a catch-22 situation here. When you are filling out a CRC card, and you are about to write down a collaborator, you must first think whether or not that collaborator will be stored in an instance variable, temporary variable, or argument. If the collaborator will be referenced via an instance variable, then it's not a collaborator at all and you should not write it down. That sounds like jumping to implementation details during the "Object Discovery" process.

If collaboration is something that is to be discovered during the "Object Discovery" process, then you cannot make restrictions on what is and what isn't a collaboration based on implementation details.