A Different Perspective on Web Applications

Notice that I said this perspective is different. I intentionally left out words like: smart, useful, good, and many other positive adjectives.

My goal for taking this perspective is to see if it fits, what problems it solves (or creates), if it makes reasoning about or building web applications any easier. The good thing about perspectives is that you can try them on, look around, and then take them off (or even discard them).

The perspective that I'm taking off for this post is the one that states that a web application extends from the user in front of their browser all the way back to the data layer. That perspective is nice and has been around for a while now. But it has some rough edges that I'm trying to smooth out.

The Rough Edges

1) Data Validation is spread throughout all three layers of a three-tiered web application architecture. I wrote about this here.

2) Presentation Layer activities are performed in two places: in the browser via JavaScript, CSS, HTML, etc. and on the server when the HTML is dynamically generated (e.g. JSP, PHP, etc.).

Perhaps these rough edges cannot be avoided and all my attempts to do so will fail. Maybe I'm the rough edge and I just need to accept that the nature of the web requires that similar things need to happen in different places. Or maybe I can just change the way I look at the problem so that the rough edges disappear.

Here is the different perspective. What if we equate a web application to a standalone desktop application like such:

Selecting the Application to Run

To start a desktop application, you run a program at a certain location (its path) with optional arguments

To start a web application, you enter a certain location (its URL) into your browser with optional arguments.

Application Start Up

When you start a desktop application, one or more resources are loaded from the disk into memory, a user interface is presented, and an event loop is entered.

When you start a web application, one or more resources are loaded from the web server into the browser, a user interface is presented, and an event loop is entered.

Additional Resources

As the desktop application is asked to do different things, it may load additional resources (e.g. user interface elements, XML files, DLLs, etc.) from the disk.

As the web application is asked to do different things, it may load additional
resource (e.g. HTML, JavaScript, CSS, images, etc.) from the web server.

Changing State

The desktop application user wants to save their changes so the application the writes the changes to disk.

The web application user wants to save their changes so the application sends the changes to the web server.


After drawing these parallels, would you say that a Desktop Application extends from the user in front of the screen all the way to the disk? Probably not. Yes, the disk is involved, but that's not part of your application proper.

So could we say that a web application starts at the user in front of the browser and ends inside the browser?

Is that too much of a stretch? Can we really equate the Internet, our web servers, and back end databases all to a storage device? I'm going to try. Maybe the analogy will work, maybe not.

Implications

A number of interesting things happen once we say that a web application is just what is going on inside the browser:

1) There are no more tiers to think about on the browser side (see also Tautology). Perhaps we can simply adopt a Model/View/Controller approach in the browser. It's perfectly okay to have data validation in an MVC application so I don't need to worry about spreading data validation through out all three tiers (yes, I am totally ignoring the web server and database layer for now). The browser is not self-sufficient; just as the desktop application loads resources from disk, the browser application will load resources from the web server.

2) The web server becomes less user-serving and more application-serving. A page served by the web server is not as much a page that the user will see as as it is some user interface resource that is loaded by the application. The difference is subtle but I'm going to note it anyway. Perhaps it's important.

3) The web server gives up much of the control it had once we start putting more focus on the browser as the web application. Many web server applications of old carefully created their HTML pages so that every link and button would initiate the proper request back to the server. If the browser is the web application, the JavaScript code can play a much larger role in deciding what resources to request.

I have really tightened my focus to the browser client and have forgotten all about the web servers, and database servers. In doing so, I've seen some nice properties emerge on the client side. For one, I can reason about the client in isolation. But have I created a terrible mess on the server side? Let's see.

What Does the Server Look Like Now?

As I start this section, I've isolated the browser client and abstracted away the rest of the application. Now I've got to deal with the Application Layer and the Data Layer. I'm starting to wonder if my attempt to gain a different perspective on three-tiered architecture will land me right back at a three-tiered architecture.

Let me do a roll call to see what we have left on the server.

Presentation Layer. Check. Granted it's not really presenting things to the user (as I pointed out earlier).

Application Layer. Check. But couldn't we put all of the Application Layer on the client? Yes we could, but should we. Browsers with good debuggers allow users to modify the DOM and the JavaScript environment. As a safety precaution, we need some code that the user can't tamper with.

Data Layer. Check. We haven't gotten rid of that either.

Conclusion

I tried on a different perspective. At first it seemed promising, and then the three tiers appeared again on the server side.

In addition, the rough edges remain although the presentation rough edge is slightly different as we have two different parties we're presenting to (i.e. the browser application and the user).

I like the way that the client looked in this perspective. The only way to see if it really works though is through using this perspective to develop some web applications.

Objects and Data Validation

I've written a lot of objects that do data validation but I have yet to come up with a standard approach to data validation for objects.

When should the validation occur?

I have usually taken the approach that as soon as something goes wrong, I want to know about it. If you call a setter method passing in invalid data, I want to catch it immediately and reject it by throwing an exception.

Ken Pugh, in his book Prefactoring mentions the use of specific data types. He would say that your setPhoneNumber() method should not take a string but it should take a PhoneNumber object. The PhoneNumber class' constructor would parse whatever string you tried to initialize it with and throw an exception if you didn't pass it a proper phone number. In this way, you can't even pass an invalid phone number to setPhoneNumber().

Delayed Data Validation

There are some cases where delayed data validation is the only way to go. Consider a very contrived but simple class whose "lower" attribute must be the lower case version of its "upper" attribute. If "lower" is 'a' then "upper" must be 'A'. If the class provides only setLower(char c) and setUpper(char c) these methods cannot do data validation. Either the class must supply a setBoth(char upperC, char lowerC) method or there must be a way to delay the data validation.

A less contrived example would be a Location object that has a City and a State property. If the object tries to make sure that the City value is always a city in the State value and that the State value always has a city by the name of the City value, users will find it difficult to use.

Not A City

Let's try something. If you had a special City value (e.g. NaCity resembling NaN for not a number) and a special State value (NaState) and setting the City would always set the State to NaState and vice versa, the validation could be simplified. If the City value is NaCity don't validate it. Hmm... but now you can have locations that don't contain City or State information. That doesn't sound very valid.

Try To Be More Accepting

Let me make things a little more complicated. Alan Cooper in his book "About Face" talks about data validation in the user interface. He thinks that a user should be able to enter incomplete information. Why should you have to discard your work if you don't have every required field filled in? Perhaps data validation should only be done in select situations? For example, I should be able to set attributes in my Customer object to whatever the data types will allow and save that invalid data. But when I want to send out some invoices to my customers, I should only send out invoices to customers that pass data validation. That way, I'm not sending mail to Fooville, or to Mars. This approach would probably require some report or view that showed all customers who were in an invalid state. That way folks can keep an eye on all of the Customers they can't yet bill.

Reporting what is actually wrong might involve a wee challenge. A Customer object might fail validation due to one of the Customer's aggregated objects. It's up to the programmer to make sure that the source of the validation failure is reported correctly.

Taking this on demand approach to validation in the business logic tier and not in the data tier puts you in a bind. Now you've got all of these invalid objects in memory, and you can't save any of them because of some database constraint. The moral of the story is to synchronize your data validation approaches throughout all of the tiers of your application.

Changing My Mind

The more I go on about data validation, the more I like the on demand approach. The nicest thing about it is that it separates data modification from data validation. You're always performing validation on stationary data.

Now when I put on my user hat, I like to know if I've made a mess of something even if I can't fix it at the moment so flagging a Customer as invalid in the user interface is probably a good idea.

There are some situations where this on demand approach to data validation won't work. For example, if we were writing a code generator for Eclipse, and we allowed the user to create a class named "<:^)" we would be doing them a disservice as the code would not compile and there may be many references to "<:^)" that they would need to change.

One Way of Data Validation

I'm sure there's not One True Way of Data Validation. You may have a legacy database that has bullet proof constraints and is guarded by German Shepherd attack stored procedures. You may have an object model that must be 100% valid at all times (e.g. Air Traffic Control systems, SDI, a super safe lethal injection device, etc.).

For future projects, I think that I will start off with the on demand approach and see if that works.

Exercises Left to the Reader

1. Consider mixed models (e.g. strict data validation mixed with an on demand validation approach). Would the mixture of data validation models be too confusing?

In a Three-Tiered Architecture Where Should Data Validation Go?

I have not had much experience with three-tiered architectures so I'm working through some of the basic concepts that are tripping me up.

Right now I'm considering where data validation should take place. That sounds like something that would fit in the business logic tier. But there are benefits of putting some form of data validation in each of the three tiers.

Presentation Tier

When writing a web application, if you can avoid a round trip, you might want to. You might not want to avoid the round trip if:

• the cost (development or throughput) of avoiding the round trip is prohibitive.
• the latency is expected to be low (e.g. intranet applications).

One way of avoiding that round trip is by performing some of the data validation on the client-side.

Most of the data validation I've encountered is very narrowly focused. For example, the user's name is validated separately from the user's age. I don't think I've ever seen cross-field validation (e.g. "A customer with status X must have a account balance below Y."). Does this suggest that the client-side validation should just be a first line of defense against invalid data?

It seems like data validation is at home in the Presentation Tier.

Business Logic Tier

Here is where data validation naturally belongs (right?). You've got a bunch of Customer objects that are being manipulated in memory. When do you want to know that your Customer data is invalid? When you try to save the Customer data to the database? Or would you rather know that the Customer data is invalid the moment you attempt to modify a Customer object in an invalid way? Personally, I like knowing that something went wrong as soon as it goes wrong.

It seems like data validation is at home in the Business Logic Tier too.

Data Tier

What good DBA would ever trust all clients of his/her data to keep that data in a valid state at all times? Enter database constraints designed to maintain the data's validity. The database constraints are the last line of defense for data validation.

It seems like data validation is at home in the Data Tier too.

Questions

The validation performed in the presentation tier is probably different from the validation performed in the data tier and the business logic tier. How does one get a good idea of what valid data is if the validation is spread over three tiers?

It would be useful if I could specify what makes the data valid in one place in the form of multiple validation rules and then for each validation rule specify the tiers where they should be enforced. It seems like this would be a fairly challenging problem to solve without some sort of framework.

How JavaScript Stirs Up The Three-Tier Architecture

A Place for Everything and Everything in it's Place

When it comes to designing web applications I thought I knew where everything should go. I thought that business logic should go in the "logic tier", and that user interface stuff should go in the "presentation tier", and that the data would be stored in the "data tier" (see three-tier Architecture). If I had been asked, "Where does JavaScript fit into all this?", I would have said that it is useful for providing basic client-side validation a la, "Customer name cannot be blank.".

Enter JavaScript

Within in the last six months or so, I've discovered two things that have shaken up my three-tiered perspective.

First, JavaScript (or at least the way it is being used) has matured quite a bit. Now those who have been following JavaScript carefully for a while will be able to calculate how deep my head has been in the sand from this last statement.

Second, a number of powerful JavaScript libraries/toolkits have been produced (e.g. JQuery, Dojo, Script.aculo.us, Prototype, GWT, etc.). I'm sure that some of these libraries have been around for a long time, but I am only discovering them now. And I'm having a great time playing with them.

The Power of JavaScript

These JavaScript libraries go a lot further than just telling the user that the "Customer name cannot be blank." They provide serious event handling, cool effects on top of JavaScript's ability to create objects with methods and properties. With all of these features, you could potentially move all three tiers of the three-tier architecture into the browser client. At least you could if you don't mind the total lack of persistence. If you want persistence, then you do have to communicate with a server (unless you find that storing your data in cookies works for your application).

Where Does Everything Go Now?

So now I'm confused. I like being able to split things up into pieces and then put the pieces somewhere. If I try to split up my web application according to its tiers, I find that JavaScript can play in all three tiers although it's typically only used in the browser. So we have some dissonance between the physical tiers and how client-side JavaScript can be used to augment each of the three tiers:

• JavaScript can create and manipulate objects just like the data tier.
• JavaScript can run business logic on the client side. This can reduce round trips to the server.
• JavaScript can present the data to the user in all sorts of ways.

When JavaScript deals with all three parts of the three-tier architecture some interesting results occur:

• The logical tiers are now split across the physical tiers (the client and the various types of servers).
  
• Code may be duplicated in different places in different languages. Customer validation in the client would be in Java while Customer validation on the business logic tier would be done in Java or C#. Copy/Paste Programming is bad but try to imagine Copy/Translate/Paste Programming.
  
• Thanks to JavaScript debuggers that can be embedded in a browser, users can inspect, the DOM and make changes to it. This has a number of security implications. A JavaScript application can't always trust its own DOM.

Simplifying the Question

Let me back up for a minute. Should JavaScript be used in all three tiers (on the client side)? I think that it's safe to say JavaScript should not generally be used in the client-side data tier. The data tier is all about persistence and JavaScript has some significant limitations in the persistence department. It's very clear that JavaScript is a very natural fit for use in the presentation tier. This leaves only the question of whether or not JavaScript should be used in the logic tier? And if so, how?

I think that JavaScript brings a lot of benefits to the client-side logic tier. Unfortunately it can only do so safely if the server mirrors that business logic and distrusts the client-side logic. Since JavaScript is so dynamic users can tamper with client-side code and make the business logic do anything they want.

Code Generation to the Rescue?

I wonder if code generation is a partial solution to this. The book, "The Pragmatic Programmer" suggested that if code/data does need to be duplicated, then there should be one source representation of that code or data and all duplicates should be generated from it.

The wonderful thing about JavaScript is that even if client-side code and server side code are generated from a single source, they will always be in synch because the client is always downloading the most recent version of the JavaScript.

Conclusion

I'll spend some more time thinking about this and then write up some thoughts. I'd be very interested in hearing how you approach this.

Thanks.

The Free Stuff You Get From {Gr|R}ails

I'm learning Grails (before that it was Rails). Up until recently, I've been really impressed by what these frameworks give you "for free": ORM, pages for performing CRUD operations on the data, one or more application servers, etc.

But as I consider the usability of the software that these frameworks give you for free, I start wondering if some of this free stuff is worth it.

The more practical side of me has been confronting the side of me that was initially so giddy over the power and RAD-ness offered by these frameworks.

Here's one recent exchange between my practical side and my giddy side...

Giddy Tim: So with just a few commands and some editing of a Customer.rb (or .groovy), I can show users a page that lists every Customer in the database!

Practical Tim: What kind of user wants to see every Customer in the database? Sure if you're interested in one of the first Customers in the list, you might be happy with the list but what about every one else? Everyone else has to some how get to the Customer they're interested in. What your users probably want is a search feature so that they can search for a particular Customer. They probably would like a list of the Customers that they deal with the most, you know, their favorites. Or maybe they want to narrow the field by listing all Customers in a particular Category.

Giddy Tim: You've got some good points there. At least I have an edit page that shows the Customer and a place where I can briefly list, but not edit or view the details of, the Customer's Orders.

Practical Tim: So you have to go to another screen to edit or view the Orders?

Giddy Tim: Yes, at least one screen. There's the "View Order" page, then I click the "Edit" link and I'm taken to the "Edit Order" page. Then when I'm done editing, I click "Save" and then I'm redirected to the "View Order" Page again. Then, if I customize the Controller, I can have it redirect me back to the "Edit Customer" Page of the Customer whose Order I wanted to edit. Even though the system knows that there is a one to many relationship between Customers and Orders there is still a one to one relationship when it comes to pages. Each "Edit" page edits a single record in the database. Simple, eh?

Practical Tim: With a powerful JavaScript library and Ajax couldn't you easily allow the user to edit Customers and their Orders all from the same page? That seems like a better user experience to me. Alan Cooper in his book "About Face" says that it's a good idea to allow users to edit data where it's displayed. It sounds like these frameworks create an unnecessary distance between editing and viewing, and part/whole relationships.

Giddy Tim: So to offer the users a good user experience, it sounds like I'll need to modify the automatically generated code extensively.

Practical Tim: Yep, see you around Giddy Tim!

Giddy Tim: Uh, just call me Tim.

I will continue to use Grails, and Rails, and maybe even Merb, but I will use them with open eyes knowing that once the frameworks have generated code for me, I've probably still got a lot of work ahead of me.

P.S. I'm sure that the code generation engines can be improved to generate more usable interfaces (although they may need some hints and guidance from the programmers).

A Very Sad Book Title

"Managing Catastrophic Loss of Sensitive Data"

When I hear this title and think of who would buy the book, I imagine some guy with tear-stained cheeks, who has only recently arrived at the final stage of grief (acceptance), sheepishly approaching the checkout at the bookstore, getting ready to say, “It’s for a friend” to the cashier.

I feel for this guy.

The SharePoint One-Question Personality Test

Do you like developing in SharePoint?

If you answered "yes"...

• You are not bothered by frequently performing repetitive tasks.
• You do not mind expending large amounts of energy in order to receive small rewards.
• You can handle large amounts of cognitive dissonance.
• You like swimming in uncharted, undocumented waters.
• You don't need constant reassurance that your code works (e.g. unit tests).

If you answered "no"...

• You prefer to automate any repetitive tasks.
• You want your energy investment to be close to the expected rewards for that investment.
• You like it when things are logical, orderly and, just make sense.
• You like navigating waters that have been mapped and documented.
• You like being constantly assured that your code works (e.g. continuous integration and unit tests).

Just a theory but I think that these two sets of answers align with two categories of people. Those who answer "yes" seem to fit into the thrill-seekers category. Those who answer "no" seem to fit into the more cautious category.