Bad Idea: Exceptions as a Validation Mechanism Wednesday, January 30, 2008

I have seen exceptions abused a lot in my travels as a software engineer. Exceptions should be used to convey "exceptional" circumstances in code. Exceptions definitely should not be used for first-chance validation purposes; a validation failure (or collection thereof) should be returned to consuming code. The client code can then choose to present these violation in some meaningful manner. Of course, it is okay to throw an exception on second-chance validation prior to performing an operation; but client code would interpret this as a failure. Throwing and catching of exceptions do come at a cost.

Ayende - How Do You Do It? Friday, January 25, 2008

In a previous post, I discussed "how it is I manage to stay current in the software development space with so many responsibilities ". The question I pose now is how do other (well known) bloggers like Ayende manage to post such detailed blog articles, in addition to all the other stuff going on in their lives. If took me six weeks to put together my two posts and PowerPoint (here, here, and here) on software testing. I do know that Raymond Chen (The Old New Thing) write posts ahead of time and stages them for release...

Book Review: "Applying Domain-Driven Design ..." Monday, January 21, 2008

Although it has been out for a while, I just got around to reading Jimmy Nilsson's "Applying Domain-Driven Design and Patterns: With Examples in C# and .NET". Although Nilsson makes a few good points, the book read like a college essay written by a highly caffeinated student. The book was basically a collection or references to other authors' material, and is all over the place. I would recommend BORROWING it but not BUYING it.

Increasing the Business Value of Software Using a Separation of Concerns Approach to Software Testing Friday, January 18, 2008

I recently rolled up a few of my previous blog posts on software quality and testing, added a little more content, and presented an hour long "tech talk" for fellow Software and QA Engineers at my company. I want to share it with the rest of you: http://www.softwareishardwork.com/Inc%20BV%20of%20Sw%20Using%20a%20SoC%20Approach%20to%20ST.ppt

Amen Brother!!! Tuesday, January 15, 2008

I fully understand the emotion and thoughts behind this: http://weblogs.asp.net/fbouma/archive/2008/01/11/the-waterfall-which-makes-agile-pundits-go-blind.aspx Top three points to take to the bank:

• Software development has lost it's connection to computer science.
• ALT.NET is created out of the wrong reasons.
• Zealots make it no longer fun to collaborate and discuss.

Does the Use of NHibernate Lead to Sloppy Coding?

I am curious to see how many of you have used NHibernate on large scale applications and to find out if you believe that the use of NHibernate leads to sloppy coding (or other problems)? I have not formulated an opinion yet, so this is not designed to be a flame war, rather, a discussion with facts and open mindedness. I have extensive NH usage experience on large scale application so do not go there either. Comments?

ALT.NET Official Position - It's Pointless Saturday, January 12, 2008

I was recently asked what my take was on the "ALT.NET movement". David Laribee who coined the phrase ALT.NET proposed ALT.NET signifies:

• The type of developer who uses what works while keeping an eye out for a better way.
• You reach outside the mainstream to adopt the best of any community: Open Source, Agile, Java, Ruby, etc.
• You’re not content with the status quo. Things can always be better expressed, more elegant and simple, more mutable, higher quality, etc.
• You know tools are great, but they only take you so far. It’s the principles and knowledge that really matter. The best tools are those that embed the knowledge and encourage the principles (e.g. Resharper.)

I would contend that these ideals are some of the many traits that distinguishes elite software engineers from mediocrity, thus no need for yet another "movement".

My two cents.

Staying Relevant in the Software Game Thursday, January 10, 2008

A coworker asked me recently how it is I manage to stay current in the software development space with so many responsibilities (work, family, etc.). My answer was: it’s difficult. When you have a family and a full time job, it can become hard to devote time to blogging, research, open source development, training, speaking engagements, user groups, etc. But, my family and I realize that to maintain a certain lifestyle (like paying the mortgage), I have to remain relevant in this industry – so sometimes sacrifices and compromises have to be made.

Software Testing Separation of Concerns Monday, January 7, 2008

Introduction Building quality software requires software testing. Granted, any testing no matter how off-base is better than zero testing. Without a firm grasp of certain testing concepts however, producing a correct and repeatable test suite becomes difficult and you can end up with a half-ass mix of somewhat useful tests. Just as developers (hopefully) strive to provide for the separation of concerns in application code via layering, object oriented programming/design, etc., so must you also be cognizant of and implement a clear-cut separation of concerns when it comes to your software testing strategy. To illustrate this, let’s look at what I call the Levels of Software Testing. Level 0: Syntactical Testing Although many developers would not think of the compiler or interpreter of their choice as a testing process and tool, these systems serve as the start of the testing journey: syntactical testing. If your code does not compile, build, link, or interpret then you have just immediately failed your test suite. One could also argue that code that is poorly documented, structured, or just plain sloppy fails another test – the maintainability test. If the code cannot be easily maintained or refactored, then the code itself becomes a possible surface for the introduction of bugs. On a side note, crashing your compiler also is an immediate test suite failure. Real world example: the code base on a recent consulting engagement written in VB.NET would consistently crash the VB.NET compiler due to its heavy use of generics. The code base was subsequently converted to C#, the compiler crashes became a non-issue, and overall quality increased by orders of magnitude Resharper helped immensely as well. Negative software quality manifests itself in mysterious ways. Level 1: Unit Testing Unit testing is a quite simple but very misunderstood testing process. MSDN, surprisingly, puts it very succinctly: “The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect. Each unit is tested separately before integrating them into modules to test the interfaces between modules. Unit testing has proven its value in that a large percentage of defects are identified during its use. Finding the error (or errors) in the integrated module is much more complicated than first isolating the units, testing each, then integrating them and testing the whole.” [1] In order to test the interfaces (i.e. point of interconnection) between modules, the use of interfaces (i.e. an abstraction of a software component), dependency inversion and injection principles, and mock and/or stub objects are the correct and preferred way to proceed. Leveraging these measures also has the desirable side effect of ensuring separation of concerns in application code as well. Note as well, if your code is not loosely coupled (and highly cohesive), then it will be difficult to do correct unit testing. It is all too unfortunate that many developers still in this day just do not get what a unit test is designed to test and not test. Consider this Wikipedia excerpt that really drives it home: “Because some classes may have references to other classes, testing a class can frequently spill over into testing another class. A common example of this is classes that depend on a database [or other component]: in order to test the class, the tester often writes code that interacts with the database [or other component]. This is a mistake, because a unit test should never go outside of its own class boundary. As a result, the software developer abstracts an interface around the database connection [or other component], and then implements that interface with their own mock object [or using a dynamic mock library]. By abstracting this necessary attachment from the code (temporarily reducing the net effective coupling), the independent unit can be more thoroughly tested than may have been previously achieved. This results in a higher quality unit that is also more maintainable.” [2] Examples of good unit test suites include those which exemplify:

• Are actually unit tests; if you see a test that connects to a live database or leverages an external component then that test is not a unit test; it is an integration test.
• Are automated and are able to execute headless (no one-off WinForms test harnesses) with code coverage enabled. This provides much needed introspection into how well the development team is adhering to the testing strategy.
• Should test the both positive and negative conditions. Failure to test edge cases and boundary conditions is a major cause of bugs which slip into QA and/or production.

Level 2: Integration Testing Integration tests are really what many developers are incorrectly calling “unit tests”. The goal of integration testing is to expose defects in the interfaces and interaction between integrated units. Given a component A which depends on unit B and unit C, an integration test should be created to ensure A, B, and C play nicely in a live situation. There are many ways to perform integration testing but generally the strategies come down to top-down, bottom-up, and functional-driven. A top-down approach entails starting your integration testing with integrated components as high up in the dependency space as feasible and working down until you have a full suite; a bottom-up approach is the opposite in that you start your integration testing with integrated components as low down in the dependency space as feasible and working up until you have a full suite. A functional-driven approach uses key feature code paths as the guiding light in integration testing. Examples of good integration test suites include those which exemplify, as blogged by Jeffrey Palermo:

• “Smaller integration [versus large system tests] tests will help narrow the area where the failure lies.” [3]
• “An integration test must be isolated in setup and teardown. If it requires some data to be in a database, it must put it there.” [3]
• “It must also run fast. If it is slow, build time will suffer, and you will run fewer builds - leading to other problems. “ [3]
• “Integration tests should be order-independent. It should not matter the order you run them. They should all pass.” [3]

Level 3: System Testing

Relying on Wikipedia for a concise definition, system testing is “testing conducted on a complete, integrated system to evaluate the system's compliance with its specified [technical and non-technical] requirements. “ [4] During system testing, certain behaviors, artifacts, and/or aspects of the system are usually tested:

• Capacity, Scalability, Performance, Load, Volume, and Stress
• Security and Error Handling
• Installation, Maintenance and Recovery
• Accessibility, UI, Usability, and Help

System testing is can be done by developers, QA, or become a shared responsibility.

Level 4: Acceptance Testing

Acceptance testing involves performing automated or manual tests by the end-user, customer, QA team, or client to validate whether or not the product meets acceptance criteria. The end result is a decision to accept the product all, some, or none of the product as being feature complete and feature correct.

Conclusion

It is easy to see that the focus of testing starts at the smallest concern (source code) and continually broadens (units, components, system) to the largest concern (customer). It forms a pyramid like paradigm very well suited to structure a successful software testing strategy.

References: [1] http://msdn2.microsoft.com/en-us/library/aa292197(VS.71).aspx [2] http://en.wikipedia.org/wiki/Unit_testing [3] http://codebetter.com/blogs/jeffrey.palermo/archive/2006/05/09/144387.aspx [4] http://en.wikipedia.org/wiki/System_testing

Software Quality is a Must

If the business value of software were expressed as a function, if could be in the form:

Value = f (Cost, Time, Scope, Quality)

This function mimics the classic software development triad of cost, time, and scope; quality being a new dimension to the software development equation. This function produces a four dimensional abstract universe in which points can describe potential business value. For example, a software project with high cost, long time, little scope, and bad quality would be considered to have low, if any, business value; on the other end of the spectrum a software project with low cost, acceptable time, on target scope, and a high level of quality would be considered to have a high business value. Of course, these extremes are for demonstrative value only; we live in the real world where the business value of real software projects vary from one to another, for reasons as unique as the project itself (poor management, poor technical ability, lack of planning, lack of vision, etc.) For a developer, more often than not, they have limited, if any, direct control over the perceived business value of a software project due to the nature of the beast: it is project managers, stakeholders, product owners, and/or executives which generally control the cost, time, and scope inputs to the function above. (Developers though, are often blamed first when software projects go sour.) What a developer CAN control is the quality input – which ironically can actually affect the other three inputs indirectly. Quality software can decrease costs by flushing out bugs in the active coding phase of the development cycle as opposed to the QA phase or even worse – in production. Of course, nothing comes without a catch: quality software can lower the amount of scope you address in a given box of time due to the development of test suites. The mere mention of reducing scope and/or increasing time strikes fear in the hearts of unenlightened project managers/stakeholders/product owners/executives because all they see is that they could sacrificing something. But in the end, the difference between a successful software project and one that falls flat on its ass is generally how well the business value equation is managed – and if quality is allowed to slip in order to eek in more scope in less time at a cheaper cost, then failure becomes self fulfilling or at the very least maintenance costs in production will be high.